Skip to content

Test microvenv #21100

Closed
Closed

Description

Refs: #20905

Complexity: 3

Create Issue


Requirements

You need a linux flavor derived from debian. Check the following before testing:

  1. Your apt package list should not have venv (python3-venv) or pip (python3-pip). You can test this by running python3 -c "import venv" or python3 -c "import pip". Both should fail.
  2. Make sure you have python 3.7 or greater

The intent of this TPI is to make sure we use microvenv (shipped with python extension) as fallback when venv is not present. It should create a simplified version of the environment and should also install pip into the environment. To verify this correctly, you must ensure that the OS you are using does not contain venv or pip pre-installed. For debian based OS these are often installed via apt, and they don't ship with them out of the box.

Verification

  1. Ensure you have python extension pre-release installed.
  2. Open a folder with python file.
  3. With the python that you were going to use to create environment, run these (both should fail) <python-path> -c "import venv" and <python-path> -c "import pip".
  4. Use the create environment command (Python: Create environment)
  5. Select venv, and go through the flow.
  6. Select the python binary which you used above.
  7. After it is done, you should be able to install using pip, from the terminal in VS Code run: .venv/bin/python -m pip install pytest.

Notes

  1. For this TPI, we only care to see if env is created and pip is installed in the env. If you got an error while installing using pip that may not be related. Do report them here and I will let you know.

Activity

  1. added this to the May 2023 milestone on Apr 21, 2023
  2. bpasero commented on Apr 25, 2023

    @bpasero

    Karthik Nadig (@karthiknadig) it would have helped me to explain better how to setup my Linux VM as you want it for testing. While you make it very clear how it should be, I am not sure how to get there.

    I am running Ubuntu 22 and when I run this:

    python3 -c "import venv"
    

    I see no error in the terminal:

    image

    When I try to uninstall python3-venv I see that it is not installed:

    image

    What steps need to be taken to be in an environment that can be used for testing? Would it be easier to share a docker container with testers that is setup exactly how you need it in the future? I think I am often hitting setup issues when I am assigned to Python test plan items, specifically because I never know how to install the correct environment.

  3. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    The intent of the TPI was to use OS images and containers that are available on the web. The reason is there are so many ways to install python and venv into an image that this would not serve the purpose of getting the variety. For example, this is the first time I have actually seen ubuntu image with venv pre-installed, let alone ubuntu without explicit venv installed (which means when the image was created python was built from full source).

    image

    I recommend using a plain debian image to test.

  4. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    Here is what you can do, to set up an image, where python3-venv is not installed but python3 -c "import venv" succeeds:

    1. if venv is installed find it, python3 -c "import venv;print(venv.__file__)" ==> /usr/lib/python3.7/venv/__init__.py
    2. Go to the location, and delete the venv folder: rm -rf /usr/lib/python3.7/venv
    3. Now try the command python3 -c "import venv"
  5. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    If you use a base image with no python installed at all then you can try this:

    1. apt update
    2. apt install python3-minimal
    3. if venv is installed find it, python3 -c "import venv;print(venv.__file__)" ==> /usr/lib/python3.7/venv/__init__.py
    4. Go to the location, and delete the venv folder: rm -rf /usr/lib/python3.7/venv
    5. Now try the command python3 -c "import venv"
  6. cwebster-99 commented on Apr 25, 2023

    @cwebster-99
    Member

    Tested with Ubuntu (where it had venv but not pip) successfully but ran into a bug with Debian

  7. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    If you see error like No module named distutils you will need to install python3-distutils.

  8. cwebster-99 commented on Apr 25, 2023

    @cwebster-99
    Member

    Testing with Debian after installing distutils was successful

  9. 3 remaining items

  10. bpasero commented on Apr 25, 2023

    @bpasero

    I have now tried to install a Debian parallels image which gave me Debian GNU Linux 10.10 and followed your steps to remove the venv manually by the rm command because I really do not want to corrupt my main Ubuntu system that I use for working on VS Code.

    However, I fail there to get Code to install even, because I miss a lot of dependencies, at least reported so by apt.

    Since it is already late today, I yield for someone else to fill in my position. I seem unable to test this.

  11. connor4312 commented on Apr 25, 2023

    @connor4312
    Member

    I had to follow the same steps as Ben to remove venv from the default Python install in Ubuntu 18. However, I then got the following error:

    Details
    2023-04-25 15:20:58.363 [info] Experiment 'pythonPromptNewFormatterExt' is active
    2023-04-25 15:20:58.363 [info] Experiment 'pythonPromptNewToolsExt' is active
    2023-04-25 15:20:58.363 [info] Experiment 'pythonSurveyNotification' is active
    2023-04-25 15:20:58.389 [info] Test server listening.
    2023-04-25 15:20:58.393 [info] Python interpreter path: /usr/bin/python3
    2023-04-25 15:20:58.426 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
    2023-04-25 15:20:58.426 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
    2023-04-25 15:20:58.426 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
    2023-04-25 15:20:58.427 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:20:58.427 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:20:58.427 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:20:59.022 [info] Starting Pylance language server.
    2023-04-25 15:20:59.798 [info] > conda info --json
    2023-04-25 15:20:59.811 [info] Found: /usr/bin/python3 --> /usr/bin/python3.6
    2023-04-25 15:20:59.811 [info] Found: /usr/bin/python3.6 --> /usr/bin/python3.6
    2023-04-25 15:20:59.813 [info] Found: /usr/include/python3.6 --> /usr/include/python3.6m
    2023-04-25 15:20:59.834 [info] > /usr/bin/python3 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:20:59.849 [info] > /usr/include/python3.6 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:20:59.852 [error] Error: Command failed: /usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    /bin/sh: 1: /usr/include/python3.6: Permission denied
    
        at ChildProcess.exithandler (node:child_process:409:12)
        at ChildProcess.emit (node:events:513:28)
        at maybeClose (node:internal/child_process:1093:16)
        at Socket.<anonymous> (node:internal/child_process:451:11)
        at Socket.emit (node:events:513:28)
        at Pipe.<anonymous> (node:net:757:14) {
      code: 126,
      killed: false,
      signal: null,
      cmd: '/usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py'
    }
    2023-04-25 15:21:01.862 [info] > /usr/include/python3.6 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:21:01.863 [error] Error: Command failed: /usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    /bin/sh: 1: /usr/include/python3.6: Permission denied
    
        at ChildProcess.exithandler (node:child_process:409:12)
        at ChildProcess.emit (node:events:513:28)
        at maybeClose (node:internal/child_process:1093:16)
        at Socket.<anonymous> (node:internal/child_process:451:11)
        at Socket.emit (node:events:513:28)
        at Pipe.<anonymous> (node:net:757:14) {
      code: 126,
      killed: false,
      signal: null,
      cmd: '/usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py'
    }
    2023-04-25 15:21:16.920 [info] Running Env creation script:  [
      '/usr/bin/python3',
      '/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py',
      '--git-ignore'
    ]
    2023-04-25 15:21:16.989 [info] Running: /usr/bin/python3 /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_microvenv.py --name .venv
    
    2023-04-25 15:21:17.025 [info] CREATE_MICROVENV.CREATING_MICROVENV
    Running: /usr/bin/python3 /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/lib/python/microvenv.py .venv
    
    2023-04-25 15:21:17.056 [info] CREATE_MICROVENV.CREATED_MICROVENV
    
    2023-04-25 15:21:17.063 [info] CREATED_VENV:/home/connor/Downloads/python env test/.venv/bin/python
    Creating: /home/connor/Downloads/python env test/.venv/.gitignore
    
    2023-04-25 15:21:17.063 [info] CREATE_VENV.DOWNLOADING_PIP
    
    2023-04-25 15:21:17.726 [info] CREATE_VENV.INSTALLING_PIP
    Running: /home/connor/Downloads/python env test/.venv/bin/python /home/connor/Downloads/python env test/.venv/pip.pyz install pip
    
    2023-04-25 15:21:17.745 [info] This version of pip does not support python 3.6 (requires >= 3.7).
    
    2023-04-25 15:21:17.770 [info] Traceback (most recent call last):
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 77, in run_process
        subprocess.run(args, cwd=os.getcwd(), check=True)
      File "/usr/lib/python3.6/subprocess.py", line 438, in run
    
    2023-04-25 15:21:17.771 [info]     output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['/home/connor/Downloads/python env test/.venv/bin/python', '/home/connor/Downloads/python env test/.venv/pip.pyz', 'install', 'pip']' returned non-zero exit status 1.
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 227, in <module>
        main(sys.argv[1:])
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 215, in main
        install_pip(args.name)
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 151, in install_pip
        "CREATE_VENV.INSTALL_PIP_FAILED",
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 79, in run_process
        raise VenvError(error_message)
    __main__.VenvError: CREATE_VENV.INSTALL_PIP_FAILED
    
    2023-04-25 15:21:17.781 [error] Error while running venv creation script:  CREATE_VENV.INSTALL_PIP_FAILED
    2023-04-25 15:21:17.781 [error] CREATE_VENV.INSTALL_PIP_FAILED
    

    Looked like my Python version was too old. This could probably be presented better, but no problem, I installed Python 3.11 from the deadsnakes PPA, and then followed the same instructions to remove venv from its install.

    I opened a new workspace folder just to make sure nothing bad was cached, but then hit this error after selecting the newly installed Python interpreter. It looks it's trying to run Python 3.6, but (a) I did not select Python 3.6 as my interpreter, and (b) it's trying to "run" /usr/include/python3.6, which is a directory (and doesn't have +x permissions, hence the error)

    Details
    2023-04-25 15:31:59.382 [info] Experiment 'pythonPromptNewFormatterExt' is active
    2023-04-25 15:31:59.382 [info] Experiment 'pythonPromptNewToolsExt' is active
    2023-04-25 15:31:59.382 [info] Experiment 'pythonSurveyNotification' is active
    2023-04-25 15:31:59.383 [info] Test server listening.
    2023-04-25 15:31:59.387 [info] Python interpreter path: /usr/bin/python3
    2023-04-25 15:31:59.411 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
    2023-04-25 15:31:59.411 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
    2023-04-25 15:31:59.411 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:31:59.411 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:31:59.411 [warning] Retry failed, storage update failed for key PYTHON_WORKSPACE_STORAGE_KEYS
    2023-04-25 15:31:59.966 [info] Starting Pylance language server.
    2023-04-25 15:32:00.224 [info] > conda info --json
    2023-04-25 15:32:00.240 [info] Found: /usr/bin/python3 --> /usr/bin/python3.6
    2023-04-25 15:32:00.240 [info] Found: /usr/bin/python3.11 --> /usr/bin/python3.11
    2023-04-25 15:32:00.240 [info] Found: /usr/bin/python3.6 --> /usr/bin/python3.6
    2023-04-25 15:32:00.242 [info] Found: /usr/include/python3.6 --> /usr/include/python3.6m
    2023-04-25 15:32:00.265 [info] > /usr/bin/python3 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:32:00.278 [info] > /usr/bin/python3.11 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:32:00.301 [info] > /usr/include/python3.6 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:32:00.304 [error] Error: Command failed: /usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    /bin/sh: 1: /usr/include/python3.6: Permission denied
    
        at ChildProcess.exithandler (node:child_process:409:12)
        at ChildProcess.emit (node:events:513:28)
        at maybeClose (node:internal/child_process:1093:16)
        at Socket.<anonymous> (node:internal/child_process:451:11)
        at Socket.emit (node:events:513:28)
        at Pipe.<anonymous> (node:net:757:14) {
      code: 126,
      killed: false,
      signal: null,
      cmd: '/usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py'
    }
    2023-04-25 15:32:01.270 [info] Running Env creation script:  [
      '/usr/bin/python3.11',
      '/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py',
      '--git-ignore'
    ]
    2023-04-25 15:32:01.324 [info] Running: /usr/bin/python3.11 -m venv --without-pip .venv
    
    2023-04-25 15:32:01.335 [info] Traceback (most recent call last):
      File "<frozen runpy>", line 198, in _run_module_as_main
      File "<frozen runpy>", line 88, in _run_code
      File "/usr/lib/python3.11/venv/__main__.py", line 2, in <module>
        from . import main
    ImportError: 
    2023-04-25 15:32:01.336 [info] cannot import name 'main' from 'venv' (unknown location)
    
    2023-04-25 15:32:01.338 [info] Traceback (most recent call last):
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 77, in run_process
        subprocess.run(args, cwd=os.getcwd(), check=True)
    
    2023-04-25 15:32:01.338 [info]   File "/usr/lib/python3.11/subprocess.py", line 571, in run
        raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['/usr/bin/python3.11', '-m', 'venv', '--without-pip', '.venv']' returned non-zero exit status 1.
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 227, in <module>
        main(sys.argv[1:])
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 191, in main
    
    2023-04-25 15:32:01.340 [info]     run_process(
      File "/home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/create_venv.py", line 79, in run_process
        raise VenvError(error_message)
    VenvError: CREATE_VENV.VENV_FAILED_CREATION
    
    2023-04-25 15:32:01.358 [error] Error while running venv creation script:  CREATE_VENV.VENV_FAILED_CREATION
    2023-04-25 15:32:01.358 [error] CREATE_VENV.VENV_FAILED_CREATION
    2023-04-25 15:32:02.313 [info] > /usr/include/python3.6 -I ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py ~/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    2023-04-25 15:32:02.317 [error] Error: Command failed: /usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py
    /bin/sh: 1: /usr/include/python3.6: Permission denied
    
        at ChildProcess.exithandler (node:child_process:409:12)
        at ChildProcess.emit (node:events:513:28)
        at maybeClose (node:internal/child_process:1093:16)
        at Socket.<anonymous> (node:internal/child_process:451:11)
        at Socket.emit (node:events:513:28)
        at Pipe.<anonymous> (node:net:757:14) {
      code: 126,
      killed: false,
      signal: null,
      cmd: '/usr/include/python3.6 -I /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/get_output_via_markers.py /home/connor/.vscode-insiders/extensions/ms-python.python-2023.7.11141008/pythonFiles/interpreterInfo.py'
    }
    2023-04-25 15:32:03.440 [info] Python interpreter path: /usr/bin/python3.11
    

    Unfortunately I could not try on my more modern Ubuntu VM due to microsoft/vscode#180843

    Maybe someone else will have better luck, unassigning myself and putting this up for verification again

  12. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    Python 3.6 is not supported. Can you install python3.9 or 3.10? you can do apt install python3.9-minimal

  13. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    Connor Peet (@connor4312) The next best is to use WSL Ubuntu

  14. roblourens commented on Apr 25, 2023

    @roblourens
    Member

    I got a devcontainer into the right starting state by manually deleting the venv module, but I get this error when running "Create Environment", does this mean it's not falling back on microvenv?

    Experiment 'pythonPromptNewFormatterExt' is active
    Experiment 'pythonPromptNewToolsExt' is active
    Experiment 'pythonSurveyNotification' is active
    Python interpreter path: /usr/bin/python3
    > conda info --json
    > /usr/bin/python3 -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.6.1/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.6.1/pythonFiles/interpreterInfo.py
    Starting Pylance language server.
    Running Env creation script:  [
      '/usr/bin/python3',
      '/home/node/.vscode-server-insiders/extensions/ms-python.python-2023.6.1/pythonFiles/create_venv.py',
      '--git-ignore'
    ]
    Traceback (most recent call last):
      File "/home/node/.vscode-server-insiders/extensions/ms-python.python-2023.6.1/pythonFiles/create_venv.py", line 167, in <module>
    
        main(sys.argv[1:])
      File "/home/node/.vscode-server-insiders/extensions/ms-python.python-2023.6.1/pythonFiles/create_venv.py", line 134, in main
        raise VenvError("CREATE_VENV.VENV_NOT_FOUND")
    __main__.VenvError: CREATE_VENV.VENV_NOT_FOUND
    
    [ERROR 2023-3-25 22:59:32.384]: Error while running venv creation script:  CREATE_VENV.VENV_NOT_FOUND
    [ERROR 2023-3-25 22:59:32.385]: CREATE_VENV.VENV_NOT_FOUND
    
  15. karthiknadig commented on Apr 25, 2023

    @karthiknadig
    MemberAuthor

    roblourens 2023.6.1 is the stable version of the extension. you need 2023.7.*.

  16. roblourens commented on Apr 25, 2023

    @roblourens
    Member

    Oh weird, I thought it installed pre-release by default on Insiders, guess not

    Edit- Actually I opened an issue for this scenario microsoft/vscode#180893

  17. roblourens commented on Apr 25, 2023

    @roblourens
    Member

    It worked, thanks

  18. connor4312 commented on Apr 26, 2023

    @connor4312
    Member

    Karthik Nadig (@karthiknadig) I did install and pick Python 3.11 in my second log, but it seems like the Python extension still tried doing stuff with Python 3.6 (or maybe those errors are extraeous)

  19. karthiknadig commented on Apr 26, 2023

    @karthiknadig
    MemberAuthor

    Connor Peet (@connor4312) Python extension will try to detect all versions of python installed, and that can fail, but that is expected and should not impact working with 3.11

  20. locked as resolved and limited conversation to collaborators on May 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions