Repository navigation
pythonTerminalEnvVarActivation shadows pyenv PATH #23227
Description
Activity
hello kbehlers Thanks for filing the issue. Is this only happening with pyenv?? Is it fine with .venv
Anthony Kim (@anthonykim1) Hi! This is only with pyenv, specifically pyenv-virtualenv.
Standard
.venvdoesn't have the issue, and looking into it that is probably because thesource .venv/bin/activatecommand prepends the venv to PATH itself..pyenv-virtualenv seems to be relying on the existing shim already in the PATH, rather than prepending the venv each time, so the shadowing occurs because the VS Code experiment seems to prepend the venv from the interpreter selected in the Python extension.
Exploring the behavior a bit more, if I first
source deactivateinstead ofpyenv deactivatethen that clears the venv from the interpreter selected in the Python extension from PATH and I can use pyenv-virtualenv normally again.Hello kbehlers Thanks much for further looking into this.
I think it can be one or either of two problems:
-
I'm also thinking about the shadowing scenarios of prepending possbily causing shadowing of pyenv adding to existing shim already in PATH. I believe we prepend to PATH all the time. When you take a look at your $PATH after you try to activate pyenv and see pyenv venv prompt, does your PATH contain pyenv related code?
-
My second guess is the way we "deactivate" for pyenv. I will need to further look into deactivate scenario for pyenv. (Im going to try to figure out if python extension does sort of different deactivation for pyenv compared to traditional .venv and if that is correctly working.
Good to hear that there is at least some workaround if you
source deactivatefirst.I'll let you know if I get update on this.
Reacted by kbehlers-
Related: #21906 Seems to be the PR where we started to use prepend to PATH. With its purpose of attempting to avoid replacing. since pyenv asks their user to manipulate PATH in init script.
- changed the title
[-]pythonTerminalEnvVarActivation experiment breaks pyenv-virtualenv[/-][+]pythonTerminalEnvVarActivation shadows pyenv PATH[/+]on Apr 29, 2024 kbehlers can you share what your PATH looks like?
We are trying to figure out which deactivation script is running when yousource deactivatevs.pyenv deactivateAnthony Kim (@anthonykim1) Sure!
Opted out from the experiment
/opt/homebrew/Cellar/pyenv-virtualenv/1.2.1/shims:/Users/username/.pyenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbinOpted into the experiment
/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zsh:/Users/username/.pyenv/versions/3.9.17/envs/first_example_env/bin:/opt/homebrew/Cellar/pyenv-virtualenv/1.2.1/shims:/Users/username/.pyenv/shims:/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zsh:/Users/username/.pyenv/versions/3.9.17/envs/first_example_env/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zsh:/Users/username/.pyenv/versions/3.9.17/envs/first_example_env/binYou can see when opted-in it has some interesting prepend and append behavior
Thanks much for providing PATH. Greatly appreciated!
The prepending is expected as: #21906 and #22905Im not able to repro the behavior on my machine exactly but I do get "pyenv-virtualenv: no virtualenv has been activated." when I try
pyenv deactivatewhen opting into the experiment. But then that still maintains Python from Pyenv virtualenv. Only when I source deactivate, it seems to bring back to "correct" Python.Still lost on why prepending would "not allow" pyenv deactivate to work because your pyenv shim is still technically in your PATH even after Python extension prepend our deactivate script in the PATH.
Anthony Kim (@anthonykim1) Ahh, I see so if I am following #21906 and #22905 to it sounds like
PATH=<activated_full_path><activated_full_path><original_path>might be expected then but I do want to highlight my path is actually ending up as
PATH=<activated_full_path><activated_full_path><original_path><activated_full_path>with the two prepended activated paths and a third appended activated_path. Probably not relevant to the problem, still a little odd to me though.
Going back to why prepending would "not allow" pyenv deactivate to work, I've been digging into this a bit more.
Built-in venv deactivate manipulates the PATH.
pyenv-virtualenv deactivate manipulates some env vars like
VIRTUAL_ENVandPYENV_VIRTUAL_ENVbut importantly it does not manipulate path. You can verify this byecho $PATHbefore and after apyenv activate first_example_envand apyenv deactivate.The way pyenv uses its shims, it expects to do all python routing internally. As with pyenv-virtualenv, pyenv also does not manipulate
PATH, instead as part of the installation instructions you modify.zshrcto add the shim toPATHupfront.It was really helpful for me to understand the behavior by repeating my Steps to Reproduce, but checking
PATH,PYENV_VERSION,VIRTUAL_ENV, andPYENV_VIRTUAL_ENVafter each step.As I understand it this means by having the venv force prepended to the PATH,
pyenv deactivatewill never be able to remove that, only asource deactivatewill. Every time a python command comes in it finds the matching executable in the venv that was force prepended and never gets to the pyenv-vritualenv or pyenv shim so they can do an internal search against their activated and/or known python versions.Out of the two paths appended
/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zsh : /Users/username/.pyenv/versions/3.9.17/envs/first_example_env/binit isn't the
/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zshthat is the problem, it is the/Users/username/.pyenv/versions/3.9.17/envs/first_example_env/bin, and that can be proven by removing only/Users/username/.pyenv/versions/3.9.17/envs/first_example_env/binfrom the path while keeping/Users/username/.vscode/extensions/ms-python.python-2024.4.1/python_files/deactivate/zshin place.Hopefully that is helpful in trying to decide how to adapt the experiment for compatibility with pyenv.
I'm experiencing a related issue. The appended paths that kbehlers pointed out are causing problems with conda environments too. Here's what I observed after selecting
my_conda_envviaPython: Select Interpreter:- While opted out of pythonTerminalEnvVarActivation, open a new terminal session, and echo the PATH:
$ echo $PATH /opt/anaconda3/envs/my_conda_env/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/anaconda3/condabin- Now
conda deactivatethe session. The updated PATH does not contain any references tomy_conda_env, and python is inaccessible, which is expected:
$ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/anaconda3/condabin $ which python python not found- Opt back into pythonTerminalEnvVarActivation, open a new terminal session, and echo the PATH. Note the extra
my_conda_envsecond from last:
$ echo $PATH /opt/anaconda3/envs/my_conda_env/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/anaconda3/envs/my_conda_env/bin:/opt/anaconda3/condabin- Now
conda deactivatethe session. The updated PATH still containsmy_conda_envand python is accessible when it should not be:
$ echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/anaconda3/envs/my_conda_env/bin:/opt/anaconda3/condabin $ which python /opt/anaconda3/envs/my_conda_env/bin/python
As a slight aside, even when opted out, the PATH in VS Code is still a bit different from what I get in a native terminal on my Mac.
- VS Code, conda deactivated:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/anaconda3/condabin - Native, conda deactivated:
/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
Since it's just condabin being reordered, I think it's fine, though a bit odd.
I might be experiencing problems related to this as well. I'm on osx with pyenv + pyenv virtualenv. pyenv/venv works fine in the OS terminal app. In vscode terminal, everything works fine if I disable vscode terminal integrations. If I source ~/.zshrc or ~/.zprofile in the vscode terminal, pyenv starts working again.
The problem I'm seeing is if I open a new terminal with default vscode settings, the wrong pyenv is entered and changing directories does not switch to the correct one.
which pip shows the wrong pyenv pip
pip list shows the wrong list of installed packagesWhen things are not working, 'which pip' shows:
/Users//.pyenv/versions/3.12.4/envs//bin/pipWhen things are working, 'which pip' shows:
/Users//.pyenv/shims/pipIf I source ~/.zshrc or ~/.zprofile in the vscode terminal, pyenv starts working again.
Correct, this is coming from lack of context in env variables. We are working API microsoft/vscode#227467 so extension has better context of env vars inside .zshrc and other shell init files.
Closing this against #24567
We are planning to get rid of this experiment for better experience with environment variables in terminal for Python.
Behaviour
When the pythonTerminalEnvVarActivation experiment is active, I can no longer use pyenv-virtualenv commands for activating and deactivating venvs in the VSCode terminal. Instead the interpreter selected in the Python extension always stays activated in the terminal.
The pyenv-virtualenv commands give the appearance of working (they change my zsh command prompt showing the active venv) but running python commands like
python --versionorpip freezeshow that the python interpreter being used isn't the one shown in my command prompt but instead the one selected using the Python extension in VSCode.It seems possibly related to how pythonTerminalEnvVarActivation prepends to
PATH.Steps to reproduce:
pyenv install 3.10pyenv install 3.9) to make it easier to see when the wrong venv is activefirst_example_envas the python interpreter in VSCode usingPython: Select Interpreterecho $PATH(this shows vscode specific paths when the experiment is enabled)python --version(should say Python 3.9.*)pyenv deactivate(doesn't end up deactivating as it should but gives the appearance of doing so by changing the command prompt)pyenv activate second_example_env(changes the command prompt, doesn't actually change the venv though)python --version(should say Python 3.10.* but still says Python 3.9.* because the environment didn't really switch)Python: Select Interpreterand choosesecond_example_envsecond_example_envwill be active no matter whatIf I instead add:
to settings.json, I can activate and deactivate venvs using pyenv-virtualenv just as I would expect, where the venv actually switches in the terminal.
Diagnostic data
Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython)