python.testing.cwd ignored when discovering or running tests

Environment data

VS Code version: 1.40.1
Extension version (available under the Extensions sidebar): 2019.11.49689
OS and version: MacOS Catalina 10.15.1
Python version (& distribution if applicable, e.g. Anaconda): 3.7.4
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): virtualenv
Relevant/affected Python packages and their versions: pytest==5.3.0
Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): Jedi

Expected behaviour

Tests are displayed in the test explorer according to python.testing.cwd setting and I am able to run single test or tests from single file

Actual behaviour

python.testing.cwd is ignored for displaying tests and I am unable to run single test or tests from single file because I am getting an error file not found.

Steps to reproduce:

[NOTE: Self-contained, minimal reproducing code samples are extremely helpful and will expedite addressing your issue]

Sample project: https://github.com/pbotsman/sample_project

Open project
Open tests/py/some_test.py to start exploring tests
Try to execute single test test_something or tests in some_test.py file.
Get the error ERROR: file not found: ./tests/py/some_test.py::test_something
(Sometimes in order to reproduce the problem I had to relaunch my VSCode)

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> ~/.virtualenvs/sample_project/bin/python -m pytest --rootdir ~/projects/sample_project --junitxml=/var/folders/0_/6w94lk1171vc_fpwbjqx02nw0000gn/T/tmp-4746yxdv7Lw3muwB.xml -v -s --log-cli-level INFO ./tests/py/some_test.py::test_something
cwd: ~/projects/sample_project/tests/py

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

console.ts:137 [Extension Host] Info Python Extension: 2019-11-20 14:06:31: getActivatedEnvironmentVariables, Class name = b, completed in 1ms, Arg 1: <Uri:/Users/pavel/projects/sample_project>, Arg 2: undefined, Arg 3: undefined
console.ts:137 [Extension Host] Info Python Extension: 2019-11-20 14:06:31: > ~/.virtualenvs/sample_project/bin/python -m pytest --rootdir ~/projects/sample_project --junitxml=/var/folders/0_/6w94lk1171vc_fpwbjqx02nw0000gn/T/tmp-5598WI4lpp4kFEsC.xml -v -s --log-cli-level INFO ./tests/py/some_test.py::test_something
console.ts:137 [Extension Host] Info Python Extension: 2019-11-20 14:06:31: cwd: ~/projects/sample_project/tests/py

Additional info

Looks like something has changed in the last release:

Previously tests would be displayed in the test explorer just as some_test.py would be root test. I mean that tests/py would be omitted in the test explorer since I have "python.testing.cwd": "tests/py" in my settings.
Currently directories tests/py are displayed in the test explorer even though I have this setting.
I can't run single test because I am always getting an error file not found. It looks like something is wrong with the file paths.
When the command
~/.virtualenvs/sample_project/bin/python -m pytest --rootdir ~/projects/sample_project --junitxml=/var/folders/0_/6w94lk1171vc_fpwbjqx02nw0000gn/T/tmp-5598WI4lpp4kFEsC.xml -v -s --log-cli-level INFO ./tests/py/some_test.py::test_something
is called my current working directory is set to python.testing.cwd and that is why I am getting the error.
So in the command mentioned above valid file path should be ./some_test.py::test_something, not ./tests/py/some_test.py::test_something.

Thanks for the bug report! We just wanted to quickly acknowledge we received it and we will triage this as soon as we can.

Hi @pbotsman 👋 Thank you for reaching out and thoroughly filling out the issue template 👍

I can confirm that I could reproduce your issue in the latest release of the extension (2019.11.49689), and that it doesn't happen in the previous release (2019.10.44104).

Seems like we don't take python.testing.cwd into account anymore when discovering tests, or running single tests using pytest.

A possible workaround while waiting for a fix would be to move the content of python.testing.cwd in the list of pytest arguments, let me know if the following works for you:

{
    "python.pythonPath": "/path/to/python",
    "python.testing.pytestArgs": [        
        "./tests/py",
        "-v",
        "-s",
        "--log-cli-level", "INFO",
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true
}

Karthik Nadig (@karthiknadig)

Karthik Nadig (@karthiknadig) great to see that you took over.

Merged a fix for this just now to main. Fix is essentially what #8678 (comment) said. If there is --rootdir in python.testing.pytestArgs, we won't override it. so for the sample repo, the settings change would be:

{
    "python.testing.pytestArgs": [
        "-v",
        "-s",
        "--log-cli-level",
        "INFO",
        "--rootdir",
        "tests/py"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}

That should work:

Try this out with this build https://github.com/microsoft/vscode-python/suites/3858667759/artifacts/95863817 if that doesn't work please provide repro steps.

Works for me

Had to restart VS Code in order tests to appear in the sidebar without being nested inside tests directory

(Apologies if this is a rediculously silly question...)
Can I install the aformentioned vsix extension on my day-to-day working VSCode instance without risk of completely breakign it without any way back?!

Can it be uninstalled once installed when this fix reaches launch?

Dan Streeter (@danstreeter)

Dan Streeter (@danstreeter) You can install this and uninstall to restore your VS Code to stable extension or any extension version.

Here is how you would do it:

Install the VSIX using Extension tab > ... > Install from vsix... option from the "..." menu.
You may have to Reload VS Code (command pallet Reload Window command). Test out to see if this works for you.
To revert back, Go to extensions tab, find the python extension and Uninstall it.
Reload VS Code (command pallet Reload Window command), go to extensions tab and install from market place. This should get you back to stable build of the extension.

Note, you can also click on the 'Gear' icon next to the extension in the extensions tab to install a specific version using "Install Another version..."

If you want to try this out is complete isolation, you could install VS Code Insiders (which can install side by side with your stable one) and install this vsix there and test it out. then uninstall the whole thing. This way it won't touch your day-to-day VS Code instance.

Karthik Nadig (@karthiknadig)

Thanks Karthik Nadig (@karthiknadig) , I've got it installed in my main 'day-to-day' instance however still not quite working for me...

Using the default provided config after 'configuring tests', I end up with the following in my settings file:

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "tests"
    ],

This matches my project layout, as all tests are stored in the ./tests directory with all actual code in ./src

Running my tests with my normal workflow still works fine by running make test which actually runs:

export PYTHONPATH=./src
pipenv run python -m pytest tests

When performing discovery with the above config, I get the following in the Python output:

~/path-to-code/.venv/bin/python3.8 ~/.vscode/extensions/ms-python.python-2021.10.1267871984-dev/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir  -s --cache-clear tests
Error 2021-09-24 19:51:20: Error discovering pytest tests:
 [r [Error]: ============================= test session starts ==============================
platform darwin -- Python 3.8.5, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /
plugins: asyncio-0.15.1, html-3.1.1, metadata-1.11.0
collected 0 items

I also get the following warning:

PytestCacheWarning: could not create cache path /.pytest_cache/v/cache/stepwise

Also, if I add any --rootdir to the config as follows:

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "my-root-dir-fake-name",
        "tests"
    ],

The error change as 'expected' with the root directory and trys to target a path on root:

~/path-to-code/.venv/bin/python3.8 ~/.vscode/extensions/ms-python.python-2021.10.1267871984-dev/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear --rootdir my-root-dir-fake-name my-root-dir-fake-name
Error 2021-09-24 19:53:03: Error discovering pytest tests:
 [r [Error]: ERROR: Directory '/my-root-dir-fake-name' not found. Check your '--rootdir' option.

Why is it trying to target root?

OOH OOH OOH - I FIXED IT!!!

I needed an absolute path in the rootdir... Using my pseudo, sanitised example above - this now works!

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "/full/path/to/my/path-to-code",
        "tests"
    ],

--- Update ---

This is much nicer:

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "${workspaceFolder}",
        "tests"
    ],

Dan Streeter (@danstreeter)

Dan Streeter (@danstreeter) can you try:

"python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "./tests"
    ],

Doing the above gives:

> ~/path-to-code/.venv/bin/python3.8 ~/.vscode/extensions/ms-python.python-2021.10.1267871984-dev/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear --rootdir ./tests ./tests
Error 2021-09-24 20:00:44: Error discovering pytest tests:
 [r [Error]: ERROR: Directory '/tests' not found. Check your '--rootdir' option.

Dan Streeter (@danstreeter)

Dan Streeter (@danstreeter) Are you missing the '.' before the './tests'? Just checking because '/tests' will not work.

Definately there:

The config in the last part of my previous post up a bit (#8678 (comment)) works perfect though.

I see. Thanks for the input. This should work for most case with this:

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--rootdir",
        "${workspaceFolder}",
        "tests"
    ],

I will tweak it, it seems like for some reason, the cwd gets set to `` (empty) string instead of . which seems to be the problem here. But that is unrelated to this.

Navigation Menu

Repository navigation

python.testing.cwd ignored when discovering or running tests #8678

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

Additional info

29 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

python.testing.cwd ignored when discovering or running tests #8678

Description

Environment data

Expected behaviour

Actual behaviour

Steps to reproduce:

Logs

Additional info

Activity

brettcannon commented on Nov 21, 2019

kimadeline commented on Nov 22, 2019

29 remaining items

twsl commented on Sep 21, 2021

karthiknadig commented on Sep 24, 2021

botsman commented on Sep 24, 2021

danstreeter commented on Sep 24, 2021

karthiknadig commented on Sep 24, 2021

danstreeter commented on Sep 24, 2021

danstreeter commented on Sep 24, 2021

karthiknadig commented on Sep 24, 2021

danstreeter commented on Sep 24, 2021

karthiknadig commented on Sep 24, 2021

danstreeter commented on Sep 24, 2021

karthiknadig commented on Sep 24, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions