Skip to content

Tests are discovered correctly but then an incorrect argument is supplied to PyTest, leading test-result discovery to fail (Git Submodules?) #17903

Closed

Description

@Mithrandir2k18

Environment data

  • VS Code version:
    Version: 1.61.2
    Commit: 6cba118ac49a1b88332f312a8f67186f7f3c1643
    Date: 2021-10-19T14:58:13.605Z
    Electron: 13.5.1
    Chrome: 91.0.4472.164
    Node.js: 14.16.0
    V8: 9.1.269.39-electron.0
    OS: Linux x64 5.4.0-90-generic snap
  • Extension version (available under the Extensions sidebar): v2021.10.1365161279
  • OS and version: Ubuntu 20.04
  • Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): None
  • Relevant/affected Python packages and their versions: PyTest 6.0.0
  • Relevant/affected Python-related VS Code extensions and their versions: VSCode Native Test Explorer
  • Value of the python.languageServer setting: Pylance

Expected behaviour

Given the following repo structure:

/work/repos/my_repo
|
-src/
    |
    - main.py
 - dependencies/
      |
     -my_git_submodule/
            - tests/
                   |
                   - test_other_main.py
           ...
-tests/
    |
    - conftest.py
    - test_sometests.py
    - integration_tests/
            |
            - test_integration_main.py

Discovered tests in workspace are run and their results displayed when doing CTRL+SHIFT+P: Test: Run All Tests.
Expect to run the command using the supplied path to the test folder.

Actual behaviour

Pytest fails to run, using the following command:

cwd: /work/repos/my_repo

Causing the error: ERROR - ModuleNotFoundError: No module named 'tests.integration_tests'
But it runs correctly when only running the tests in the tests folder and that can be achieved by clicking "Run All" on the "tests" subgroup:

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347RaAiZqepCFUO.xml ./tests
cwd: /work/repos/my_repo

This works without any issues, however this is not run when doing the VSCode Run All tests function and I have not been able to configure this working to command to be used.

Another way to get it working is ignoring the submodules in dependencies(they should be unit-tested using their own repos workflows) using --ignore-glob=dependencies/* in python.testing.pytestArgs. However I don't see why not manually ignoring submodules should break test(result) discovery.

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347jcCkQFFhe9c0.xml --ignore-glob=dependencies/* ./tests
cwd: /work/repos/my_repo

So the first command fails while the two other commands work. I am not sure if it is intended for the first command to fail.
When using pytest from the terminal, all that's required to get it to run is a simple pytest.ini like this:

[pytest]
testpaths = 
    tests

and the simply doing pytest with the repo as cwd, no ignoring the dependencies folder required.

Steps to reproduce:

Replicate above repo structure with Pytest files in /tests and in /tests/integration_tests and include another repo as a submodule that also contains its own tests.

Activity

  1. added
    bugIssue identified by VS Code Team member as probable bug
    triage-neededNeeds assignment to the proper sub-team
    on Nov 2, 2021
  2. karthiknadig commented on Nov 2, 2021

    @karthiknadig
    Member

    This will be resolved with #17242

  3. locked as resolved and limited conversation to collaborators on Dec 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

area-testingbugIssue identified by VS Code Team member as probable bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions