Unittest framework : Automatic test discovery not working if test has a relative import

This is with Python Language Server version 0.3.43.0 on Windows 10, Anaconda python 3.7.3.

Have a folder with the following structure:

mypkg
  --> foo.py
  --> __init__.py
mypkg_test
  --> __init__.py
  --> base.py
  --> test_foo.py

The files base.py, and both __init__.py are empty.

Here is foo.py:

def goo():
    return 5

Here is test_foo.py:

import unittest
from . import base
from mypkg.foo import goo


class FooTest2(unittest.TestCase):
    def test_foo(self):
        x = goo()
        self.assertTrue(5 == x)
        
if __name__ == '__main__':
    unittest.main()

Here is settings.json in the .vscode folder in that directory:

{
    "python.testing.unittestArgs": [
        "-v",
        "-s",
        "./mypkg_test",
        "-p",
        "test*.py"
    ],
    "python.testing.pytestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": true
}

If you comment out from . import base in test_foo.py, then the test is discovered. Commenting and uncommenting that line makes the test discovered versus not discovered, as evidenced by seeing the buttons Run Test|Debug Test in the editor.

If you rename mypkg_test to test, update the test settings to point to the newly named directory, and leave the source with the line from . import base present, then the test is discovered.

So the name of the test folder seems to affect test discovery when doing a relative import inside one of the tests.

We have a big source code with this naming structure (and I don't control the naming) and base.py has a lot more stuff in it, and we can run tests just fine from the command line, but not from VS Code.

If you rename mypkg_test to test, update the test settings to point to the newly named directory, and leave the source with the line from . import base present, then the test is discovered.

I am not able to confirm this. For me, tests are still not discovered if I rename mypkg_test to test. But having a relative import breaking discovery is still a bug.

As a workaround, I would ask you to try the pytest framework instead of unittest. Use Python: Configure Tests from command palette to configure pytest, I can confirm it works.

settings.json would look something like this

{
    "python.testing.pytestEnabled": true,
    "python.testing.nosetestsEnabled": false,
    "python.testing.unittestEnabled": false,
    "python.testing.pytestArgs": [
        "<Test_directory_name>"
    ]
}

Kartik Raj (@karrtikr)

Kartik Raj (@karrtikr) We are working with an existing large code base that already uses unittest, so switching to pytest really isn't an option. Hopefully, this bug will get fixed

Tests work fine from command line, python -m unittest discover -v -s ./transform_text/ -p "test_*.py" -t . but with the same settings in vscode it doesn't discover anything.

For those still running into this issue, here's an explanation from what I've found.

In the original issue description, the start directory is set to the test directory. That means that unittest uses the inside of that directory as the starting location. By using from . import ..., the Python file is trying to access the test folder itself, but since unittest started inside the folder, Python doesn't know about the folder itself. To resolve this, unittest provides a top level directory option that redirects Python to a different directory when starting relative import resolution. For the original issue, that would be -t ., the project's root directory.

The extension parses unittestArgs and extracts the start directory and the test pattern. Every other argument is completely ignored (when running unit test discovery). The typescript code that handles discovery would need to add the top level directory argument when parsing the args. Additionally, <ms-python.python>/pythonFiles/testing_tools/unittest_discovery.py would need to be able to accept that top level directory argument, since that file is responsible for invoking unittest's loader discovery.

I am trying to see if I can fix it on my fork of the repo. I will create a PR if I am successful.

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

Thanks!

Eleanor Boyd (@eleanorjboyd)

Hello! We have just finished our testing rewrite and are beginning the roll out to users. If you are able, it would be very helpful to know if your issue still exists on the rewrite! To use the rewrite yourself just add ”python.experiments.optInto": ["pythonTestAdapter"] to your user settings. You can confirm you have the rewrite enabled by setting your log level to trace, via the Developer: Set Log Level command in the command palette. Then check to see if Experiment 'pythonTestAdapter' is active is in your python logs.

Thanks!
Eleanor Boyd (@eleanorjboyd)

I don't see this as being fixed. The output in the python window is showing an error, and in the "testing" panel, it says "Unittest Discovery Error". Python output shows:

2023-11-21 10:25:30.092 [info] Discover tests for workspace name: vscode_utbug2 - uri: c:\Code\Misc\vscode_utbug2
2023-11-21 10:25:30.092 [info] Running discovery for unittest using the new test adapter.
2023-11-21 10:25:30.100 [info] Discovering unittest tests with arguments: c:\Users\irv\.vscode\extensions\ms-python.python-2023.20.0\pythonFiles\unittestadapter\discovery.py,--udiscovery,-v,-s,./mypkg_test,-p,test_*.py

2023-11-21 10:25:30.101 [info] > conda run -n base310 --no-capture-output python ~\.vscode\extensions\ms-python.python-2023.20.0\pythonFiles\get_output_via_markers.py ~\.vscode\extensions\ms-python.python-2023.20.0\pythonFiles\unittestadapter\discovery.py --udiscovery -v -s ./mypkg_test -p test_*.py
2023-11-21 10:25:30.101 [info] cwd: .
2023-11-21 10:25:31.802 [info] >>>PYTHON-EXEC-OUTPUT

2023-11-21 10:25:31.879 [info] Test server connected to a client.
2023-11-21 10:25:31.893 [error] Unittest test discovery error 
 Failed to import test module: test_foo
Traceback (most recent call last):
  File "C:\Anaconda3\envs\base310\lib\unittest\loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "C:\Anaconda3\envs\base310\lib\unittest\loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "c:\Code\Misc\vscode_utbug2\mypkg_test\test_foo.py", line 2, in <module>
    from . import base
ImportError: attempted relative import with no known parent package

2023-11-21 10:25:31.894 [info] Test server connected to a client.
2023-11-21 10:25:31.894 [info] <<<PYTHON-EXEC-OUTPUT

2023-11-21 10:25:31.894 [info] ResultResolver EOT received for discovery.

With that setting, if I uncomment from . import base, then the tests are discovered.

Irv Lustig (@Dr-Irv)

Irv Lustig (@Dr-Irv), this might help: 9a5363c. It was just merged in today therefore you could test it on insiders when the new release comes out tomorrow or next week when stable comes out. Thanks!

Irv Lustig (@Dr-Irv)

Irv Lustig (@Dr-Irv), this might help: 9a5363c. It was just merged in today therefore you could test it on insiders when the new release comes out tomorrow or next week when stable comes out. Thanks!

If I opt into the ”python.experiments.optInto": ["pythonTestAdapter"] as you described, the little icon for test discovery disappears in VS Code.

I'm not sure which extension (pylance? python?) I should look at to determine if I have the right version installed or not.

Irv Lustig (@Dr-Irv)

Irv Lustig (@Dr-Irv), it is the python extension. You should check to see if the extension is enabled (sometimes i have to just open a python file in the editor then it will enable), you can see the python interpreter selected in the bottom right of the screen.

You can also try using the command palette to look for run test / discover test there.

Let me know if that brings the icon back!

Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.

Happy Coding!

Navigation Menu

Repository navigation

Unittest framework : Automatic test discovery not working if test has a relative import #6881

19 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Unittest framework : Automatic test discovery not working if test has a relative import #6881

Description

Activity

MikhailArkhipov commented on Aug 7, 2019

karrtikr commented on Aug 13, 2019

Dr-Irv commented on Aug 14, 2019

19 remaining items

2bdkid commented on Jul 21, 2021

kkaja123 commented on May 30, 2022

eleanorjboyd commented on Oct 30, 2023

Dr-Irv commented on Nov 21, 2023

eleanorjboyd commented on Nov 27, 2023

Dr-Irv commented on Dec 6, 2023

eleanorjboyd commented on Dec 6, 2023

github-actions commented on Jan 6, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions