Repository navigation
Unittest discovery silently fails when Pytest is not installed #21688
Description
Activity
realized this is likely the fix: #21726
- added a commit that references this issue
on Aug 1, 2023 I ran into this as well, so if you were expecting it to be fixed, I don't think it is.
I was also curious why pytest is used to do discovery for
unittest? It just knows about everything?pytest should not be needed if you are just using
unittest, will investigate and put in a fix!roblourens, testing this and unable to reproduce the behavior. Could you send over the test file you had loaded and the settings you picked (when configuring testing so it might look something like below)
{ "python.testing.unittestArgs": ["-v", "-s", ".", "-p", "*test*.py"], "python.testing.pytestEnabled": false, "python.testing.unittestEnabled": true }Because we have not heard back with the information we requested, we are closing this issue for now. If you are able to provide the info later on, then we will be happy to re-open this issue to pick up where we left off.
Happy Coding!
- locked as resolved and limited conversation to collaborators
on Nov 12, 2023 Repro steps
- My potentially irrelevant user settings:
"python.experiments.optInto": [ "pythonTestAdapter" ], "python.terminal.executeInFileDir": true, "python.createEnvironment.trigger": "prompt",
- Navigate to a directory on Windows Terminal (pwsh profile)
- Run the following lines
mkdir example-python-tests cd example-python-tests ni sample.py ni test_sample.py mkdir .vscode ni .vscode/settings.json code-insiders .-
VS Code Insiders should be open now. Create a new venv Python environment and confirm that pytest isn't installed in the venv
-
Copy the following to sample.py
def xyz(): return 0
- Copy the following to test_sample.py
import unittest class TestScratch(unittest.TestCase): def test_add(self): pass
- Copy the following to .vscode/settings.json
{ "python.testing.unittestArgs": [ "-v", "-s", ".", "-p", "test_*.py" ], "python.testing.pytestEnabled": false, "python.testing.unittestEnabled": true }- Click on the beaker icon in the activity bar to go to the tests view
- 🐛 pytest Discovery error is shown
hm tried these steps and still not seeing it, will keep my eye out as I have seen this before but also couldn't repro it a second time.
From: #22833 filed by Luciana Abud (@luabud) with repo steps
Type: Bug
Behaviour
When trying to set up test discovery with unittest, I can only get pytest to trigger discovery to trigger. Even though I have the following settings in my workspace:
"python.testing.unittestArgs": [ "-v", "-s", "./tests", "-p", "test_*.py" ], "python.testing.pytestEnabled": false, "python.testing.unittestEnabled": trueSteps to reproduce:
- Create a new empty folder
- Add a file called e.g. test_1.py with the following content:
import unittest class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertEqual("hello".upper(), "HELLO") def test_isupper(self): self.assertTrue("HELLO".isupper()) self.assertFalse("Hello".isupper()) def test_split(self): s = "hello world" self.assertEqual(s.split(), ["hello", "world"]) # check that s.split fails when the separator is not a string with self.assertRaises(TypeError): s.split(2)
- Click on the test explorer, select configure tests, select "unittest" and then "." and then "test_*.py"
- See that it will fail pytest discovery
Diagnostic data
- Python version (& distribution if applicable, e.g. Anaconda): 3.12.0
- Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
- Value of the
python.languageServersetting: Pylance
Output for
Pythonin theOutputpanel (View→Output, change the drop-down the upper-right of theOutputpanel toPython)024-02-01 18:09:54.242 [debug] Testing: Manually triggered test refresh 2024-02-01 18:09:54.242 [debug] Testing: Clearing all discovered tests 2024-02-01 18:09:54.242 [debug] Testing: Forcing test data refresh 2024-02-01 18:09:54.242 [debug] Testing: Refreshing all test data 2024-02-01 18:09:54.242 [debug] Found cached env for REDACTED\AppData\Local\Programs\Python\Python312\python.exe 2024-02-01 18:09:54.244 [info] Discover tests for workspace name: sample - uri: REDACTED\Apps\sample 2024-02-01 18:09:54.244 [info] Running discovery for unittest using the new test adapter. 2024-02-01 18:09:54.245 [info] All environment variables set for pytest discovery for workspace REDACTED\Apps\sample: REDACTED 2024-02-01 18:09:54.245 [debug] Found cached env for REDACTED\AppData\Local\Programs\Python\Python312\python.exe 2024-02-01 18:09:54.263 [debug] Activation Commands received undefined for shell cmd, resource REDACTED\Apps\sample and interpreter REDACTED\AppData\Local\Programs\Python\Python312\python.exe 2024-02-01 18:09:54.279 [debug] Running pytest discovery with command: -m pytest -p vscode_pytest --collect-only for workspace REDACTED\Apps\sample. 2024-02-01 18:09:54.280 [info] > ~\AppData\Local\Programs\Python\Python312\python.exe -m pytest -p vscode_pytest --collect-only 2024-02-01 18:09:54.280 [info] cwd: . 2024-02-01 18:09:54.353 [error] REDACTED\AppData\Local\Programs\Python\Python312\python.exe: No module named pytest 2024-02-01 18:09:54.363 [error] Subprocess exited unsuccessfully with exit code 1 and signal null on workspace REDACTED\Apps\sample. 2024-02-01 18:09:54.363 [error] Subprocess exited unsuccessfully with exit code 1 and signal null on workspace REDACTED\Apps\sample. Creating and sending error discovery payload 2024-02-01 18:09:54.363 [error] pytest test discovery error for workspace: REDACTED\Apps\sample The python test process was terminated before it could exit on its own, the process errored with: Code: 1, Signal: null for workspace REDACTED\Apps\sample 2024-02-01 18:09:54.363 [info] ResultResolver EOT received for discovery. 2024-02-01 18:09:54.363 [debug] deferredTill EOT resolved for REDACTED\Apps\sample 2024-02-01 18:09:54.363 [info] Disposing data receiver for REDACTED\Apps\sample and deleting UUID; pytest discovery.User Settings
venvFolders: "<placeholder>" languageServer: "Pylance" testing • unittestArgs: "<placeholder>" • unittestEnabled: true experiments • optInto: ["pythonTestAdapter","regionCommentDiagnostics"]Extension version: 2024.0.0
VS Code version: Code - Insiders 1.87.0-insider (442c133, 2024-02-01T09:49:33.086Z)
OS version: Windows_NT x64 10.0.22621
Modes:System Info
Item Value CPUs 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz (8 x 2995) GPU Status 2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabledLoad (avg) undefined Memory (System) 31.71GB (12.58GB free) Process Argv --log trace --log ms-python.autopep8=debug --crash-reporter-id 11494669-52ca-4f2c-aa0e-29172189cc8e Screen Reader no VM 0% A/B Experiments
vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vsc_aa:30263845 vscod805:30301674 vsaa593:30376534 py29gd2263:30784851 c4g48928:30535728 2i9eh265:30646982 962ge761:30841072 pythongtdpath:30726887 welcomedialog:30812478 pythonidxpt:30768918 pythonnoceb:30776497 asynctok:30898717 dsvsc013:30777762 dsvsc014:30777825 dsvsc015:30821418 pythontestfixt:30866404 pythonregdiag2:30926734 pyreplss1:30879911 pythonmypyd1:30859725 pythoncet0:30859736 pythontbext0:30879054 accentitlementst:30870582 dsvsc016:30879898 dsvsc017:30880771 dsvsc018:30880772 8082a590:30953407 edj9j583:30943796 fegfb526:30952798 7j2b6412:30951517 bg6jg535:30946824Luciana Abud (@luabud) tried the steps and got success with no reference to the pytest error. Are you able to repro this if you try multiple times or is it flaky for you too?
seems to be flaky too, I am unable to repro this today
Eleanor Boyd (@eleanorjboyd) ah just reproduced it again!
funny enough by downgrading to the previous release version I can't repro (could be a coincidence I guess!)
same settings:
{ "python.testing.unittestArgs": [ "-v", "-s", "./tests", "-p", "*test*.py" ], "python.testing.pytestEnabled": false, "python.testing.unittestEnabled": true }one thing I noticed though is that whenever I can't reproduce it, it's because the tests had been discovered before successfully. So for example, I clone a repo for the first time, I configure the test features to use unittest all using v2024.0.0, I can repro the issue. But then when I downgrade to v2023.22,1, tests are discovered. Then I upgrade to 2024.0.0, retrigger test discovery, it continues to work. Could be a big coincidence, but this it was consistently the case for me.
When I open repos that had been configured to use unittest before, it simply works. It can really just reproduce it when configuring repos that I had never configured before.never mind, must be a coincidence cause I just repro'ed it using the old version (which makes sense, since all the reports here use older versions):
found the issue and am working on a solution:
issue: when you activate the python extension, it checks if unittest enabled, if false then just sets the adapter to pytest. This means then if someone sets their testing type to unittest after activation the adapter is still the pytest adapter and so pytest is run.
solution: instead of creating adapters on activation, they should instead be created dynamically when the call for either run or discovery comes in. Then there is no chance of the wrong one being pre-selected.error is now surfaced explaining you need to reload as there is a misalignment- this is sufficient given the frequency it shows up
Testing #13301