Repository navigation
Test classes no longer work with pytest 6.2.5 #22457
Description
Behaviour
Expected vs. Actual
This bug occurs with pytest version 6.2.5. All tests should be visible in the Test Explorer, with classes (e.g. class TestWidget) appearing as nested levels in the hierarchy.
However, if a test class is present (or multiple test classes), all other tests are hidden except for the tests in that class.
Steps to reproduce:
- Define a test file
test_example.pywith the following text:def test_one(): pass class TestExample: def test_two(): pass
- Run test discovery in the Test Explorer.
- Only test_two is visible.
- If you delete
TestExample, then test_one becomes visible again.
Diagnostic data
- Extension version: v2023.21.13131007
- Python version (& distribution if applicable, e.g. Anaconda): 3.9.12
- Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
- Value of the
python.languageServersetting: Pylance
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
<Module test_example.py>
<Function test_outer>
2023-11-09 15:40:25.183 [info]
<Class TestClass>
<Function test_inner>
User Settings
languageServer: "Pylance"
testing
• pytestArgs: "-vv"
• pytestEnabled: true
Activity
TomeSq commentedon Nov 13, 2023
I had the same symptoms.
I reverted back to v2023.8.0 and now I can detect it correctly.
karthiknadig commentedon Nov 16, 2023
You can opt out of the
pytestadapter experiment, if that is causing this issue. You should be able to see which experiment you are in in Output > Python logs.karthiknadig commentedon Nov 16, 2023
This seems to be working for me, with the experimental test adapter:
Would it be possible for you to share both logs and settings?
matthew-mcallister commentedon Nov 18, 2023
Sorry, I figured out the problem. The plugin no longer works with pytest 6.2.5. Updating to the latest version fixed. Updating the issue.
[-]Bug: pytest classes suppress discovery in Test Explorer[/-][+]Classes no longer work with pytest 6.2.5[/+]eleanorjboyd commentedon Nov 27, 2023
Hi Matthew McAllister (@matthew-mcallister), to clarify does it now work with the extension after updating or does this problem still exist? Thanks
4 remaining items
matthew-mcallister commentedon Nov 27, 2023
Yes, the issue is that the plugin is incompatible with pytest 6.2.5; this is a breaking change to the plugin. What to do about that is up to the maintainers to decide. Either the extension should continue to work with older versions of pytest to maintain compatibility, or there should be a version check and an error report letting the user know that their version of pytest is not compatible. That way users do not need to grapple with a mysterious bug.
eleanorjboyd commentedon Dec 18, 2023
Karthik Nadig (@karthiknadig) what are you thoughts on the best way to check what version of pytest a user is on and provide an error message as a result?
eleanorjboyd commentedon Dec 19, 2023
seems like pytest 6.2.5 is still supported and that only versions below 4.6 are no longer supported. This means I should support pytest 6.2.5. Will investigate how to do so
blubberdiblub commentedon Jan 27, 2024
FWIW, there are three ways (that I'm aware of) to find out what pytest version is installed. In order of preference, best first, they are:
Run the
pytestcommand line tool found on thePATHfor executables or, if a specific pytest is configured in the settings, run that. Either way, run it with the--versionoption and the output will bepytest 7.4.4or whatever the current version is. Capture that, parse the version and compare against what minimum version you require.If you already have the possibility to run Python code, it's possible to import the pytest module and get the version from that. Execute
import pytestand look atpytest.version_tupleorfrom pytest import version_tupleand look atversion_tuple(or any other way you come up with that is equivalent to one of these 2). The value is a tuple containing the major, minor and micro version numbers as integers, so you can easily compare it against a minimally required version.If for some reason you can neither run an executable nor run Python code, there's still the possibility to go through all possible Python import paths and look for a directory that matches
pytest-*.dist-info, where the part that matches the asterisk is the version (or alternatively read it from theVersion:entry in thepytest-*.dist-info/METADATAfile, but the former method should be simpler). The problem with this technique is that apart from a possiblePYTHONPATHenv variable that may or may not be set, you are unable to reliably determine what the set of possible Python import paths is without executing Python and would have to resort to guessing / using a predefined list. Therefore avoid this technique like the plague and go for one of the first two unless you absolutely have no other choice.eleanorjboyd commentedon Dec 13, 2024
Hello! Just as an update here I discussed with the pytest developers: pytest-dev/pytest#12253 and agree with their assessment that pytest 7.0.0 should be the lowest version we will support in the extension at this time. I will update the docs to reflect that, thanks
eleanorjboyd commentedon Jan 9, 2025
We have now added a minimum version for pytest for the extension which is > 6.2.5. Thanks