Repository navigation
PYTEST_ADDOPTS env var not respected for test debug configuration with debugpy #22820
Description
Type: Bug
Behaviour
Expected vs. Actual
When using pytest-cov options are present in settings.json python.testing.pytestArgs, I would expect to be able to disable pytest-cov during test debugging when using the documenations suggested workaround of setting "env": {"PYTEST_ADDOPTS": "--no-cov" } in the test configuration with purpose "debug-test". This still works when specifying "type": "python" in the launch configuration, but when specifying "type": "debugpy" as suggested by the deprication warning, it does not.
Actual behaviour when "type": "debugpy" and "env": {"PYTEST_ADDOPTS": "--no-cov" } is set in launch configuration and "--cov" is included in python.testing.pytestArgs in settings.json, the test debugger does not stop on breakpoints. If I remove "--cov" form python.testing.pytestArgs or set "type": "python" in the launch config, breakpoints work as expected. It appears that the environment variables are being properly set in the test environment.
Steps to reproduce:
- add the follwing to settings.json "python.testing.pytestArgs": ["--cov", "tests/"]
- set the following test debug configuration in launch.json
{
"name": "Python: Debug Tests",
// "type": "python", // Works when type is set to python
"type": "debugpy",
"request": "launch",
"program": "${file}",
"purpose": [
"debug-test"
],
"env": {
"PYTEST_ADDOPTS": "--no-cov"
},
"console": "integratedTerminal",
"justMyCode": true
}, - set a breakpoint in a pytest function
- run the test function via the debug panel, debug indiviual test button
- observe that breakpoints are not stopped at
- remove --cov from settings.json
- run the test function via the debug panel, debug indiviual test button
- observe that breakpoints work as expected
- modify launch.json config to use "type": python
- add --cov back to settings.json pytest args
- run the test function via the debug panel, debug indiviual test button
- observer that breakpoints work as expected
Diagnostic data
- Python version (& distribution if applicable, e.g. Anaconda): 3.11.4
- Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
- Value of the
python.languageServersetting: Default
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)
XXX
User Settings
languageServer: "Pylance"
testing
• pytestArgs: "<placeholder>"
• pytestEnabled: true
experiments
• optInto: ["pythonTestAdapter"]
Extension version: 2023.25.10292213
VS Code version: Code - Insiders 1.86.0-insider (05047486b6df5eb8d44b2ecd70ea3bdf775fd937, 2024-01-31T03:29:26.801Z)
OS version: Linux x64 6.5.0-15-generic snap
Modes:
System Info
| Item | Value |
|---|---|
| CPUs | 13th Gen Intel(R) Core(TM) i7-13700 (24 x 5100) |
| GPU Status | 2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off |
| Load (avg) | 0, 0, 0 |
| Memory (System) | 30.97GB (20.27GB free) |
| Process Argv | --no-sandbox --force-user-env --unity-launch --crash-reporter-id 3aa60ae4-37a9-4e56-8adf-187f0d7e643f |
| Screen Reader | no |
| VM | 0% |
| DESKTOP_SESSION | ubuntu |
| XDG_CURRENT_DESKTOP | Unity |
| XDG_SESSION_DESKTOP | ubuntu |
| XDG_SESSION_TYPE | x11 |
A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vstes516:30244333
vscod805cf:30301675
vsaa593:30376534
py29gd2263:30784851
c4g48928:30535728
962ge761:30841072
pythongtdpath:30726887
welcomedialog:30812478
pythonidxpt:30768918
pythonnoceb:30776497
asynctok:30898717
dsvsc013:30777762
dsvsc014:30777825
dsvsc015:30821418
pythontestfixt:30866404
pythonregdiag2:30926734
pyreplss1:30879911
pythonmypyd1:30859725
pythoncet0:30859736
pythontbext0:30879054
accentitlementst:30870582
dsvsc016:30879898
dsvsc017:30880771
dsvsc018:30880772
d34g3935:30953408
b5d27386:30943795
fegfb526:30952798
7j2b6412:30951517
bg6jg535:30946824
ccp1r6:30950905
Activity
eleanorjboyd commentedon Mar 27, 2024
Hello! Could you include your logs from the
pythonoutput channel? For your logs, can you first set your log level to trace via theDeveloper: set log levelcommand in the command palette? I am specifically looking to see if we can find where the command is called to debug and see the args passed there.Paula (@paulacamargo25) any other thoughts / suggestions on what to try to debug this?
alenzo-arch commentedon Apr 11, 2024
eleanorjboyd commentedon Apr 15, 2024
Hi! So debugging and coverage don't work together (more on that here #19322). Seems like you are adding coverage in the test args then attempting to remove it in the debug launch config environment? So it seems like the pytest arg is being respected and not the env var correct? I think this is how pytest does precedence and therefore would be the expected outcome.
alenzo-arch commentedon Apr 16, 2024
Hi Eleanor Boyd (@eleanorjboyd) so long story short, it appears this has been resolved in a recent update. I have modified my lauch.json to use debugpy again and I am able to get it to stop at breakpoints using the env var
I'm now on
Python extension: v2024.5.11021008 (pre-release)
Version: 1.89.0-insider
Commit: a2194e8
Date: 2024-04-16T05:50:26.167Z
Electron: 28.2.8
ElectronBuildId: 27744544
Chromium: 120.0.6099.291
Node.js: 18.18.2
V8: 12.0.267.19-electron.0
OS: Linux x64 6.5.0-27-generic snap
Using launch config:
{ "name": "Python: Debug Tests", "type": "debugpy", // This used to not work // "type": "python", "request": "launch", "program": "${file}", "purpose": [ "debug-test" ], "env": { "PYTEST_ADDOPTS": "--no-cov", }, "console": "integratedTerminal", "justMyCode": true },And settings.json
I can't comment on how pytest does precedence of args vs env vars. Quick search didn't make it clear for me.
However, the VSCode documentation recommends the setup that I have (using the PYTEST_ADDOPTS env var) to disable coverage during debugging but allow coverage when running with the built in test runner. I had been using this for quite a while with
From VSCode docs:
eleanorjboyd commentedon Apr 25, 2024
Hi alenzo-arch, glad this is working! Generally the rule should be Command-line arguments take precedence over environment variables so if you run into a situation where that is not the case please let me know and I can look into it further.
We do know that coverage and debugging doesn't work so we remove coverage args from the user args when calling debugging so it is likely having to do with that since we had an issue a few weeks ago with the arg parsing not working as desired.
Generally this problem will be helped by our proposal to allow custom arguments per run type so hopefully we will get working on that soon and that could be useful to you. Thanks!