Debugger doesn't stop at breakpoints with pytest if pytest-cov is used

Environment data

VS Code version: 1.19.3
Python Extension version: 2018.1.0
Python Version: 3.6.2
OS and version: Windows 10 latest update

Actual behavior

My initial setup has this setup.cfg file:

[tool:pytest]
addopts = --cov=mymodulename

This adds automatically a "--cov" to every "pytest" call, but prevents VSCode to stop at breakpoints. If I remove the file (or at least the --cov), I get back my breakpoints stop.
I changed my setup to put that in my Travis file instead, but I feel like this should work :/

Expected behavior

I should be able to have this file (and coverage) and debug at the same time.

Steps to reproduce:

Install pytest and pytest-cov
Create a setup.cfg as describe before
Create even a simple test with a breakpoint inside

Note that the test is executed correctly. Even the "run test"/"debug test" button on each test executes correctly this individual test.

Laurent Mazuel (@lmazuel)

Laurent Mazuel (@lmazuel) apologies for the delay.
Looks like its not possible to debug the code with coverage enabled (http://pytest-cov.readthedocs.io/en/latest/debuggers.html).
I guess we'll have to disable 'coverage' on the fly when debugging (using the flag "--no-cov").

Hmm, looks like we're not the only ones having trouble with debugging + pytest + coverage (https://www.jetbrains.com/help/pycharm/run-debug-configuration-py-test.html)

I've tried passing in the flag --no-cov, but that causes pytest to fall over of pytest-cov isnt installed.

I guess we could try to determine if Pytest-cov is installed or not.

Laurent Mazuel (@lmazuel)

Laurent Mazuel (@lmazuel)
Unfortunately we're not going to be able to resolve this easily, hence we're closing this.
My suggestion is to modify the pytest args in the settings.json to manually pass in the --no-cov flag to enable debugging.
I understand this isn't the best solution, as you'll be modifying the settings on and off.

Eric Hansen (@ericchansen)

Eric Hansen (@ericchansen) seems to me that you are specifying launch configuration in settings.json. I ran into the same issue as you. What fixed it is realizing that the docs specify that customized configurations are parsed from the launch.json file in the .vscode folder of the current workspace. I hope moving your configuration out of settings.json into launch.json will fix this issue for you as well!

On a side note, it would be nice if these launch settings could be parsed from settings.json since it is cumbersome to add a launch.json into every Python project individually.

San Kilkis (@skilkis)

San Kilkis (@skilkis) could you include an example of a launch.json that resolves the issue for you?

Sterling G. Baird (@sgbaird)

Sterling G. Baird (@sgbaird) here is the launch.json that resolved the issue for me. I've hidden it from view to not clutter the debug configuration menu 😊

{
  "version": "0.2.0",
  "configurations": [
    {
      // Disable cov to allow breakpoints when launched from VS Code Python
      "name": "Python: Debug Tests",
      "type": "python",
      "request": "launch",
      "program": "${file}",
      "purpose": ["debug-test"],
      "console": "internalConsole",
      "justMyCode": false,
      "presentation": {
        "hidden": true
      },
      "env": {
        "PYTEST_ADDOPTS": "--no-cov"
      }
    }
  ]
}

EDIT: Original fix was provided by Alexander Wells (@AlexanderWells-diamond) in the answer above

San Kilkis (@skilkis)

San Kilkis (@skilkis) I use multi-root workspaces. launch.json isn't part of that workflow. The launch section of the workspace file should be treated the same as a launch.json (and if it's not being treated the same, that's a bug).

I can confirm that debugging in 1.68.0-insider doesn't work with this workspace (see below). You can see that all of the relevant settings in the launch section of this multi-root workspace are the same as San Kilkis (@skilkis)'s settings.

{
    "folders": [
        {
            "path": "../../../repos/repo-a"
        },
        {
            "path": "../../../repos/repo-b"
        }
    ],
    "launch": {
        "compounds": [],
        "configurations": [
            {
                "console": "integratedTerminal",
                "env": {
                    "PYTEST_ADDOPTS": "--no-cov"
                },
                "justMyCode": false,
                "name": "Python: Debug",
                "purpose": [
                    "debug-test"
                ],
                "request": "launch",
                "type": "python"
            },
            {
                "console": "integratedTerminal",
                "env": {
                    "PYTEST_ADDOPTS": "--no-cov"
                },
                "name": "Python: Current File",
                "program": "${file}",
                "request": "launch",
                "type": "python"
            },
            {
                "console": "integratedTerminal",
                "cwd": "${fileWorkspaceFolder}",
                "env": {
                    "PYTEST_ADDOPTS": "--no-cov"
                },
                "name": "Python: Current File in Current Workspace",
                "program": "${file}",
                "request": "launch",
                "type": "python"
            }
        ],
        "version": "0.2.0"
    },
    "settings": {
        "debug.internalConsoleOptions": "openOnFirstSessionStart",
        "files.trimTrailingWhitespace": true,
        "python.defaultInterpreterPath": "~/virtualenvs/dev/bin/python3",
        "python.testing.autoTestDiscoverOnSaveEnabled": true,
        "python.testing.pytestEnabled": true,
        "python.testing.unittestArgs": [
            "-v",
            "-s",
            ".",
            "-p",
            "*test*.py",
            "-f"
        ],
        "python.testing.unittestEnabled": true,
        "terminal.integrated.profiles.linux": {
            "bash": {
                "args": [
                    "-l"
                ],
                "path": "/bin/bash"
            }
        }
    }
}

The only way to make the debugger work is to remove addopts = --cov repo-a ... from your pyproject.toml, setup.cfg, etc.

If it works in a launch.json as San Kilkis (@skilkis) claims, but not in a workspace file, that's a bug.

Eric Hansen (@ericchansen)

Eric Hansen (@ericchansen) I've created a minimal reproduction of your configuration and it also does not work for me. The code I used to test it is located at skilkis/vscode_python_pytest_multi_root_breakpoint_issue. Would you mind testing the launch.json file I linked above in a non multi-root workspace? This so that we can confirm that it works fine on your system.

After this I believe we can make a feature request to have support for settings.json and multi-root workspaces!

San Kilkis (@skilkis)

San Kilkis (@skilkis) I tested this with launch.json vs. the "launch" setting in settings.json.

# settings.json
// REDACTED
    "launch": {
        "configurations": [
            {
                "name": "Python: Global Debug Tests",
                "type": "python",
                "request": "launch",
                "program": "${file}",
                "purpose": ["debug-test"],
                "console": "integratedTerminal",
                "justMyCode": true,
                "env": {"PYTEST_ADDOPTS": "--no-cov"}
            }
        ],
        "compounds": []
    },
// REDACTED

vs.

# launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Debug Tests",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "purpose": ["debug-test"],
            "console": "integratedTerminal",
            "justMyCode": true,
            "env": {"PYTEST_ADDOPTS": "--no-cov"}
        }
    ]
}

Both options show up in the "Run and Debug" sidebar and work as expected if selected. But if I comment out the contents of launch.json or delete the file, the "Python: Global Debug Tests" option will still be there but will not apply the env variable meaning the debugger will not work correctly. This is definitely a bug with the "launch" option vs. launch.json behavior.

You can recreate this with any tests.

This is actually worse than it seems at first blush. Not only will running with coverage enabled fail to respect breakpoints in PyCharm, but I've also found it causes manually inserted breakpoint() lines to stop at locations that are unpredictable and not the specified breakpoint. Not even close. This was with using the arguments of --cov and --cov-report=term.

This is what I use under settings.json to workaround this globally for the "Testing" feature (not sure about "Run and Debug"):

"python.testing.pytestArgs": [
    // Coverage is not supported by vscode:
    // https://github.com/Microsoft/vscode-python/issues/693
    // Note that this will make pytest fail if pytest-cov is not installed,
    // if that's the case, then this option needs to be be removed (overrides
    // can be set at a workspace level, it's up to you to decide what's the
    // best approach). You might also prefer to only set this option
    // per-workspace (wherever coverage is used).
    "--no-cov"
],

As mentioned in the comment in the setting, I make sure I have pytest-cov installed in my environment, even if that project is not making use of it. If I don't want or can't install pytest-cov, I can set overrides per workspace.

I don't use "Run and Debug" a lot (but I do use a lot of "Testing") and don't remember having an issue with this there so I don't know if I already had the workaround in place or I just didn't hit it.

PS.: In my settings I also make use of -s, but that's unrelated to this issue.

vscode: Add dependency on python3-pytest-cov due to its settings

Closing as we do not plan to implement this work.

Navigation Menu

Repository navigation

Debugger doesn't stop at breakpoints with pytest if pytest-cov is used #693

Environment data

Actual behavior

Expected behavior

Steps to reproduce:

26 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Debugger doesn't stop at breakpoints with pytest if pytest-cov is used #693

Description

Environment data

Actual behavior

Expected behavior

Steps to reproduce:

Activity

lmazuel commented on Feb 2, 2018

DonJayamanne commented on Feb 22, 2018

DonJayamanne commented on Feb 22, 2018

DonJayamanne commented on Feb 22, 2018

26 remaining items

skilkis commented on May 19, 2022

sgbaird commented on May 19, 2022

skilkis commented on May 19, 2022

ericchansen commented on May 19, 2022

skilkis commented on May 19, 2022

ShannonTully commented on Jun 2, 2022

edmundsj commented on Nov 16, 2022

samueloph commented on Dec 18, 2022

cwebster-99 commented on Jan 12, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions