Skip to content

Python.testing.pytestArgs does not work with option -m #21798

Closed
Closed
Python.testing.pytestArgs does not work with option -m#21798
Assignees
eleanorjboyd
Labels
area-testing

Description

@mondwan
Mond WAN (mondwan)
opened on Aug 10, 2023

Type: Bug

Behaviour

Expected vs. Actual

For settings.json

{
"python.testing.pytestArgs": ["-m", "lazy"]
}

Expect to see vscode to run pytest with argument in TEST RESULT UI

Running test with arguments: --rootdir /home/vdc/git/six --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-1147287WvATg175YF9w.xml -m lazy

But, -m is gone

Running test with arguments: --rootdir /home/vdc/git/six --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-1147287NQVjhdMkoSyI.xml

However, if you do something like that, it will work

{
"python.testing.pytestArgs": ["-m lazy"]
}

Steps to reproduce:

  1. Set -m in the settings.json
{
"python.testing.pytestArgs": ["-m", "lazy"]
}
  1. Run the vscode test

Personally, I clone a open source project six to replicate this issue. HOpefully, this can be replicated with any python project

Diagnostic data

  • Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
  • Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
  • Value of the python.languageServer setting: Jedi
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python) 

2023-08-10 13:28:40.743 [info] Discover tests for workspace name: six - uri: /home/vdc/git/six
2023-08-10 13:28:40.765 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-08-10 13:28:40.765 [info] cwd: .
2023-08-10 13:28:45.858 [info] Discover tests for workspace name: six - uri: /home/vdc/git/six
2023-08-10 13:28:45.858 [warning] Unknown command line option passed into args parser for tests '-m lazy'. Please report on https://github.com/Microsoft/vscode-python/issues/new
2023-08-10 13:28:45.858 [warning] Unknown command line option passed into args parser for tests '-m lazy'. Please report on https://github.com/Microsoft/vscode-python/issues/new
2023-08-10 13:28:45.884 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear "-m lazy"
2023-08-10 13:28:45.884 [info] cwd: .

User Settings 


languageServer: "Jedi"

linting
• pycodestyleEnabled: true
• pydocstyleEnabled: true

formatting
• provider: "black"
• blackArgs: "<placeholder>"

testing
• promptToConfigure: false
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• autoTestDiscoverOnSaveEnabled: false

Extension version: 2023.14.0
VS Code version: Code 1.81.0 (6445d93c81ebe42c4cbd7a60712e0b17d9463e97, 2023-08-02T12:37:13.485Z)
OS version: Windows_NT x64 10.0.19044
Modes:
Remote OS version: Linux x64 5.4.0-148-generic
Remote OS version: Linux x64 5.4.0-148-generic
Remote OS version: Linux x64 5.4.0-148-generic

Activity

  1. github-actions
    added
    triage-neededNeeds assignment to the proper sub-team
    on Aug 10, 2023
  2. Karthik Nadig (karthiknadig)
    assigned
    Eleanor Boyd (eleanorjboyd)
    on Aug 10, 2023

  3. eleanorjboyd commented on Aug 11, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Aug 11, 2023
    Member

    Hello! Could you try this on the rewrite, we have a new testing architecture. To try it yourself, you need to add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"].

    You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your user settings then check for Experiment 'pythonTestAdapter' is active in your python logs.

    Let me know if the rewrite fixes your issue. Thanks!

  4. github-actions
    added
    info-neededIssue requires more information from poster
    on Aug 11, 2023

  5. mondwan commented on Aug 21, 2023

    @mondwan
    Mond WAN (mondwan)
    on Aug 21, 2023
    Author

    Hi, I tried with following

    {
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": ["-m", "lazy"],
    "python.experiments.optInto": ["pythonTestAdapter"],
    "python.analysis.logLevel": "Trace"
}

    But, it sounds like the experiment option cannot be active, and the option does not work as expected neither

    2023-08-21 11:22:11.849 [info] Telemetry is disabled
2023-08-21 11:22:11.849 [info] Experiments are disabled, only manually opted experiments are active.
2023-08-21 11:22:11.853 [info] Test server listening.
2023-08-21 11:22:11.959 [info] > ./venv/bin/python -I ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/get_output_via_markers.py ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/interpreterInfo.py
2023-08-21 11:22:12.034 [info] Python interpreter path: ./venv/bin/python
2023-08-21 11:22:12.552 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.037 [warning] Retry failed, storage update failed for key PYTHON_GLOBAL_STORAGE_KEYS
2023-08-21 11:22:13.934 [info] Starting Jedi language server for six.
2023-08-21 11:22:28.337 [info] Discover tests for workspace name: six - uri: XXXXXXXXXXXXXXXXXXXXXXXX
2023-08-21 11:22:28.412 [info] > . ./venv/bin/activate && echo 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' && python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/printEnvVariables.py
2023-08-21 11:22:28.412 [info] shell: bash
2023-08-21 11:22:28.476 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
2023-08-21 11:22:28.476 [info] cwd: .
  6. github-actions
    removed
    info-neededIssue requires more information from poster
    on Aug 21, 2023

  7. eleanorjboyd commented on Aug 21, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Aug 21, 2023
    Member

    hm interesting. Did you reload your vscode windows? Also, you set it in your user settings json not your workspace one correct?

  8. github-actions
    added
    info-neededIssue requires more information from poster
    on Aug 21, 2023

  9. mondwan commented on Aug 22, 2023

    @mondwan
    Mond WAN (mondwan)
    on Aug 22, 2023
    Author

    As I am using a remote vscode, I have tried to edit my host's settings.json.

    ..
    "python.experiments.optInto": ["pythonTestAdapter"],
..

    Then, I tried to reload the vscode. But, the results are the same, which is does not work as expected.

  10. github-actions
    removed
    info-neededIssue requires more information from poster
    on Aug 22, 2023

  11. mondwan commented on Aug 24, 2023

    @mondwan
    Mond WAN (mondwan)
    on Aug 24, 2023
    Author

    Here is how I reproduce this bug in a public repository

    Clone a public repository https://github.com/benjaminp/six
    Setup the settings.json as described
    Run the tests as described

    Hope this helps

  12. eleanorjboyd commented on Aug 29, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Aug 29, 2023
    Member

    related to #21845. We will need to overhaul args now that we have more flexibility with our architecture. This issue will be resolved by that. Thanks

  13. Eleanor Boyd (eleanorjboyd)
    added
    area-testing
    and removed
    triage-neededNeeds assignment to the proper sub-team
    on Sep 19, 2023

  14. eleanorjboyd commented on Dec 6, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Dec 6, 2023
    Member

    Hi! Have you had any luck with this issue recently? We just expanded the experiment to include everyone so you should be on the rewrite (without the need for the setting). From your logs they always included 2023-08-21 11:22:28.476 [info] > ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear which is the path to the old version of the code. Let me know if there are any changes, the overhaul to the args will also fix the problem but in the meantime it might help. Thanks

  15. Eleanor Boyd (eleanorjboyd)
    added
    info-neededIssue requires more information from poster
    on Dec 6, 2023

  16. mondwan commented on Dec 7, 2023

    @mondwan
    Mond WAN (mondwan)
    on Dec 7, 2023
    Author

    Is it released? I just tried. It still does not work

    I am using ms-python.python 2023.20.0

  17. eleanorjboyd commented on Dec 7, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Dec 7, 2023
    Member

    Are you still seeing ./venv/bin/python ~/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/testing_tools/run_adapter.py in your ags? It should be enabled now unless you have a specific setting disabling all experiments.

  18. mondwan commented on Dec 7, 2023

    @mondwan
    Mond WAN (mondwan)
    on Dec 7, 2023 via email
    Author

  19. github-actions commented on Jan 7, 2024

    @github-actions
    github-actionsbot
    on Jan 7, 2024 – with GitHub Actions

    Hey Eleanor Boyd (@eleanorjboyd), this issue might need further attention.

    Mond WAN (@mondwan), you can help us out by closing this issue if the problem no longer exists, or adding more information.

  20. robertdj commented on Jan 17, 2024

    @robertdj
    Robert Dahl Jacobsen (robertdj)
    on Jan 17, 2024

    Any news on this issue?

    I have noticed that I cannot debug tests if I use ["-m lazy"], but I can if I use ["-m", "lazy"].

  21. eleanorjboyd commented on Jan 17, 2024

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jan 17, 2024
    Member

    All args must be in separate strings in the array so ["-m", "lazy"] is correct, if that is working then this is the desired behavior. Mond WAN (@mondwan), are you seeing this too? If so this issue is resolved and I will close, thanks

  22. robertdj commented on Jan 18, 2024

    @robertdj
    Robert Dahl Jacobsen (robertdj)
    on Jan 18, 2024

    Oops! That was lacking some context.

    If I use ["-m", "lazy"] (or in my case ["-m", "not lazy"]) then the mark filtering is NOT respected, but I CAN debug.

    If I use ["-m lazy"] then the mark filtering IS respected, but I CANNOT debug.

  23. mondwan commented on Jan 18, 2024

    @mondwan
    Mond WAN (mondwan)
    on Jan 18, 2024
    Author

    hi Robert Dahl Jacobsen (@robertdj)
    Then, there are 2 bugs. mine one is the 1st situation. In other words, the mark filtering does not work as expected.
    As suggested by Eleanor Boyd (@eleanorjboyd) , ["-m", "lazy"] should be the expected usage. But, mark filtering in this context does not work as expected

    hi Eleanor Boyd (@eleanorjboyd)
    I have double checked just now. It still does not work as expected. Can you show your settings & results?

  24. eleanorjboyd commented on Jan 25, 2024

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jan 25, 2024
    Member

    apologies I got mixed up, this is waiting on #21845 and that proposal will fix the incorrect arg parsing. Sorry and thank you for your patience!

  25. Brett Cannon (brettcannon)
    removed
    info-neededIssue requires more information from poster
    on Mar 13, 2024

  26. eleanorjboyd commented on Dec 10, 2024

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Dec 10, 2024
    Member

    We removed arg parsing for pytest so this should no longer be an issue. I gave it a try and it seems to be resolved, please comment if you are still having an issue and I can re-open. Thanks!

  27. Eleanor Boyd (eleanorjboyd)
    closed this as completedon Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-testing

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions