Skip to content

Test results not updated if test is run via codelens #6787

Closed
#16769
Closed
Test results not updated if test is run via codelens#6787
#16769
Labels
area-testingbugIssue identified by VS Code Team member as probable bugverifiedVerification succeeded
Milestone
August 2021

Description

@Flamefire
Alexander Grund (Flamefire)
opened on Jul 31, 2019

Environment data

  • VS Code version: 1.36
  • Extension version (available under the Extensions sidebar): 2019.6.24221
  • OS and version: Linux Mint 19.1
  • Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.0
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): None
  • Relevant/affected Python packages and their versions: pytest-3.9.1, same with pytest 5.0.1
  • Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version #3977): not jedi

Expected behaviour

After fixing a test case and running it via the codelens "Run Test" the success icon in the codelens and the error squiggle on the function should reflect that result

Actual behaviour

Running a test removes the success/failure icon in front of the codelens. Error squiggles are added if the test fails, but not removed, when it succeeds.

Steps to reproduce:

  1. create test function (e.g. def test_foo(): assert True)
  2. Use "Run All Tests" -> tick mark appears in code lens
  3. change to assert False. Run with codelens "Run Test" -> Tick dissappears, error squiggles appear
  4. change back to assert True and run with codelens -> Output shows success, no tick, error squiggles persist
  5. Run with "Run All Tests" -> tick mark appears in code lens, error squiggles removed

Logs

Logs look good, e.g. Python Test Log has =========================== 1 passed in 0.06 seconds ===========================

Activity

  1. Alexander Grund (Flamefire)
    added
    triage-neededNeeds assignment to the proper sub-team
    bugIssue identified by VS Code Team member as probable bug
    on Jul 31, 2019
  2. Don Jayamanne (DonJayamanne)
    self-assigned this
    on Jul 31, 2019
  3. Don Jayamanne (DonJayamanne)
    added
    triage
    on Jul 31, 2019
  4. ghost removed
    triage-neededNeeds assignment to the proper sub-team
    on Jul 31, 2019

  5. DonJayamanne commented on Aug 2, 2019

    @DonJayamanne
    Don Jayamanne (DonJayamanne)
    on Aug 2, 2019

    Thanks for reporting this issue, please could you:

    • Send the contents of your settings.json (I'm interested in see the test framework you are using - unittest, nose or pytest)
    • Send the output from the Test Log (please send everything)
    • Finally, please could you send a screenshot of the issue.
  6. Don Jayamanne (DonJayamanne)
    added
    info-neededIssue requires more information from poster
    area-testing
    and removed
    info-neededIssue requires more information from poster
    on Aug 2, 2019

  7. Flamefire commented on Aug 2, 2019

    @Flamefire
    Alexander Grund (Flamefire)
    on Aug 2, 2019
    Author

    I'm using pytest with versions above in pyenv, no other related options used

    Test output does not show anything unexpected, just test passed/failed as when running pytest directly

    Will add files once I'm back at the machine

  8. Flamefire commented on Aug 7, 2019

    @Flamefire
    Alexander Grund (Flamefire)
    on Aug 7, 2019
    Author

    Settings.json: (grep python)

    "python.formatting.provider": "yapf",
"python.jediMemoryLimit": -1,
"python.testing.pytestEnabled": true,
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestArgs": ["-c", "/dev/null"],
"python.jediEnabled": false,
"python.pythonPath": "/home/alex/.pyenv/shims/python",
"python.testing.cwd": "",

    Test.log

    python /home/alex/.vscode/extensions/ms-python.python-2019.8.29288/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear -c /dev/null
python /home/alex/.vscode/extensions/ms-python.python-2019.8.29288/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear -c /dev/null
python /home/alex/.vscode/extensions/ms-python.python-2019.8.29288/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear
============================= test session starts ==============================
platform linux -- Python 3.6.0, pytest-5.0.1, py-1.7.0, pluggy-0.12.0
Using --randomly-seed=1565185563
rootdir: /home/alex/taurushome/io-500-tests
plugins: console-scripts-0.1.7, repeat-0.7.0, randomly-1.2.3, nbval-0.9.1, cov-2.7.1, profiling-1.3.0
collected 1 item

tests/test_run.py .                                                      [100%]

-------------- generated xml file: /tmp/tmp-16882srqMED1q0pzx.xml --------------
=========================== 1 passed in 0.15 seconds ===========================

    Screenshort after clicking "Run Test": pytest

    Cause

    I found the issue to be solved when removing the -c /dev/null from pytestArgs. I was using this to force vanilla pytest which is not affected by e.g. my users .pytestrc (or what that file is called)

    Somehow this seems to interfere with VSC test result discovery.

  9. Don Jayamanne (DonJayamanne)
    removed their assignment
    on Aug 8, 2019
  10. Don Jayamanne (DonJayamanne)
    added
    reason-preexisting
    needs PR
    and removed
    triage
    on Aug 8, 2019

  11. DonJayamanne commented on Aug 8, 2019

    @DonJayamanne
    Don Jayamanne (DonJayamanne)
    on Aug 8, 2019

    I can replicate this at my end.
    For some reason the discovered tests are being reported as existing in the root directory and not under tests folder.

  12. bersbersbers commented on Oct 4, 2019

    @bersbersbers
    bersbersbers
    on Oct 4, 2019

    I see a similar issue with nose, without any extra arguments and regardless of the way I start the tests (side bar, run all tests, ...). python.testing.pytestArgs is not set.

    If this issue is more general, could the issue description be changed, e.g., to "Test results not updated if tests reside in subdirectory"?

  13. Brett Cannon (brettcannon)
    added
    verifiedVerification succeeded
    on Jul 29, 2021

  14. brettcannon commented on Jul 29, 2021

    @brettcannon
    Brett Cannon (brettcannon)
    on Jul 29, 2021
    Member

    The traditional codelenses are gone in the future test provider UI, but I can verify that refreshing from the gutter works.

    Animation

  15. Karthik Nadig (karthiknadig)
    added this to the August 2021 milestone on Aug 5, 2021
  16. Karthik Nadig (karthiknadig)
    closed this as completedin #16769on Aug 5, 2021
  17. github-actions
    locked as resolved and limited conversation to collaborators on Sep 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-testingbugIssue identified by VS Code Team member as probable bugverifiedVerification succeeded

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions