Skip to content

Tests discovery failing on symlink folder #20718

Closed
Closed
Tests discovery failing on symlink folder#20718
Assignees
eleanorjboyd
Labels
area-testinginfo-neededIssue requires more information from postertriage-neededNeeds assignment to the proper sub-team

Description

@ZacCranko
Zac Cranko (ZacCranko)
opened on Feb 19, 2023

Type: Bug

  1. Open a remote-ssh connection to a server, on which run
  2. poetry new github_test_discovery_issue
  3. code-insiders github_test_discovery_issue
  4. Create a new file touch tests/test_foo.py then add to it
def test_foo():
    assert True
  1. Configure vscode to use pytest with the following settings.json (the one generated by vscode by default)
{
    "python.testing.pytestArgs": [
        "tests"
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true
}
  • After following the above steps, the test explorer shows only the folder 'tests', but does not discover test_foo:

Screenshot 2023-02-20 at 9 26 07 am

  • Pytest run from the terminal in this folder succeeds in discovering and running test_foo
➜  github_test_discovery_issue pytest tests/
====================== test session starts ======================
platform linux -- Python 3.8.10, pytest-7.2.1, pluggy-1.0.0
rootdir: /mnt/disks/persist/code/github_test_discovery_issue
plugins: typeguard-2.13.3, anyio-3.6.2
collected 1 item                                                

tests/test_foo.py .                                       [100%]

======================= 1 passed in 0.01s =======================
  • Please help. Thanks!

Extension version: 2023.3.10481011
VS Code version: Code - Insiders 1.76.0-insider (Universal) (96a795c, 2023-02-17T22:20:52.580Z)
OS version: Darwin arm64 22.3.0
Modes:
Sandboxed: Yes
Connection to 'ssh-remote+zac-dev' could not be established
Remote OS version: Linux x64 5.13.0-1023-gcp

System Info
Item Value
CPUs Apple M2 (8 x 24)
GPU Status 2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
metal: disabled_off
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_renderer: enabled_on
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off
Load (avg) 5, 4, 3
Memory (System) 16.00GB (0.74GB free)
Process Argv --crash-reporter-id d2f555fd-c740-461a-926d-b30c1004a468
Screen Reader no
VM 0%

Connection to 'ssh-remote+zac-dev' could not be established

Item Value
Remote SSH: zac-dev
OS Linux x64 5.13.0-1023-gcp
CPUs AMD EPYC 7B12 (240 x 2250)
Memory (System) 400.47GB (394.47GB free)
VM 0%
A/B Experiments 
vsliv695:30137379
vsins829:30139715
vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30422396
pythontb:30258533
pythonptprofiler:30281269
vshan820:30294714
pythondataviewer:30285072
vscod805cf:30301675
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593cf:30376535
pythonvs932:30404738
cppdebug:30492333
vscaac:30438845
vsclangdf:30492506
c4g48928:30535728
dsvsc012:30540252
pynewext54:30618038
pylantcb52:30590116
pyindex848:30611229
nodejswelcome1:30587009
pyind779:30611226
pythonsymbol12:30651887
2i9eh265:30646982
showlangstatbar:30659908
pythonb192:30661256

Activity

  1. github-actions
    added
    triage-neededNeeds assignment to the proper sub-team
    on Feb 19, 2023
  2. Karthik Nadig (karthiknadig)
    assigned
    Eleanor Boyd (eleanorjboyd)
    on Feb 21, 2023

  3. eleanorjboyd commented on Feb 21, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Feb 21, 2023
    Member

    Could you share your configuration for your poetry environment? Thanks

  4. Eleanor Boyd (eleanorjboyd)
    added
    info-neededIssue requires more information from poster
    on Feb 21, 2023

  5. ZacCranko commented on Feb 21, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Feb 21, 2023
    Author

    Sure, here's the output from poetry config --list.

    cache-dir = "/home/zcranko/.cache/pypoetry"
experimental.new-installer = true
experimental.system-git-client = false
installer.max-workers = null
installer.no-binary = null
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/zcranko/.cache/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"

  6. eleanorjboyd commented on Feb 22, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Feb 22, 2023
    Member

    Hi, can you also send logs from the python output? Sorry I forgot that in my first message. You can get these by going to view -> output -> and select python from the dropdown on the right. Thanks!

  7. ZacCranko commented on Feb 22, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Feb 22, 2023
    Author

    No problem, it's as follows:

    Experiment 'pythonSurveyNotificationcf' is active
Experiment 'pythonPromptNewToolsExt' is active
LSP Notebooks experiment is enabled
LSP Notebooks interactive window support is enabled
> conda info --json
> poetry env list --full-path
cwd: .
Python interpreter path: /bin/python
> /bin/python -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/interpreterInfo.py
> /usr/bin/python -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/interpreterInfo.py
> /mnt/disks/persist/code/mdj-toolkit/.venv/bin/python -I ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/get_output_via_markers.py ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/interpreterInfo.py
Starting Pylance language server.
> /bin/python ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
cwd: .

  8. corey-dawson commented on Feb 23, 2023

    @corey-dawson
    corey-dawson
    on Feb 23, 2023

    I have the same issue, where the the test tab won't discover my pytest inside the "tests" folder. I have a feeling it is because get_current_traceback changed to DebugTraceback.
    using conda with python 3.9.16
    Here is a section of my output;

    ImportError: cannot import name 'get_current_traceback' from 'werkzeug.debug.tbtools' (C:\Users\myuser\Miniconda3\envs\analytics\lib\site-packages\werkzeug\debug\tbtools.py)
ERROR conda.cli.main_run:execute(49): `conda run python c:\Users\myuser\.vscode\extensions\ms-python.python-2022.20.2\pythonFiles\get_output_via_markers.py c:\Users\myuser\.vscode\extensions\ms-python.python-2022.20.2\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir c:\Users\myuser\Documents\Git Repos\battery_placement_opt_py -s --cache-clear tests` failed. (See above for error)

  9. eleanorjboyd commented on Feb 23, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Feb 23, 2023
    Member

    corey-dawson can you file a separate issue, since you are using conda and not poetry. Thanks!

  10. ZacCranko commented on Feb 23, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Feb 23, 2023
    Author

    Eleanor Boyd (@eleanorjboyd) Please let me know if there's anything further you need

  11. eleanorjboyd commented on Feb 23, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Feb 23, 2023
    Member

    Hi Zac Cranko (@ZacCranko), can you please run this from your command line and see if it returns the expected output discover pytest -- --rootdir . -s --cache-clear tests? If that doesn't work can you try disabling color on the output? Sometimes this interferes with our ability to parse the pytest output.

  12. ZacCranko commented on Feb 23, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Feb 23, 2023
    Author
    ➜  github_test_discovery_issue /bin/python ~/.vscode-server-insiders/extensions/ms-python.python-2023.3.10531009/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
[{"rootid": ".", "root": "/mnt/disks/persist/code/github_test_discovery_issue", "parents": [{"id": "./tests", "kind": "folder", "name": "tests", "parentid": ".", "relpath": "./tests"}, {"id": "./tests/test_foo.py", "kind": "file", "name": "test_foo.py", "parentid": "./tests", "relpath": "./tests/test_foo.py"}], "tests": [{"id": "./tests/test_foo.py::test_foo", "name": "test_foo", "source": "./tests/test_foo.py:1", "markers": [], "parentid": "./tests/test_foo.py"}]}]

    Looks like test_foo is being discovered with that command. Viz. "tests": [{"id": "./tests/test_foo.py::test_foo"...

    Can you confirm I've run it correctly (I added the run_adapter prefix). Also I'm not sure what you mean about disabling colour on the output, if you explain to me what you want me to try I'm happy to give it a go.

  13. corey-dawson commented on Feb 24, 2023

    @corey-dawson
    corey-dawson
    on Feb 24, 2023

    corey-dawson can you file a separate issue, since you are using conda and not poetry. Thanks!

    will do

  14. eleanorjboyd commented on Feb 27, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Feb 27, 2023
    Member

    Zac Cranko (@ZacCranko), no need to append the run_adapter prefix can you try it like this python -m pytest --rootdir . -s --cache-clear. Secondly I am not sure the exact setting but feel free to ask the poetry team here if you can't find it: https://github.com/python-poetry/poetry.

  15. ZacCranko commented on Feb 27, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Feb 27, 2023
    Author

    I needed the run_adapter prefix for the discover command in discover pytest -- --rootdir . -s --cache-clear tests to do something.

    The output of poetry run python -m pytest --rootdir . -s --cache-clear is as follows:

    ➜  github_test_discovery_issue poetry run python -m pytest --rootdir . -s --cache-clear

================================= test session starts =================================
platform linux -- Python 3.8.10, pytest-7.2.1, pluggy-1.0.0
rootdir: /mnt/disks/persist/code/github_test_discovery_issue
collected 1 item                                                                      

tests/test_foo.py .

================================== 1 passed in 0.01s ==================================

  16. 17 remaining items

  17. github-actions
    removed
    info-neededIssue requires more information from poster
    on Apr 6, 2023

  18. eleanorjboyd commented on Apr 10, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Apr 10, 2023
    Member

    Thank you everyone for your responses and help in figuring out this issue! Closing as it seems the issue is resolved but please let me know if more conversation is needed. Thanks!

  19. Eleanor Boyd (eleanorjboyd)
    closed this as completedon Apr 10, 2023
  20. github-actions
    added
    info-neededIssue requires more information from poster
    on Apr 10, 2023
  21. Eleanor Boyd (eleanorjboyd)
    changed the title [-]Tests not discovered[/-] [+]Tests discovery failing on symlink folder[/+] on Apr 10, 2023

  22. ZacCranko commented on Apr 10, 2023

    @ZacCranko
    Zac Cranko (ZacCranko)
    on Apr 10, 2023
    Author

    Hi Eleanor Boyd (@eleanorjboyd), in what sense is the issue resolved. Does vscode Python now support properly resolving symlinks?

  23. github-actions
    removed
    info-neededIssue requires more information from poster
    on Apr 10, 2023

  24. davidsben commented on Apr 17, 2023

    @davidsben
    Dave Brown (davidsben)
    on Apr 17, 2023

    Just to echo Zac Cranko (@ZacCranko) 's sentiment. Has a fix been implemented in the extension? The discussion above was, at best, a workaround to a bug in the extension (and / or vscode). Outside of vscode there is no problem using pytest in the symlink case, it only fails to work when used within the vscode setup.

  25. karthiknadig commented on Apr 17, 2023

    @karthiknadig
    Karthik Nadig (karthiknadig)
    on Apr 17, 2023
    Member

    This should not be closed, yet. But the fix for this is that we created a pytest plugin to detect pytest tests via this PR here #20631 . Which should be able to detect these, but we still need to verify that this case is handled by it.

  26. Karthik Nadig (karthiknadig)
    reopened this on Apr 17, 2023

  27. eleanorjboyd commented on Jul 11, 2023

    @eleanorjboyd
    Eleanor Boyd (eleanorjboyd)
    on Jul 11, 2023
    Member

    Consolidating all issues on testing and symlinks to issue #9347. Let me know if you do not think this other issue is representative of the bug you are facing. Thanks!

  28. Eleanor Boyd (eleanorjboyd)
    closed this as completedon Jul 11, 2023
  29. github-actions
    added
    info-neededIssue requires more information from poster
    on Jul 11, 2023
  30. Eleanor Boyd (eleanorjboyd)
    added
    area-testing
    on Aug 2, 2023
  31. github-actions
    locked as resolved and limited conversation to collaborators on Sep 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-testinginfo-neededIssue requires more information from postertriage-neededNeeds assignment to the proper sub-team

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions