Skip to content

pytest test discovery failed with pytest-pep8 installed #7287

Closed
Closed
pytest test discovery failed with pytest-pep8 installed#7287
Assignees
kimadeline
Labels
area-testingbugIssue identified by VS Code Team member as probable bug
Milestone
2019 November Sprint 1

Description

@zztalker
Pavel Zaikin (zztalker)
opened on Sep 9, 2019

Environment data

Version: 1.38.0
Commit: 3db7e09
Date: 2019-09-03T21:51:09.716Z
Electron: 4.2.10
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Linux x64 4.15.0-36-generic
Extension Name: Python
Id: ms-python.python
Version: 2019.9.34911
Publisher: Microsoft
Python version: 3.5/3.6/3.7

Expected behavior

Tests discovered

Actual behavior

Tests discovering failed.

The workaround is to remove --pep8 keyword from pytestArgs property.

Steps to reproduce:

  1. clone https://github.com/zztalker/just_for_test ( git clone https://github.com/zztalker/just_for_test)
  2. Install requrements (pip install -r requirements.txt), we need only pytest-pep8
  3. Open VS Code from folder (code .)
  4. Turn on test discovery and add in settings:
{
    "python.testing.pytestArgs": [
        "--pep8",
        "."
    ],
    "python.testing.unittestEnabled": false,
    "python.testing.nosetestsEnabled": false,
    "python.testing.pytestEnabled": true
}
  1. Try to discover tests.

Logs

Output for Python Test Log in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

python /home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/run_adapter.py discover pytest -- -s --cache-clear --pep8 .
Test Discovery failed: 
Error: /home/pz/.local/lib/python3.7/site-packages/pep8.py:110: FutureWarning: Possible nested set at position 1
  EXTRANEOUS_WHITESPACE_REGEX = re.compile(r'[[({] | []}),;:]')
============================= test session starts ==============================
platform linux -- Python 3.7.4, pytest-4.6.3, py-1.8.0, pluggy-0.12.0
rootdir: /home/pz/snippets/cool-tests, inifile: pytest.ini
plugins: pep8-1.0.6
DBG parse item <Pep8Item test_pytest1.py> test_pytest1.py None
collected 9 items
<Pep8Item test_pytest1.py>
<Module test_pytest1.py>
  <Function test_pytest1_test1>
  <Function test_pytest1_test2>
  <Function test_even_failed>
  <Function test_odd_failed>
  <Function test_random>
<Pep8Item test_foldered/test_pytest2.py>
<Module test_foldered/test_pytest2.py>
  <Function test_pytest2_test3>
  <Function test_pytest2_test4>
DBG parse item <Pep8Item test_pytest1.py> test_pytest1.py None
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 498, in perform_collect
INTERNALERROR>     session=self, config=self.config, items=items
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 80, in pytest_collection_modifyitems
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 65, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 163, in parse_item
INTERNALERROR>     testroot, relfile = _split_fspath(str(item.fspath), fileid, item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 142, in <lambda>
INTERNALERROR>     _split_fspath=(lambda *a: _split_fspath(*a)),
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 232, in _split_fspath
INTERNALERROR>     relsuffix = fileid[1:]  # Drop (only) the "." prefix.
INTERNALERROR> TypeError: 'NoneType' object is not subscriptable
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 206, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 249, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 259, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/_pytest/main.py", line 501, in perform_collect
INTERNALERROR>     hook.pytest_collection_finish(session=self)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/hooks.py", line 289, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 87, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/manager.py", line 81, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/pz/.local/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 94, in pytest_collection_finish
INTERNALERROR>     test, parents = self.parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 65, in parse_item
INTERNALERROR>     return parse_item(item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 163, in parse_item
INTERNALERROR>     testroot, relfile = _split_fspath(str(item.fspath), fileid, item)
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 142, in <lambda>
INTERNALERROR>     _split_fspath=(lambda *a: _split_fspath(*a)),
INTERNALERROR>   File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_pytest_item.py", line 232, in _split_fspath
INTERNALERROR>     relsuffix = fileid[1:]  # Drop (only) the "." prefix.
INTERNALERROR> TypeError: 'NoneType' object is not subscriptable

========================== 1 warnings in 0.02 seconds ==========================

Traceback (most recent call last):
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/run_adapter.py", line 18, in <module>
    main(tool, cmd, subargs, toolargs)
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/__main__.py", line 90, in main
    parents, result = run(toolargs, **subargs)
  File "/home/pz/.vscode/extensions/ms-python.python-2019.9.34911/pythonFiles/testing_tools/adapter/pytest/_discovery.py", line 35, in discover
    raise Exception('pytest discovery failed (exit code {})'.format(ec))
Exception: pytest discovery failed (exit code 3)

Activity

  1. Pavel Zaikin (zztalker)
    added
    triage-neededNeeds assignment to the proper sub-team
    bugIssue identified by VS Code Team member as probable bug
    on Sep 9, 2019
  2. Kim-Adeline Miguel (kimadeline)
    added
    triage
    on Sep 9, 2019
  3. Kim-Adeline Miguel (kimadeline)
    self-assigned this
    on Sep 9, 2019
  4. ghost removed
    triage-neededNeeds assignment to the proper sub-team
    on Sep 9, 2019

  5. kimadeline commented on Sep 9, 2019

    @kimadeline
    Kim-Adeline Miguel (kimadeline)
    on Sep 9, 2019

    Hi Pavel Zaikin (@zztalker) 👋 Thank you for reporting this issue and providing sample code 👍
    There is no requirements.txt in the repo you linked, so I installed the following packages in order to reproduce your issue:

    pytest==4.6.3
pytest-pep8==1.0.6

    Notes

    • Issue can be reproduced with 2019.9.34911 and 2019.8.30787 (did not try earlier versions of the extension)
    • Directly calling pytest --pep8 . in a terminal window works
  6. Kim-Adeline Miguel (kimadeline)
    removed their assignment
    on Sep 9, 2019
  7. Kim-Adeline Miguel (kimadeline)
    added
    area-testing
    needs PR
    reason-preexisting
    and removed
    triage
    on Sep 9, 2019

  8. zztalker commented on Sep 11, 2019

    @zztalker
    Pavel Zaikin (zztalker)
    on Sep 11, 2019
    Author

    Kim-Adeline Miguel (@kimadeline) My apologies, I've forgotten to push it (requirements.txt) to the repo. I suppose all worked well till:

    Name: Python
Id: ms-python.python
Description: Linting, Debugging (multi-threaded, remote), Intellisense, code formatting, refactoring, unit tests, snippets, and more.
Version: 2019.5.18875
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.python

  9. zztalker commented on Sep 11, 2019

    @zztalker
    Pavel Zaikin (zztalker)
    on Sep 11, 2019
    Author

    I believe, that problem in section, and will try to look into code for more details:

    <Pep8Item test_pytest1.py>

  10. Hecatron commented on Sep 23, 2019

    @Hecatron
    Hecatron (Hecatron)
    on Sep 23, 2019

    I also ran into this same problem / error output when using --codestyle in the pytest options
    which runs pycodestyle

  11. kergoth commented on Sep 26, 2019

    @kergoth
    Christopher Larson (kergoth)
    on Sep 26, 2019

    pytest-cram fails in this way as well.

  12. 6 remaining items

  13. Kim-Adeline Miguel (kimadeline)
    closed this as completedon Nov 4, 2019
  14. ghost removed
    needs PR
    on Nov 4, 2019
  15. Kim-Adeline Miguel (kimadeline)
    reopened this on Nov 4, 2019
  16. ghost added
    triage-neededNeeds assignment to the proper sub-team
    on Nov 4, 2019
  17. Kim-Adeline Miguel (kimadeline)
    added
    needs PR
    on Nov 4, 2019
  18. ghost removed
    triage-neededNeeds assignment to the proper sub-team
    on Nov 4, 2019
  19. Eric Snow (ericsnowcurrently)
    added a commit that references this issue on Nov 6, 2019

    Skip tests generated by pytest plugins during discovery. (#8242)

    355a325
  20. Karthik Nadig (karthiknadig)
    modified the milestones: 2019 October Sprint 3, 2019 November Sprint 1 on Nov 6, 2019
  21. Karthik Nadig (karthiknadig)
    assigned
    Kim-Adeline Miguel (kimadeline)
    on Nov 7, 2019
  22. Karthik Nadig (karthiknadig)
    removed
    in review
    on Nov 11, 2019

  23. kimadeline commented on Nov 12, 2019

    @kimadeline
    Kim-Adeline Miguel (kimadeline)
    on Nov 12, 2019

    ✅ Validated in 2019.12.48501-dev, test discovery doesn't fail with "python.testing.pytestEnabled": true and the following settings:

     "python.testing.pytestArgs": [
        "--pep8",
        "."
    ],
  24. Kim-Adeline Miguel (kimadeline)
    closed this as completedon Nov 12, 2019
  25. ghost removed
    needs PR
    on Nov 12, 2019
  26. lock
    locked as resolved and limited conversation to collaborators on Nov 20, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-testingbugIssue identified by VS Code Team member as probable bug

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions