Repository navigation
1.12.0: pytest fails in ropetest/contrib/autoimporttest.py::AutoImportTest::test_search_submodule unit and some pytest warnings #749
Copy link
Copy link
Closed
Labels
bugUnexpected or incorrect user-visible behaviorUnexpected or incorrect user-visible behavior
Description
python3 -sBm build -w --no-isolation- because I'm calling
buildwith--no-isolationI'm using during all processes only locally installed modules - install .whl file in </install/prefix> using 'installer` module
- run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
- build is performed in env which is
cut off from access to the public network(pytest is executed with-m "not network")
Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rope-1.12.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rope-1.12.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.4, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rope-1.12.0
configfile: pyproject.toml
plugins: subprocess-1.5.0
collected 2072 items
ropetest/advanced_oi_test.py ............................................................ [ 2%]
ropetest/builtinstest.py ............................................................... [ 5%]
ropetest/codeanalyzetest.py ................................................................................................................................................... [ 13%]
ropetest/doatest.py .... [ 13%]
ropetest/historytest.py ........................................ [ 15%]
ropetest/objectdbtest.py ................... [ 16%]
ropetest/objectinfertest.py ................................. [ 17%]
ropetest/projecttest.py ........................................................................................................................ [ 23%]
ropetest/pycoretest.py ................................................................................................................................ [ 29%]
ropetest/pyscopestest.py .............................................. [ 31%]
ropetest/reprtest.py ................ [ 32%]
ropetest/runmodtest.py ......... [ 33%]
ropetest/serializer_test.py ............................................................ [ 35%]
ropetest/simplifytest.py ............... [ 36%]
ropetest/type_hinting_test.py ..............................x.................x.................x.................x.......... [ 41%]
ropetest/versioningtest.py ..... [ 41%]
ropetest/contrib/autoimporttest.py .................F........ [ 42%]
ropetest/contrib/changestacktest.py . [ 42%]
ropetest/contrib/codeassisttest.py .................................................................................................................................................. [ 49%]
........................ [ 51%]
ropetest/contrib/finderrorstest.py ..... [ 51%]
ropetest/contrib/findittest.py ........... [ 51%]
ropetest/contrib/fixmodnamestest.py .... [ 51%]
ropetest/contrib/generatetest.py .............................. [ 53%]
ropetest/contrib/autoimport/autoimporttest.py ................ [ 54%]
ropetest/contrib/autoimport/modeltest.py ........... [ 54%]
ropetest/contrib/autoimport/parsetest.py ... [ 54%]
ropetest/contrib/autoimport/utilstest.py ........... [ 55%]
ropetest/refactor/__init__.py ...................................................................... [ 58%]
ropetest/refactor/change_signature_test.py ........................................ [ 60%]
ropetest/refactor/extracttest.py .............................................s..........sss....x......................s............................................................. [ 67%]
...................... [ 68%]
ropetest/refactor/importutilstest.py ............................................................................................................................................... [ 75%]
ropetest/refactor/inlinetest.py ............................................................................................... [ 80%]
ropetest/refactor/movetest.py .......................................................................... [ 83%]
ropetest/refactor/multiprojecttest.py .... [ 84%]
ropetest/refactor/patchedasttest.py ..........................s..s.s.........................................................ssssssssss............................s.s.s..s.......... [ 91%]
. [ 91%]
ropetest/refactor/renametest.py ......................................................................................... [ 95%]
ropetest/refactor/restructuretest.py ....................... [ 96%]
ropetest/refactor/similarfindertest.py ..................................... [ 98%]
ropetest/refactor/suitestest.py .......s............. [ 99%]
ropetest/refactor/usefunctiontest.py ............ [100%]
========================================================================================= FAILURES ==========================================================================================
___________________________________________________________________________ AutoImportTest.test_search_submodule ____________________________________________________________________________
self = <ropetest.contrib.autoimporttest.AutoImportTest testMethod=test_search_submodule>
def test_search_submodule(self):
self.importer.update_module("build")
import_statement = ("from build import env", "env")
> self.assertIn(import_statement, self.importer.search("env", exact_match=True))
E AssertionError: ('from build import env', 'env') not found in []
ropetest/contrib/autoimporttest.py:116: AssertionError
===================================================================================== warnings summary ======================================================================================
ropetest/advanced_oi_test.py: 60 warnings
ropetest/builtinstest.py: 63 warnings
ropetest/codeanalyzetest.py: 15 warnings
ropetest/historytest.py: 40 warnings
ropetest/objectdbtest.py: 19 warnings
ropetest/objectinfertest.py: 33 warnings
ropetest/projecttest.py: 127 warnings
ropetest/pycoretest.py: 123 warnings
ropetest/pyscopestest.py: 46 warnings
ropetest/reprtest.py: 16 warnings
ropetest/runmodtest.py: 9 warnings
ropetest/type_hinting_test.py: 76 warnings
ropetest/versioningtest.py: 5 warnings
ropetest/contrib/autoimporttest.py: 26 warnings
ropetest/contrib/changestacktest.py: 1 warning
ropetest/contrib/codeassisttest.py: 170 warnings
ropetest/contrib/finderrorstest.py: 5 warnings
ropetest/contrib/findittest.py: 11 warnings
ropetest/contrib/fixmodnamestest.py: 4 warnings
ropetest/contrib/generatetest.py: 30 warnings
ropetest/contrib/autoimport/autoimporttest.py: 18 warnings
ropetest/contrib/autoimport/utilstest.py: 5 warnings
ropetest/refactor/__init__.py: 59 warnings
ropetest/refactor/change_signature_test.py: 40 warnings
ropetest/refactor/extracttest.py: 165 warnings
ropetest/refactor/importutilstest.py: 143 warnings
ropetest/refactor/inlinetest.py: 95 warnings
ropetest/refactor/movetest.py: 74 warnings
ropetest/refactor/multiprojecttest.py: 8 warnings
ropetest/refactor/renametest.py: 89 warnings
ropetest/refactor/restructuretest.py: 23 warnings
ropetest/refactor/similarfindertest.py: 32 warnings
ropetest/refactor/usefunctiontest.py: 12 warnings
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/rope/base/project.py:229: DeprecationWarning: Delete once deprecated functions are gone
self._init_source_folders()
ropetest/advanced_oi_test.py: 139 warnings
ropetest/projecttest.py: 5 warnings
ropetest/runmodtest.py: 46 warnings
ropetest/refactor/multiprojecttest.py: 4 warnings
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/rope/base/libutils.py:34: DeprecationWarning: relative is deprecated
return relative(project.address, path)
ropetest/contrib/autoimporttest.py: 27 warnings
ropetest/contrib/autoimport/autoimporttest.py: 17 warnings
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/rope/contrib/autoimport/sqlite.py:124: DeprecationWarning: The default value for `AutoImport(memory)` argument will change to use an on-disk database by default in the future. If you want to use an in-memory database, you need to pass `AutoImport(memory=True)` explicitly.
warnings.warn(
ropetest/refactor/__init__.py::TaskHandleTest::test_getting_job_name
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/ropetest/refactor/__init__.py:1125: DeprecationWarning: Just use JobSet.name attribute/property instead
self.assertEqual("test job set", jobs.get_name())
ropetest/refactor/__init__.py::TaskHandleTest::test_getting_job_name
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/ropetest/refactor/__init__.py:1126: DeprecationWarning: Just use JobSet.job_name attribute/property instead
self.assertEqual(None, jobs.get_active_job_name())
ropetest/refactor/__init__.py::TaskHandleTest::test_getting_job_name
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/ropetest/refactor/__init__.py:1129: DeprecationWarning: Just use JobSet.name attribute/property instead
self.assertEqual("test job set", jobs.get_name())
ropetest/refactor/__init__.py::TaskHandleTest::test_getting_job_name
/home/tkloczko/rpmbuild/BUILD/rope-1.12.0/ropetest/refactor/__init__.py:1131: DeprecationWarning: Just use JobSet.job_name attribute/property instead
self.assertEqual("job1", jobs.get_active_job_name())
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
SKIPPED [1] ropetest/refactor/extracttest.py:1870: This test requires version of Python higher than 3.11
SKIPPED [1] ropetest/refactor/extracttest.py:3451: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/extracttest.py:3475: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/extracttest.py:3496: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/extracttest.py:2515: This test requires version of Python lower than 3.8
SKIPPED [1] ropetest/refactor/patchedasttest.py:435: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:422: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:405: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:1304: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1290: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1373: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1387: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1336: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1457: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1425: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1405: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1484: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1359: This test requires version of Python higher than 3.10
SKIPPED [1] ropetest/refactor/patchedasttest.py:1236: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:1216: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:1226: This test requires version of Python lower than 3.5
SKIPPED [1] ropetest/refactor/patchedasttest.py:1157: This test requires version of Python higher than 3.11
SKIPPED [1] ropetest/refactor/suitestest.py:204: This test requires version of Python higher than 3.10
XFAIL ropetest/type_hinting_test.py::DocstringNoneAssignmentHintingTest::test_hint_or - reason: failing configuration (but should work)
XFAIL ropetest/type_hinting_test.py::DocstringNotImplementedAssignmentHintingTest::test_hint_or - reason: failing configuration (but should work)
XFAIL ropetest/type_hinting_test.py::PEP0484CommentNoneAssignmentHintingTest::test_hint_or - reason: failing configuration (but should work)
XFAIL ropetest/type_hinting_test.py::PEP0484CommentNotImplementedAssignmentHintingTest::test_hint_or - reason: failing configuration (but should work)
XFAIL ropetest/refactor/extracttest.py::ExtractMethodTest::test_extract_method_f_string_false_format_value_in_regular_string - reason:
FAILED ropetest/contrib/autoimporttest.py::AutoImportTest::test_search_submodule - AssertionError: ('from build import env', 'env') not found in []
=========================================================== 1 failed, 2043 passed, 23 skipped, 5 xfailed, 1884 warnings in 23.04s ===========================================================That unit has been failing with prev version as well.
List of installed modules in build env:
Package Version
----------------------------- -------
alabaster 0.7.16
Babel 2.14.0
build 1.0.3
charset-normalizer 3.3.2
cppclean 0.13
distro 1.9.0
dnf 4.18.2
docutils 0.20.1
exceptiongroup 1.1.3
gpg 1.23.2
idna 3.6
imagesize 1.4.1
importlib-metadata 7.0.1
iniconfig 2.0.0
installer 0.7.0
Jinja2 3.1.3
libdnf 0.72.0
MarkupSafe 2.1.3
nodeenv 1.8.0
packaging 23.2
platformdirs 4.1.0
pluggy 1.3.0
Pygments 2.17.2
pyproject_hooks 1.0.0
pytest 7.4.4
pytest-subprocess 1.5.0
python-dateutil 2.8.2
pytoolconfig 1.3.1
pytz 2023.3
PyYAML 6.0.1
requests 2.31.0
setuptools 69.0.3
six 1.16.0
snowballstemmer 2.2.0
Sphinx 7.1.2
sphinx-autodoc-typehints 1.25.2
sphinx-rtd-theme 2.0.0
sphinxcontrib-applehelp 1.0.4
sphinxcontrib-devhelp 1.0.5
sphinxcontrib-htmlhelp 2.0.5
sphinxcontrib-jquery 4.1
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.7
sphinxcontrib-serializinghtml 1.1.10
tabulate 0.9.0
tomli 2.0.1
urllib3 1.26.18
wheel 0.42.0
zipp 3.17.0Please let me know if you need more details or want me to perform some diagnostics.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugUnexpected or incorrect user-visible behaviorUnexpected or incorrect user-visible behavior
Activity
musicinmybrain commentedon Jan 18, 2024
I see the same test failure while working on updating the
python-ropepackage in Fedora Linux. It looks like there were several test failures in the previous version, but this is the only one remaining in 1.12.0. I haven’t made a serious effort to find the cause of the failure, but I’m also happy to do any investigation that might help here upon request.lieryan commentedon Feb 13, 2024
This should be fixed now in #755.
kloczek commentedon Feb 15, 2024
Just tested 0.23.0 + all commits from master (up to 4deab03) and pytest still fails
Here is pytest output
kloczek commentedon Feb 15, 2024
One sec I see that now test suite wants
pip..kloczek commentedon Feb 15, 2024
Hmm looks like
pipis used to installexternal_fixturepkgmodule.If that module needs be installed to test something please use
pipinside venv to do that instead polluting list of installed modules.Additionally pytest shows some
DeprecationWarningwarningsDetails
kloczek commentedon Feb 15, 2024
I'm going to add in my package build procedure (temporary) add to --deselect list those 3 failing units to have working build procedure.
lieryan commentedon Feb 22, 2024
Thanks for the feedback @kloczek. Hmm... I think that would mean we'd need to create a venv. IIRC, venv should've also installed pip into the venv, but I'm not quite sure if it requires network to do that.
The DeprecationWarnings are fine. They are all issued by rope for the benefit of editor integration authors, we have nothing to fix here on rope's side. Maybe they can be suppressed during tests though since they're just noise in the tests.
kloczek commentedon Feb 22, 2024
I'm building all my packages in dedicated build env in which are installed ONLY exact build dependencies so effectively already it is like venv.
Is it anything wrong with list of modules with versions which I've dropped opening this ticket? 🤔
lieryan commentedon Feb 22, 2024
The tests needs to install
external_fixturepkg, because some of the tests needs to have a package with a predictable structure to test that autoimport is indexing the package correctly. The test installs theexternal_fixturepkgpackage from a .whl file that's crafted specifically for testing autoimport and is included prepackaged in rope's source archive.It could probably be theoretically possible to setup and install the test package without using pip, but I'm not familiar enough with how package installation process or venv works to trust that I'd have set things up properly, especially as venv have slightly different structure in different OSes.
kloczek commentedon Feb 22, 2024
Moment so you want to say that
ropecan alter its functionality by downloading from public network some random .whl archives and you need to test that? 🤔If that is true PLEASE stop that and please FLAG that some modules are missing because it is nothing more than asking for security related issue.
lieryan commentedon Feb 23, 2024
No, that wasn't what I was saying,
ropedoes not need to install anything at runtime.It's ropetest, our test suite, that need to install a package because it need to simulate what the user is doing, which is installing a package into their project's environment. Installing a package is part of a test's setting up a fixture, not the system under test (SUT). And it's not random .whl, this wheel is built specifically for ropetest and is included in the test suite itself, specifically because I don't want it to download random .whls off the internet even if it's just for running tests.
kloczek commentedon Feb 23, 2024
Why test suite needs to simulate user? 🤔
Test suite should test module CODE.
Whatever is the reason downloading and installing anything during the test suite execution is bad.
Did you know that most of the distros production build infras are intentionally cut off from access to the public network? 🤔
Please add that fact to your testing methodology.
Many python modules test suite already support pytest mark
networkto allow easy skip all units requiring access to the public network.lieryan commentedon Feb 23, 2024
And that is perfectly fine. Again, as I keep saying, the test suite is not downloading anything; installing using a package manager does not necessarily entail downloading off public network. The .whl package that we need is included as part of the test suite, pip do not require network access to install a wheel that has no dependency. All that is needed is installing a wheel file that is included in the test suite, in an environment that is controlled by the test suite.
Rope is a developer tooling library, it analyses the packages you have installed into your python environment to provide the autoimport/autocompletion services. To a program like rope, the content of the project's venv is input files, not code dependencies. That's the important distinction that you seem to be missing here.
We need to create a directory containing files that looks like a venv folder that users would have set up for their Python project because that's the input file for a developer tool. I'm not quite sure what else we can do here short of just poorly reimplementing what venv and pip does as test setup code.