Repository navigation
"Intermittent" import error when checking a module that import another module which imports a 3rd party package #7276
Description
Hi,
Sorry for the title. I try to explain myself better.
I have found a strange behaviour in mypy related to an import error in a imported module. That is, an error in a module 'b' importing a module 'c' while checking module 'a', which imports 'b'. 'c', of course, is a 3rd party package.
In such a case, mypy reports nothing on a first round. But if it is run again, it reported a error: Cannot find module named 'c'. On a third run, nothing is reported again, while on a fourth run the error re-appears, and so on...
This happens even when the type check has been disabled in the import line.
A possible scenario to reproduce this "bug" is as follows:
-
Create a 'strange_report' package (with an empty
__init__.py). -
Create a subpackage called 'a' inside 'strange_report' (with an empty
__init__.pytoo). -
Write an
a.pymodule (importer module) such as:
from a.b import B
b = B() # just for visually check that a.py was run
- Write a
b.pymodule (imported module)
from ruamel.yaml import YAMLError # type: ignore
class B:
def __init__(self):
print(f"New 'B' object {self} has been created")
-
Open terminal and change to directory 'strange_report'.
-
Module
a.pycan be run withpython -m a.a -
If you run
mypy a/a.pyfor the first time, you do not get any reports (however, directory '.mypy_cache' is created). -
But if you run
mypy a/a.pyonce again, you get an error:
madera:strange_error jv$ mypy a/a.py
a/b.py:1: error: Cannot find module named 'ruamel'
a/a.py:1: error: Cannot find module named 'ruamel'
a/a.py:1: note: See https://mypy.readthedocs.io/en/latest/running_mypy.html#missing-imports
- You can repeat 7. and 8. again and again with similar results. Same results are gotten with or without
# type: ignore.
However, if cache directory '.mypy_cache' is deleted before running mypy every time, it does not report anything.
Curiously enough, if you check mypy a/b.py it consistently reports nothing.
I am running on:
- macOS HIgh Sierra 10.13.6
- Python 3.7.4
- mypy 0.720
- ruamel.yaml 0.16.0 (so, yes, ruamel.yaml is installed; I use this 3rd party package because it is causing the bug in my system; no any particular purposes, I imagined that same bug can appeared with similar packages but cannot confirm that)
- Any mypy configuration beyond the default one.
Thank for your attention. I hope that will be helpful.
Activity
ilevkivskyi commentedon Aug 12, 2019
Thanks for reporting! Does this also happen with
--ignore-missing-imports?Quibi commentedon Sep 9, 2019
Oh, I am sorry. I missed your request.
mypy a/a.py --ignore-missing-importsdoes not raised the error.Indeed, this is even more puzzling since according to documentation:
Do not
--ignore-missing-importsCLI argument and# type: ignoreshould cause the same error (or any error at all)?Thank you.
jsonmcfall commentedon Oct 3, 2019
We hit this error at least several times a day.
It seems to alway show up with:
AssertionError: Cannot find module for ruamel
JukkaL commentedon Oct 7, 2019
Does using
--no-incrementalget rid of the issue?Quibi commentedon Oct 7, 2019
@jmcfallargo Of course, you must install ruamel.yaml first: https://pypi.org/project/ruamel.yaml/
It is just an example. I suspect that this issue can appear with other packages too, but I could not confirm that -- so I use this one in particular in my example.
Quibi commentedon Oct 7, 2019
@JukkaL No issues with
mypy a/a.py --no-incremental(in mypy 0.730). It is the same as deleting the cache by hand if I correctly understand the documentation. But this does not solve the issue, does not it?Thank you.
JukkaL commentedon Oct 8, 2019
Correct.
Yeah, it doesn't solve the issue.
I'm increasing priority, at least until we better understand why (or how frequently) this happens, since incremental mode issues can be painful to diagnose and work around.
Quibi commentedon Oct 10, 2019
Thank you, @JukkaL Sorry for not being able to provider further insights on this issue.
asfaltboy commentedon Jan 16, 2020
I think what makes ruamel.yaml package "special" is that it uses the
ruamelnamespace (namely a directory in e.gsite-packageswith no__init__.py.Could it be that we're not using the PEP 420 traversal (added by Barry Warsaw in PR #7219) since we're not targetting this package directly?
Back Ref: I had reported a similar issue with ruamel here: https://bitbucket.org/ruamel/yaml/issues/328/mypy-complains-that-ruamel-is-missing-type
domdfcoding commentedon Oct 23, 2020
For future reference the issue @asfaltboy linked to is now at https://sourceforge.net/p/ruamel-yaml/tickets/328/
Make --namespace-packages the default
lint.sh: run mypy in non incremental mode
hauntsaninja commentedon Aug 6, 2021
Fixed by #10937
Make --namespace-packages the default
[TECH-491] Attempt to solve intermittent import mypy error
[TECH-491] Attempt to solve intermittent import mypy error
[TECH-491] Attempt to solve intermittent import mypy error
[TECH-491] Attempt to solve intermittent import mypy error
sigma67 commentedon Mar 15, 2024
I am still seeing this issue with
ruamelpulled in via another dependency.mypy 1.9.0--no-incrementalfixes itWorkaround for mypy alternating between understanding ruamel and not
[mypy] Add `--no-incremental` flag to temporarily resolve mypy flake
[roll] Roll fuchsia [mypy] Add `--no-incremental` flag to temporarily…
hauntsaninja commentedon Mar 21, 2025
See #12664 for current issues with
ruamel.yamlUpdate pre-commit hooks versions