Repository navigation
Spurious mypy errors due to incomplete third party stubs #708
Description
The python2 stubs for both sqlalchemy and boto are incomplete, which is totally fine! I hope to be able to contribute some time to building them out.
What I think may be a problem is that mypy assumes that the stubs are complete, and so will report that an invocation of a perfectly valid function is an error. Would it be possible to designate various stubs and/or subdirectories as "open"/"incomplete", such that the behavior becomes stub_file.get(key, Any) instead of the current (and correct when stubs are complete) stub_file[key]?
Activity
[-]Spurious mypy failures due to incomplete third party stubs[/-][+]Spurious mypy 0.4.6 failures due to incomplete third party stubs[/+]snakescott commentedon Nov 24, 2016
For a repro, you can run
mypy --py2 --fast-parser -sonfrom sqlalchemy.orm import Load. When I run it, I get an errorerror: Module 'sqlalchemy.orm' has no attribute 'Load'While I understand that improving stubs is the right long term fix for this class of issues, it would be nice to be able to choose to treat Load as Any in the meantime.
[-]Spurious mypy 0.4.6 failures due to incomplete third party stubs[/-][+]Spurious mypy 0.4.6 errors due to incomplete third party stubs[/+]gvanrossum commentedon Nov 24, 2016
I don't think I like this. If a stub is wrong, it's just wrong -- we should fix the stub, not try to tweak mypy to deal with it. The first version of a stub could be very simple -- usually the output from stubgen is good enough (sometimes with small tweaks). That'll define all classes and methods, at least.
(Update: also I don't like having 0.4.6 in the title. This will presumably be an ongoing problem.)
[-]Spurious mypy 0.4.6 errors due to incomplete third party stubs[/-][+]Spurious mypy errors due to incomplete third party stubs[/+]snakescott commentedon Nov 24, 2016
As a user who has not dived in to typeshed, stubgen, etc development, my feedback would be that the initial stub versions anecdotally do not seem to be good enough.
I've been working with mypy over the past week to get my Python 2 codebase "under test". In this phase I want to make as little change as possible to both my codebase and the underlying toolchain -- my goal is to bring something like https://github.com/zulip/zulip/blob/master/tools/run-mypy online and have it pass.
I ended up hitting stub-related false positive errors during my initial passes and resorted to running with a
--custom-typeshed-dirand removingbotoandsqlalchemyfromthird_party/2.I'm happy to close this issue until I have time to work with typeshed, stubgen, and friends and am able to have a more informed discussion?
gvanrossum commentedon Nov 24, 2016
That work-around is a pretty heavy hammer. I am guessing this is happening in too many places in your codebase to consider just putting
# type: ignoreon the offending lines?As a better solution, PEP 484 states that a top-level
def __getattr__(name) -> Any: ...in a stub should mean that the stub is incomplete. (I'd completely forgotten about this, sorry.) See also #5. I don't think mypy currently supports this, and I can't find an existing issue for that, soI propose to use this issue to track that featurewe should open an issue there.snakescott commentedon Nov 24, 2016
I'll rerun mypy without --custom-typeshed-dir and report back with my findings about error count (re
# type: ignore).One issue with this approach (I think?) is that it requires developers to be able to distinguish legitimate type errors (we want to fix!) from incomplete stub errors (we want to ignore!). As a PEP-484 evangelist I feel like that's a reasonable bar for me, but not sure it is one I want to impose on my entire team right off the bat.
gvanrossum commentedon Nov 24, 2016
snakescott commentedon Nov 24, 2016
oh, definitely! But heavy use of SQA and boto in our codebase + incomplete stubs can increase those odds. Rare stubs false positives should be fine, but there's some tipping point where too high a false positive rate becomes problematic.
I am hoping to run in "easy mode" as our team warms to mypy. The more of our e2e workflows I put in place, the lower our imprecision percentage is, and the more value folks see from type annotations in IDEs and CI, the more I expect people to be willing to invest in typeshed fixes &etc.
gvanrossum commentedon Nov 25, 2016
Fair enough. Curious about your total LoC count. FWIW the sqlalchemy stubs we currently have came out of a similar situation with Zulip...
snakescott commentedon Nov 25, 2016
I'll try to provide whatever stats I can (LoC, frequency of errors, etc) by early next week.