Repository navigation
Inconsistent behavior w.r.t missing builtins #14547
Description
Issue
Some of the builtins fixtures are sparse to the point that they're missing even the declaration of dict, list, tuple etc. types. The austerity is justified by improved tests performance, and it's understood that specific tests can specify a broader fixture without slowing down other tests.
However, some core features rely on the presence of those types, e.g.
- functions'
**kwargs - modules'
__annotations__ namedtuple's attributes like_field_types
I've observed 3 different strategies used to handle the types absence:
- crash (example:
**kwargshandling) - give up (example:
__annotations__not being added) - fallback to
object(example:NamedTupleattributes)
I think we should recommend and, ideally, consolidate on one strategy.
Motivation
Case in point: Recently while implementing #14526, I had to "play" a whack-a-mole with unrelated tests breaking. I've used **kw which required me to add dict to a number of builtins fixtures, which in turn caused __annotations__ to materialize where they didn't before, and failed some other tests.
I think ideally we shouldn't have this whack-a-mole, and tests should be less brittle.
Recommendation
Which strategy to adopt? While it's good practice to structure code to be more testable, I think it's normally discouraged to have code paths that are only due to testing. Since those builtins should universally exist in "production", I think crash should be our strategy.
Since code typically relies only on the presence of those classes (not their methods), we can put in place a rule (validated by tests) that any builtins fixture must define a baseline of types, even if empty. This shouldn't affect performance greatly, and provide a comfortable baseline.
Activity
JelleZijlstra commentedon Jan 28, 2023
The fixtures in general are a confusing pain point for developing mypy. It would be better if we can just use the typeshed stubs in tests.
As I understand it, the reason for using the fixtures is that typeshed's builtins.pyi is big and parsing it takes a long time. Perhaps we can come up with some clever form of caching to alleviate that concern.
ikonst commentedon Jan 28, 2023
I agree, that'll be a big win for contributors' ergonomics.
As an intermediate step I'd suggest eliminating the fallback strategies from the code, i.e. fail-fast.
Fail-fast on missing builtins (#14550)
ikonst commentedon Jan 30, 2023
Another case - falling back from
types.ModuleTypetobuiltins.object:mypy/mypy/checkexpr.py
Lines 392 to 398 in 91e8581
To exclude this special-case, we'd have to
import typesin all builtinsclass tuple: passto all builtinsThe
typesfixture is pretty small, so unless I'm suggesting "death by a thousand cuts", I think we should go ahead with this. This will also trigger fixing some tests that are obviously wrong.