Skip to content

Require abstract classes in stubs to be explicitly marked - #5223

Merged
ilevkivskyi merged 8 commits into
python:masterfrom
JelleZijlstra:noabstract
Jun 27, 2018
Merged

ilevkivskyi merged 8 commits into
python:masterfrom
JelleZijlstra:noabstract

Conversation

@JelleZijlstra

@JelleZijlstra JelleZijlstra commented Jun 15, 2018 •

Copy link
Copy Markdown
Member

Fixes #3692.

In typeshed, we have had many bugs because of classes that were not meant to be abstract, but were inferred by mypy to be abstract because they didn't implement some methods from an abstract base class. I call such classes "accidentally abstract"; python/typeshed#1476 has a list of cases where this bug appeared.

It is hard to catch these bugs because they only show up in user code when somebody tries to instantiate the class, and when reviewing changes to typeshed you have to manually go over all methods to find instances of this bug. Therefore, I'd like to have mypy point out accidentally abstract classes automatically.

This PR throws an error if a class:

  • Has any abstract method;
  • Does not itself have any method decorated with @abstractmethod (i.e., all abstract methods are inherited);
  • Does not have ABCMeta as an explicit metaclass, or Protocol as an explicit base class;
  • Is defined in a stub file.
    This generates about five false positives in typeshed's Python 3 code, which we can fix by manually adding ABCMeta as a metaclass. On the other hand, there have been at least 30 accidentally abstract classes in typeshed that would have been caught by this patch, although I've fixed most of them by now.

This PR is going to fail tests for now because not all typeshed classes are fixed yet. If we're in agreement that this PR is acceptable, I will work on getting typeshed ready for it by adding the right ABCMeta metaclasses.

@ilevkivskyi

Copy link
Copy Markdown
Member

I totally like the idea in principle, but what to do with the test failures (mypy uses many test fixtures)? Maybe limit this to only typeshed stubs (in contrast to all stubs)? There is a corresponding flag in SemanticAnalyzer.

@JelleZijlstra

Copy link
Copy Markdown
Member Author

I think the failures in mypy's stubs won't be too hard to fix.

The is_typeshed_stub_file flag doesn't work for typeshed's mypy_test.py, which would be somewhat beside the point of this feature. Also, in general the feature should be useful for all stub packages, not just for typeshed.

@ilevkivskyi

Copy link
Copy Markdown
Member

I think this can be merged after you are done with typeshed updates and it is synced.

JelleZijlstra added a commit to JelleZijlstra/typeshed that referenced this pull request Jun 17, 2018
This is needed to make TextIOWrapper and a few classes that inherit
from it concrete in 3.3. I am actually not sure whether this method
exists at runtime; there's nothing in
https://docs.python.org/3/library/io.html#io.TextIOWrapper
that suggests it was added in 3.4. In any case, it hardly matters
since 3.3 usage should be very rare now.

This appears to be the last typeshed change needed to unblock
python/mypy#5223. @ilevkivskyi would you mind reviewing?
@JelleZijlstra JelleZijlstra mentioned this pull request Jun 17, 2018
@JelleZijlstra

Copy link
Copy Markdown
Member Author

Going to close and re-open to see if that forces Appveyor to re-run (Travis already passed).

@ilevkivskyi

Copy link
Copy Markdown
Member

All tests pass now, but I would wait until tomorrow to see if there any objections from @JukkaL

@JelleZijlstra

Copy link
Copy Markdown
Member Author

@JukkaL do you have any feedback here? (Also, thanks for fixing the merge Ivan.)

@ilevkivskyi

Copy link
Copy Markdown
Member

FWIW this PR passes testpr cleanly.

@ilevkivskyi

Copy link
Copy Markdown
Member

I assume the silence means no.

@ilevkivskyi
ilevkivskyi merged commit b316ac2 into python:master Jun 27, 2018
@JelleZijlstra
JelleZijlstra deleted the noabstract branch June 27, 2018 16:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Catch accidentally abstract classes

2 participants