Make BytesIO inherit from BufferedIOBase. - #4082

The error messages from mypy selftest are quite concerning: test-data/stdlib-samples/3.2/test/test_base64.py:60: error: Argument 2 to "decode" has incompatible type "BytesIO"; expected "IO[bytes]". That means BytesIO is no longer compatible with IO[bytes], which seems likely to break a lot of real-world code using BytesIO. What happens if you make it inherit from IO[bytes] as well as BufferedIOBase?

I'll try that. I removed the explicit inheritance from BinaryIO because other classes there which should be compatible (afaik), such as BufferedReader don't have it.

I'm not really sure how that works, though. I'm unsure how mypy (or other type-checkers) handle that since AFAIK IO is not a Protocol.

stdlib/3/io.pyi:80: error: Definition of "__enter__" in base class "IOBase" is incompatible with definition in base class "IO"

I find that a bit weird, since IOBase's __enter__ is typed as

    def __enter__(self: _T) -> _T: ...

Where _T is _T = TypeVar('_T', bound=IOBase), thus for BytesIO it should return BytesIO, which by definition now is IO[bytes], which should therefore satisfy the definition from IO:

    @abstractmethod
    def __enter__(self) -> 'IO[AnyStr]':
        pass

I mean, at least in my mind, it should. Maybe I'm misinterpreting how mypy computes it.

Can you try putting def __enter__(self: _T) -> _T: ... explicitly in BytesIO?

Now there's just a minor discrepancy in the argument name of seek, which should be __pos instead of __offset.

The argument is positional-only though, so I'm not quite sure why stubtest complains about it. @hauntsaninja is that intentional?

Maybe that's because it should be positional-only in the stub and isn't? It seems it's only positional-only at runtime.

Also there are other issues on Python 3.5 and 3.6 apparently:

error: io.BytesIO.read1 is inconsistent, stub argument "__size" has a default value but runtime argument does not

Stub: at line 66

def (self: io.BufferedIOBase, builtins.int =) -> builtins.bytes

Runtime:

def (self, size, /)

error: io.BytesIO.readline is inconsistent, runtime argument "size" has a default value of type None, which is incompatible with stub argument type builtins.int

Stub: at line 47

def (self: io.IOBase, builtins.int =) -> builtins.bytes

Runtime:

def (self, size=None, /)

Maybe those should actually be Optional[builtins.int]?

Yeah, it's intentional that stubtest complains about positional-only argument names in this case. In general, it's more permissive for positional-only args, but will complain if the names are totally different. The exact logic is https://github.com/python/mypy/blob/10195b8d3e1fa7a94f9a9c28b9cd10770f1c449a/mypy/stubtest.py#L305 Agreed that it's annoying in this case and that we should whitelist by adding io.BytesIO.seek to tests/stubtest_whitelists/py3_common.txt. In the medium term, I want to make stubtest check inherited methods more permissively.

@tarcisioe The double underscore prefix in the stub is interpreted to mean the argument positional-only.

For the read1 error, I think you might have to override, conditional on sys.version_info.

For the readline error, at least based on _pyio, it could make sense to change IOBase.readline to __size: Optional[int] = ... (https://github.com/python/cpython/blob/b45af1a5691e83b86321fc52d173f66cf891ce5f/Lib/_pyio.py#L548)

I'll tackle those ASAP, @hauntsaninja! Thanks for the explanations.

@JelleZijlstra @hauntsaninja I had to make some other changes, but as far as I was able to check. they are consistent. Can you guys please take a look?

I think all is done now.

Thanks for getting this tricky change through, @tarcisioe !