Repository navigation
TextIOWrapper stub does not match stdlib documentation #6061
Description
The official documentation for TextIOWrapper implies that the underlying buffer is an instance of (or implements the interface of) io.BufferedIOBase, but typeshed annotates it as IO[bytes], leading to the following counterintuitive behavior:
# repro.py
import typing
from io import TextIOWrapper, BufferedIOBase
def badfunc(stream: BufferedIOBase) -> TextIOWrapper:
return TextIOWrapper(stream)
def goodfunc(stream: typing.IO[bytes]) -> TextIOWrapper:
return TextIOWrapper(stream)18:12 [snoopjedi@denton ~]
$ python3 -m mypy repro.py
repro.py:6: error: Argument 1 to "TextIOWrapper" has incompatible type "BufferedIOBase"; expected "IO[bytes]"
Found 1 error in 1 file (checked 1 source file)
This example is reduced from a question a user asked in the #python IRC channel on Libera.net, regarding this user code in rdflib.
Version information
18:12 [snoopjedi@denton ~]
$ python3 -m mypy --version
mypy 0.910
18:12 [snoopjedi@denton ~]
$ python3 -V
Python 3.8.10
Activity
aucampia commentedon Sep 21, 2021
See also: https://stackoverflow.com/questions/58394410/why-does-mypy-complain-about-textiowrapper-receiving-gzipfile-as-argument-1
aucampia commentedon Sep 21, 2021
Maybe related: #4146
srittau commentedon Sep 22, 2021
I/O types are a mess. We generally recommend to use tight protocols in argument positions to avoid these kind of issues. That said, I'd be open to an experimental PR that changes the buffer type to
BufferedIOBaseto judge the impact this change would have.aucampia commentedon Sep 22, 2021
Is there some place that the relationship between IO[bytes], IO[str], BufferdIOBase, RawIOBase and StringIOBase is defined? I know IO[...] is marked for deprecation, but I'm actually not sure what the intended replacement is. I would guess it is the ...IOBase types, but this is not made explicit. I think actually the reasoning for using IO[bytes] is that maybe TextIOWrapper works fine for both RawIOBase and BufferedIOBase, and IO[bytes] is less verbose than Union[RawIOBase,BufferedIOBase]. I will try dig into this deeper sometime to confirm.
srittau commentedon Sep 22, 2021
The general guideline is to use protocols for arguments and concrete types (derived from
IO, at least for now) for return types.IOand its sub-classesTextIOandBinaryIOare "protocol-like" type-only classes, so their relationship to the other classes is a bit hard to pin down.JelleZijlstra commentedon Sep 22, 2021
See also python/typing#829 for some discussion of this area.
aucampia commentedon Sep 25, 2021
Maybe somewhat related: python/mypy#11193
aucampia commentedon Sep 26, 2021
I think maybe the root of all problems here come from #6077, if that was resolved this would be somewhat inconsequential.
aucampia commentedon Sep 27, 2021
Given the response in #6077, I think this is then not a bug. But I think maybe some documentation to this effect would be good, as it was somewhat surprising behaviour to me at least.
aucampia commentedon Sep 27, 2021
I think this should be closed. This problem arose because I read the
open()documentation, and figured, if it says it returnsio.TextIOBase, I should type things like that. It is clear to me now that I should not. I guess there won't be any simple solutions to this conundrum anytime soon though, and probably the best interim solution will come from python/typing#851 - it would be nice if the official python docs forio.*IOBasemade it clear they should not be used for typing, but that will maybe be hard to achieve right now.Thanks for the help and information.