Repository navigation
Add typing.py class describing a PEP 3118 buffer object #71688
Description
| BPO | 27501 |
|---|---|
| Nosy | @gvanrossum, @rhettinger, @jstasiak, @ilevkivskyi, @dmoisset |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2016-07-12.18:07:23.800>
labels = ['type-feature', 'library']
title = 'Add typing.py class describing a PEP 3118 buffer object'
updated_at = <Date 2020-05-19.18:11:43.834>
user = 'https://github.com/dmoisset'bugs.python.org fields:
activity = <Date 2020-05-19.18:11:43.834>
actor = 'gvanrossum'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2016-07-12.18:07:23.800>
creator = 'Daniel Moisset'
dependencies = []
files = ['43719']
hgrepos = []
issue_num = 27501
keywords = ['patch']
message_count = 8.0
messages = ['270261', '270315', '270321', '270323', '270334', '270412', '270437', '357522']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'rhettinger', 'jstasiak', 'levkivskyi', 'Daniel Moisset']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27501'
versions = ['Python 3.6']
Activity
dmoisset commentedon Jul 12, 2016
The buffer protocol is a low level C protocol, but even if it doesn't expose a specific python API, it's currently not possible to say "this object implements the buffer protocol" in Python
This might be useful for some applications, including PEP-484 typing hints. It would be useful to have a collections.abc.Buffer which already registers the types that already support the protocol (bytes, strings, array.array, struct, mmap, ctype arrays/pointers, etc)
rhettinger commentedon Jul 13, 2016
I'll defer to typing gurus as to whether something like this might make sense in typing.py, but it doesn't make much sense as a collections.abc without an API exposed in pure python and without useful mixin methods. By way of comparison, consider that the named tuple protocol is in typing.NamedTuple but not in collections.abc; instead, we just use _fields to recognize it when the need arises (which is almost never).
I would like collections.abc to remain a place for ABCs that can be usefully subclassed, that provide guaranteed behaviors, and that are in fact related to collections. The module's utility will be impaired if it becomes a dumping ground for miscellaneous type hinting concepts and registrations.
Early on there was some experimentation with the "one-trick ponies" in collections.abc such as Callable and Hashable that turned-out to almost never be used in practice, so we don't want to continue to that line of development when there are more promising avenues such as typing.py.
dmoisset commentedon Jul 13, 2016
Thanks for the feedback. Just for the context, I opened this issue also based on this email: http://permalink.gmane.org/gmane.comp.python.devel/156411
Probably there are two different discussions to have here:
Is the "Buffer" a useful ABC to have even if there's no python API involved? My thoughts (that come from working with typing and PEP-484 and numpy) are "yes"
Where is the right place to put it? I suggested collections.abc after seeing the thread I linked, although I wouldn't mind if this was in typing.py or somewhere else.
rhettinger commentedon Jul 13, 2016
Since collections.abc is the wrong place, I'll change the title of this to typing and see if any of the typing aficionados think it is sufficiently useful (they are deliberately focusing on common use cases first since the whole static typing endeavor is exploratory and this limits of what should be done and what isn't really needed or useful isn't yet well understood).
[-]Create a collections.abc class that describes PEP 3118 buffer[/-][+]Add typing.py class describing a PEP 3118 buffer object[/+]dmoisset commentedon Jul 13, 2016
OK. Just to give an obvious example of a place where this would be useful for annotations is in the argument for the memoryview() builtin (currently declared with "# TODO arg can be any obj supporting the buffer protocol" in the standard library typesheds)
dmoisset commentedon Jul 14, 2016
As a description of the idea I'm attaching a rough patch with an implementation. Some tests fail because of the following:
I didn't find a good place to register the Buffer ABC for mmap.mmap, and array.mmap; the modeules themselves are C extensions (and it's possible but weird to register the ABC from the C API); I could import those in typing.py but then I'd make typing depend from mmap and array which also feels wrong; so I didn't do that and the related tests that I added fail.
I had a similar situation with io.BytesIO. In that case there's a python module and I'm doing the abc.register there, but now io imports typing, and the test_site fails because many new modules (typing and its dependencies) are imported on startup. The alternative is to make typing depend on io (in this case it could be a reasonable idea)
After these, I'm feeling that typing.py may not be the best place for this (but given that collections.abc isn't either, I'm not sure where's the right place)
rhettinger commentedon Jul 14, 2016
Looking at the patch, I'm thinking that this endeavor isn't at all worthwhile. We don't need an ABC for everything especially if there in no pure python API involved. I recommend closing this one until there is a proven need and a clear path forward.
ilevkivskyi commentedon Nov 26, 2019
Cross-ref to the typing issue python/typing#593. It looks like there is some interest in this feature.
JelleZijlstra commentedon Apr 25, 2022
I just posted PEP-688 proposing a version of this idea.
AlexWaygood commentedon Mar 15, 2023
@JelleZijlstra, do we still need to keep this issue open now that we have #102500?
JelleZijlstra commentedon Mar 15, 2023
Agree, closing as a duplicate and let's use #102500 to track the implementation of PEP 688.