Repository navigation
PyUnicode_FSDecoder() accepts arbitrary iterable #70941
Description
| BPO | 26754 |
|---|---|
| Nosy | @pitrou, @vstinner, @pjenvey, @vadmium, @serhiy-storchaka |
| Dependencies | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-08-14.06:30:31.064>
created_at = <Date 2016-04-14.08:06:10.499>
labels = ['interpreter-core', 'type-bug']
title = 'PyUnicode_FSDecoder() accepts arbitrary iterable'
updated_at = <Date 2016-08-14.06:30:31.063>
user = 'https://github.com/serhiy-storchaka'bugs.python.org fields:
activity = <Date 2016-08-14.06:30:31.063>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-08-14.06:30:31.064>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2016-04-14.08:06:10.499>
creator = 'serhiy.storchaka'
dependencies = ['26800']
files = ['43340', '43451']
hgrepos = []
issue_num = 26754
keywords = ['patch']
message_count = 9.0
messages = ['263378', '263388', '263394', '263396', '263695', '268213', '268795', '268798', '272107']
nosy_count = 6.0
nosy_names = ['pitrou', 'vstinner', 'pjenvey', 'python-dev', 'martin.panter', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue26754'
versions = ['Python 3.6']
Activity
serhiy-storchaka commentedon Apr 14, 2016
PyUnicode_FSDecoder() accepts not only str and bytes or bytes-like object, but arbitrary iterable, e.g. list.
Example:
I think accepting arbitrary iterables is unintentional and weird behavior.
pitrou commentedon Apr 14, 2016
I agree this doens't make sense.
vadmium commentedon Apr 14, 2016
I agree it is a bit strange. It looks like it is a victim of PyBytes_FromObject() doing more than it says; its documentation only mentions the buffer protocol, not accepting iterables.
serhiy-storchaka commentedon Apr 14, 2016
PyUnicode_FSDecoder() is used in following functions in the stdlib:
This is behavior of PyUnicode_FSDecoder() from the start (bpo-9542). All above functions accepted only str in 3.1, thus accepting bytes object and others was new feature.
None tests are failed if reject non-str and non-bytes argument in PyUnicode_FSDecoder(). But none tests are failed even if disable support of bytes argument (there is a lack of tests for bytes path).
What should we do?
Add a warning when the argument neither str nor supporting the buffer protocol.
Drop support of non-str and not supporting the buffer protocol arguments without a warning.
Drop support of non-str and not supporting the buffer protocol arguments without a warning, and add a warning when the argument neither str nor bytes.
Drop support of non-str and non-bytes arguments without a warning.
pjenvey commentedon Apr 18, 2016
See bpo-26800 for reasoning to go with #4
serhiy-storchaka commentedon Jun 11, 2016
Proposed patch makes PyUnicode_FSDecoder() rejecting arbitrary iterables.
python-dev commentedon Jun 18, 2016
New changeset 2e48c2c4c733 by Serhiy Storchaka in branch '3.5':
Issue bpo-26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
https://hg.python.org/cpython/rev/2e48c2c4c733
New changeset e18ac7370113 by Serhiy Storchaka in branch 'default':
Issue bpo-26754: PyUnicode_FSDecoder() accepted a filename argument encoded as
https://hg.python.org/cpython/rev/e18ac7370113
serhiy-storchaka commentedon Jun 18, 2016
Following patch deprecates the support of bytes-like objects (except bytes itself) in PyUnicode_FSDecoder() for consistency with bpo-26800.
python-dev commentedon Aug 6, 2016
New changeset 818f22f9ab02 by Serhiy Storchaka in branch 'default':
Issue bpo-26754: Undocumented support of general bytes-like objects
https://hg.python.org/cpython/rev/818f22f9ab02