Repository navigation
email.header.decode_header sometimes returns bytes, sometimes str #65691
Description
| BPO | 21492 |
|---|---|
| Nosy | @warsaw, @ezio-melotti, @bitdancer, @berkerpeksag, @louisabraham |
| Dependencies |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2014-05-13.09:08:05.923>
labels = ['3.7', '3.8', 'expert-email', '3.9', 'type-feature', 'library', 'docs']
title = 'email.header.decode_header sometimes returns bytes, sometimes str'
updated_at = <Date 2019-06-07.18:44:38.667>
user = 'https://bugs.python.org/jimminter'bugs.python.org fields:
activity = <Date 2019-06-07.18:44:38.667>
actor = 'terry.reedy'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Library (Lib)', 'email']
creation = <Date 2014-05-13.09:08:05.923>
creator = 'jim_minter'
dependencies = ['6302']
files = []
hgrepos = []
issue_num = 21492
keywords = []
message_count = 5.0
messages = ['218419', '218442', '218450', '218451', '344522']
nosy_count = 7.0
nosy_names = ['barry', 'ezio.melotti', 'r.david.murray', 'docs@python', 'berker.peksag', 'jim_minter', '[email protected]']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21492'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9']
Activity
jimminter commentedon May 13, 2014
I may well be wrong, but I believe it's erroneous that in the second example above, b'foo' is returned instead of the expected 'foo'.
berkerpeksag commentedon May 13, 2014
email.header.decode_header() implements rfc-2047 and the "foo" header doesn't match the syntax described in rfc-2047 (see "2. Syntax of encoded-words").
See the code for more information:
See the "6.1. Recognition of 'encoded-word's in message headers" section at http://www.rfc-base.org/txt/rfc-2047.txt
bitdancer commentedon May 13, 2014
The error is actually the first case returning string rather than bytes. See bpo-6302.
bitdancer commentedon May 13, 2014
Hmm. It looks like we decided that we couldn't fix the behavior for backward compatibility reasons. In 3.4 you can use the new email policies to get automatic, correct stringification of headers.
ezio-melotti commentedon Jun 4, 2019
If we can't fix the behavior, it should at least be documented.
Currently the docs says "This function returns a list of (decoded_string, charset) pairs containing each of the decoded parts of the header.". One could assume that this means that a Unicode string is returned, but and as far as I can tell, "decoded_string" means decoded from the format used by the header, not from bytes -- in fact the example below shows a byte string.
bpo-24797 suggest an alternative solution, but there is no indications about it in the docs except an easy-to-miss note about the new API at the top.
Coincidentally as I was reporting this issue I also found the recently opened bpo-37139. There are also a few other reports: bpo-24797, bpo-37139, bpo-32975, bpo-6302, bpo-4661.
If this method is not actually deprecated, I would document the current behavior (i.e. sometimes it returns bytes, sometimes unicode -- bonus points if there's a simple rule to predict which one), explain that it exists for legacy/backward-compatibility reasons, and point to the alternatives.
FWIW here are 3 more samples that show the inconsistency.