Skip to content

email package should work better with unicode #44753

Closed
Closed
email package should work better with unicode#44753
Assignees
bitdancer
Labels
stdlibStandard Library Python modules in the Lib/ directorytopic-unicodetype-bugAn unexpected behavior, bug, or error

Description

@warsaw
warsaw
opened on Mar 21, 2007
BPO 1685453
Nosy @warsaw, @devdanzin, @merwok, @bitdancer

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = 'https://github.com/bitdancer'
closed_at = <Date 2010-12-27.17:04:58.053>
created_at = <Date 2007-03-21.18:39:03.000>
labels = ['type-bug', 'library', 'expert-unicode']
title = 'email package should work better with unicode'
updated_at = <Date 2011-12-05.18:14:12.309>
user = 'https://github.com/warsaw'

bugs.python.org fields:

activity = <Date 2011-12-05.18:14:12.309>
actor = 'r.david.murray'
assignee = 'r.david.murray'
closed = True
closed_date = <Date 2010-12-27.17:04:58.053>
closer = 'r.david.murray'
components = ['Library (Lib)', 'Unicode']
creation = <Date 2007-03-21.18:39:03.000>
creator = 'barry'
dependencies = []
files = []
hgrepos = []
issue_num = 1685453
keywords = []
message_count = 7.0
messages = ['31612', '84700', '84753', '100550', '124715', '148880', '148882']
nosy_count = 9.0
nosy_names = ['barry', 'ocean-city', 'ajaksu2', 'eric.araujo', 'r.david.murray', 'bgamari', 'l0nwlf', 'pebbe', 'sivang']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1685453'
versions = ['Python 2.6', 'Python 3.1', 'Python 2.7', 'Python 3.2']

Activity

  1. warsaw commented on Mar 21, 2007

    @warsaw
    warsaw
    on Mar 21, 2007
    MemberAuthor

    This is a catch-all issue for improving the email package's handling of unicode. For now, please add issues/problems you find with email & unicode to this tracker item.

    For example:

    MIMEText()'s first argument should accept a unicode if _charset is also given. It should not be necessary to manually encode the first argument into an 8-bit string.

  2. warsaw
    self-assigned this
    on Mar 21, 2007
  3. warsaw
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Mar 21, 2007
  4. warsaw
    self-assigned this
    on Mar 21, 2007
  5. warsaw
    added
    stdlibStandard Library Python modules in the Lib/ directory
    on Mar 21, 2007

  6. devdanzin commented on Mar 30, 2009

    @devdanzin
    devdanzinmannequin
    on Mar 30, 2009
    Mannequin

    Link to bpo-1681333, bpo-4487, bpo-1443875, bpo-1555842, bpo-4661, bpo-1078919, bpo-963906,
    bpo-1379416 and bpo-1368247.

  7. devdanzin
    added
    topic-unicode
    type-bugAn unexpected behavior, bug, or error
    on Mar 30, 2009

  8. ocean-city commented on Mar 31, 2009

    @ocean-city
    ocean-citymannequin
    on Mar 31, 2009
    Mannequin

    Probably these are related too. bpo-5259 bpo-5304

  9. pebbe commented on Mar 6, 2010

    @pebbe
    pebbemannequin
    on Mar 6, 2010
    Mannequin

    In Python 3.1.1, email.mime.text.MIMEText accepts an 8-bit charset, but not utf-8.

    I think you should not have to specify a charset. All strings are unicode now, so I think the package should choose an appropriate charset based on the characters in the text, us-ascii, some iso-8859 charset, or utf-8, whatever fits.

    Python 3.1.1 (r311:74480, Oct  2 2009, 11:50:52)                                                                                              
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2                                                                                 
Type "help", "copyright", "credits" or "license" for more information.                                                                        
>>> from email.mime.text import MIMEText                                                                                                      
>>> text = 'H\u00e9'                                                                                                                          
>>> msg = MIMEText(text, 'plain', 'iso-8859-1')                                                                                               
>>> print(msg.as_string())                                                                                                                    
Content-Type: text/plain; charset="iso-8859-1"                                                                                                
MIME-Version: 1.0                                                                                                                             
Content-Transfer-Encoding: quoted-printable                                                                                                   
                                                                                                                                              
H=E9                                                                                                                                          
>>> msg = MIMEText(text, 'plain', 'utf-8')                                                                                                    
Traceback (most recent call last):                                                                                                            
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 269, in set_charset                                                            
    cte(self)                                                                                                                                 
TypeError: 'str' object is not callable                                                                                                       
                                                                                                                                              
During handling of the above exception, another exception occurred:                                                                           
                                                                                                                                              
Traceback (most recent call last):                                                                                                            
  File "<stdin>", line 1, in <module>                                                                                                         
  File "/my/opt/Python-3/lib/python3.1/email/mime/text.py", line 30, in __init__                                                              
    self.set_payload(_text, _charset)                                                                                                         
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 234, in set_payload                                                            
    self.set_charset(charset)                                                                                                                 
  File "/my/opt/Python-3/lib/python3.1/email/message.py", line 271, in set_charset                                                            
    self._payload = charset.body_encode(self._payload)                                                                                        
  File "/my/opt/Python-3/lib/python3.1/email/charset.py", line 380, in body_encode                                                            
    return email.base64mime.body_encode(string)                                                                                               
  File "/my/opt/Python-3/lib/python3.1/email/base64mime.py", line 94, in body_encode                                                          
    enc = b2a_base64(s[i:i + max_unencoded]).decode("ascii")                                                                                  
TypeError: must be bytes or buffer, not str                                                                                                   
>>>
  10. warsaw
    assigned
    bitdancer
    and unassigned
    warsaw
    on May 5, 2010

  11. bitdancer commented on Dec 27, 2010

    @bitdancer
    bitdancer
    on Dec 27, 2010
    Member

    Now that we are primarily focused on Python3 development, collecting "unicode" issues is not really all that useful (at least not to me, and I'm currently doing the email maintenance), so I'm closing this. All the relevant issues are assigned to me anyway, so I'll be dealing with them by and by.

  12. bitdancer
    closed this as completedon Dec 27, 2010

  13. sivang commented on Dec 5, 2011

    @sivang
    sivangmannequin
    on Dec 5, 2011
    Mannequin

    I am having hard time parsing all the text/html and text/plain parts of a message, concatenating them into a string. I am thinking of writing some custom code to do manual handling of this...

    If this could be fixed that would be great. The issues are converting from and to ascii/unicode or whatever encoding/charset the part uses.

  14. bitdancer commented on Dec 5, 2011

    @bitdancer
    bitdancer
    on Dec 5, 2011
    Member

    That particular problem will get fixed in the next version of the email package (hopefully in Python3.3), but that isn't ready yet.

  15. ezio-melotti
    transferred this issue fromon Apr 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

stdlibStandard Library Python modules in the Lib/ directorytopic-unicodetype-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions