Repository navigation
locale.strxfrm raises MemoryError #51329
Description
| BPO | 7080 |
|---|---|
| Nosy | @loewis, @pitrou |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2009-10-19.19:49:59.948>
created_at = <Date 2009-10-07.23:01:41.779>
labels = ['extension-modules', 'type-crash']
title = 'locale.strxfrm raises MemoryError'
updated_at = <Date 2009-10-19.19:49:59.946>
user = 'https://bugs.python.org/drukker'bugs.python.org fields:
activity = <Date 2009-10-19.19:49:59.946>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2009-10-19.19:49:59.948>
closer = 'pitrou'
components = ['Extension Modules']
creation = <Date 2009-10-07.23:01:41.779>
creator = 'drukker'
dependencies = []
files = ['15078', '15158', '15167']
hgrepos = []
issue_num = 7080
keywords = ['patch']
message_count = 10.0
messages = ['93728', '94079', '94085', '94092', '94186', '94242', '94254', '94255', '94256', '94258']
nosy_count = 3.0
nosy_names = ['loewis', 'pitrou', 'drukker']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue7080'
versions = ['Python 3.1', 'Python 3.2']
Activity
drukker commentedon Oct 7, 2009
The strxfrm function in the locale module can potentially raise a
MemoryError.
The failing malloc is in Modules/_localemodule.c, line 291.
This is because the variable n0 of type Py_ssize_t is passed to
PyArg_ParseTuple, which expects an int when PY_SSIZE_T_CLEAN is not defined.
Patch attached, which also fixes an unrelated memory leak.
drukker commentedon Oct 15, 2009
Could someone triage this? And review my simple fix? I think it should
be a release blocker.
It seems to me to be a problem on all 64-bit non-Windows platforms, or
any platform if using UCS-4. The 4 high-order bytes of an 8-byte
Py_ssize_t variable are not initialized.
Thanks.
pitrou commentedon Oct 15, 2009
Nice catch. I wonder why not all of our modules are PY_SSIZE_T_CLEAN in
py3k.
This function is not actually tested, could you add a test to the
regression suite?
drukker commentedon Oct 15, 2009
All right.
The function strcoll also isn't tested (except for issue bpo-3303). I'll
look into that one, too.
drukker commentedon Oct 17, 2009
I've added the tests.
I found that on Windows, strxfrm has been unavailable in py3k since
r61306, because of an omission in PC/pyconfig.h. (cf. patch)
In determining whether I should first test for the existence of strcoll
(or strxfrm) in the locale module, as was done in TestMiscellaneous, I
found this is no longer needed in py3k since r61339. (cf. patch) (In
python2 such checks are still necessary.)
drukker commentedon Oct 19, 2009
The tests in TestEnUSCollation I added don't work for all encodings
(e.g. Asian ones, or ASCII). Now checked for encodings which are known
to work.
Found and fixed a bug (result not returned) in getpreferredencoding in
Lib/locale.py.
This test is skipped on Mac, since r30377. I've tested it on 10.3
(PPC), and it passed, so I enabled it again.
I did, however, find that FreeBSD and Mac OS X have known bugs in
wcscoll and wcsxfrm: http://www.freebsd.org/cgi/man.cgi?query=wcsxfrm
(cf. patch)
pitrou commentedon Oct 19, 2009
Nice work, thanks!
One detail: when giving you credit, who should I mention? just "egreen"?
drukker commentedon Oct 19, 2009
I've entered my real name.
pitrou commentedon Oct 19, 2009
Ok, thanks.
pitrou commentedon Oct 19, 2009
The patch is now committed to py3k (r75539) and 3.1 (r75541), closing.