Repository navigation
Crash in _ctypes_alloc_callback #58562
Description
| BPO | 14354 |
|---|---|
| Nosy | @brettcannon, @ronaldoussoren, @amauryfa, @pitrou, @ned-deily, @meadori, @willingc |
| Superseder | |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2016-06-23.04:57:15.964>
created_at = <Date 2012-03-17.20:55:43.738>
labels = ['ctypes', 'type-crash']
title = 'Crash in _ctypes_alloc_callback'
updated_at = <Date 2016-06-23.09:30:48.986>
user = 'https://bugs.python.org/ogre'bugs.python.org fields:
activity = <Date 2016-06-23.09:30:48.986>
actor = 'berker.peksag'
assignee = 'none'
closed = True
closed_date = <Date 2016-06-23.04:57:15.964>
closer = 'willingc'
components = ['ctypes']
creation = <Date 2012-03-17.20:55:43.738>
creator = 'ogre'
dependencies = []
files = ['24919']
hgrepos = []
issue_num = 14354
keywords = []
message_count = 10.0
messages = ['156205', '156210', '156218', '156219', '156236', '156242', '156243', '156430', '192425', '269106']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'ronaldoussoren', 'amaury.forgeotdarc', 'pitrou', 'ned.deily', 'meador.inge', 'ogre', 'willingc']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '16575'
type = 'crash'
url = 'https://bugs.python.org/issue14354'
versions = ['Python 2.7', 'Python 3.2']
Activity
ogre commentedon Mar 17, 2012
I have reproduced this crash in Apple's default 2.7.1 python, and in 2.7.3 built from source myself. But only in release mode. If I rebuild 2.7.3 in debug, the crash goes away.
The attached file reproduces the issue, which has to do with a union containing multiple structs being used as a value for a callback argument. I've stripped it down as much as I can. Removing any fields from either the union or the Dice struct will cause this to no longer crash.
The original source of this is libtcod's python wrapper library, which seems to work fine on platforms other than Mac, though it had a lot of other problems on 64-bit systems before I got to this one. This issue may also be more 64-bit specific than Mac specific.
This is the callstack from Apple's python:
#0 0x00000001010c7712 in _ctypes_alloc_callback ()
#1 0x00000001010c4a1c in PyCData_AtAddress ()
#2 0x0000000100050afa in icu::DigitList::getDouble ()
#3 0x000000010000bd32 in PyObject_Call ()
#4 0x000000010008bf63 in ubrk_swap ()
#5 0x000000010008ecd8 in triedict_swap ()
#6 0x000000010008ed4d in triedict_swap ()
#7 0x00000001000a608f in ucnv_openStandardNames ()
#8 0x00000001000a614f in ucnv_openStandardNames ()
#9 0x00000001000a72a2 in ucnv_getUnicodeSet ()
#10 0x00000001000b72af in ucnv_getDisplayName ()
#11 0x0000000100000e88 in ?? ()
ogre commentedon Mar 17, 2012
I just built python 3.2.2 from source, and reproduced the issue there as well. Same location. Here's the slightly more informative stack trace from my release-with-symbols 3.2.2 build:
#0 _ctypes_alloc_callback (callable=0x7fff5fbfef20, converters=0x100000003, restype=0x7fff5fbfef20, flags=1606414112) at callbacks.c:432
#1 0x00000001010c5395 in PyCFuncPtr_new (type=0x7fff5fbfefc0, args=0x101010580, kwds=0x7fff5fbfefc0) at _ctypes.c:3372
#2 0x000000010004986c in type_call (type=0x1006ca940, args=0x101041b90, kwds=0x0) at typeobject.c:676
#3 0x0000000100008605 in PyObject_Call (func=0x1006ca940, arg=0x101041b90, kw=0x0) at abstract.c:2149
#4 0x000000010008afcc in do_call [inlined] () at /Users/ogre/src/Python-3.2.2/Python/ceval.c:4141
#5 0x000000010008afcc in PyEval_EvalFrameEx (f=0x7fff5fbff1c0, throwflag=1606414784) at ceval.c:3944
#6 0x000000010009131b in PyEval_EvalCodeEx (_co=0x7fff5fbff260, globals=0x0, locals=0x1066c8800000101, args=0x7fff5fbff260, argcount=1606414944, kws=0x7fff5fbff260, kwcount=0, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at ceval.c:3350
#7 0x000000010009139f in PyEval_EvalCode (co=0x101066828, globals=0x101066c88, locals=0x0) at ceval.c:767
#8 0x00000001000b0a21 in run_mod [inlined] () at /Users/ogre/src/Python-3.2.2/Python/pythonrun.c:1783
#9 0x00000001000b0a21 in PyRun_FileExFlags (fp=0x7fff7c677ee0, filename=0x101066828 "\002", start=0, globals=0x101066c88, locals=0x0, closeit=1, flags=0x7fff5fbff420) at pythonrun.c:1740
#10 0x00000001000b2992 in PyRun_SimpleFileExFlags (fp=0x7fff7c677ee0, filename=0x10104ac20 "testctypes.py", closeit=1606415200, flags=0x7fff5fbff360) at pythonrun.c:1265
#11 0x00000001000c45af in run_file [inlined] () at /Users/ogre/src/Python-3.2.2/Modules/main.c:297
#12 0x00000001000c45af in Py_Main (argc=1606415440, argv=0x7fff5fbff450) at main.c:692
#13 0x0000000100001522 in main (argc=17197096, argv=0x100609fe0) at python.c:59
converters doesn't seem to be pointing at valid data.
(gdb) p *converters
$2 = {
ob_refcnt = 3302829852670,
ob_type = 0xe0000000200
}
But, being an optimized build, it's hard to say (especially for me, having never debugged python itself before) if that's the real data or some optimizer-mangled version.
ned-deily commentedon Mar 17, 2012
This is likely related to bpo-13370 which documents test_ctypes failures when using either of the llvm-based compilers (clang or llvm-gcc) supplied with recent versions of Xcode 4. test_ctypes and your test both do not fail when Python is compiled with the standard (non-llvm) gcc-4.2 supplied in Xcode 3.2.6 (for OS X 10.6).
ogre commentedon Mar 18, 2012
Thanks for that. This does seem to be the case. I rebuilt with CC=gcc-4.2 and my short sample and the full library I took it from both work fine.
amauryfa commentedon Mar 18, 2012
I also noticed that the script crashes on Debian 64bit as well, versions 2.6 and 2.7.3rc2 at least, optimized builds.
So it's not a clang issue after all.
amauryfa commentedon Mar 18, 2012
There is a out-of-bounds error in Modules/_ctypes/libffi/src/x86/ffi64.c:
at line 225, classes[i + pos] can go outside the allocated memory for classes (MAX_CLASSES=4).
This code is not prepared to received "structures" with a small size (<32bytes) but where individual elements total more than 32bytes.
libffi support for unions is weak; at least ctypes should not use FFI_TYPE_STRUCT for unions.
amauryfa commentedon Mar 18, 2012
Not a Mac issue.
ogre commentedon Mar 20, 2012
It's maybe not directly relevant to fixing this, but I worked around it on the project where this came up by redefining dice as (c_int * 4) and col as (c_uint8 * 3) in the union, then using ctypes.cast to get those as pointers to the actual struct. That seems to work just fine for now.
ronaldoussoren commentedon Jul 6, 2013
The example uses unions, which libffi doesn't really support (see for example <https://github.com/atgreen/libffi/issues/33\>).
There is an issue in the python tracker about union support in ctypes: bpo-16575.
FWIW. I can still reproduce the issue with x86_64 on OSX with both the 2.7 and default branches, while i386 does not crash.
I intend to close this as a duplicate of bpo-16575 as that better explains the underlying problem.
willingc commentedon Jun 23, 2016
As per Ronald's last comment "close this as a duplicate of bpo-16575 as that better explains the underlying problem".
Closing this dated and duplicate issue.