Repository navigation
ctypes unittest crashes with libffi 3.4.2 #88722
Description
| BPO | 44556 |
|---|---|
| Nosy | @ned-deily, @dopplershift, @isuruf, @jakirkham, @h-vetinari, @erykoff |
| Superseder |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2021-11-21.00:16:41.337>
created_at = <Date 2021-07-03.01:27:44.161>
labels = ['ctypes', '3.8', 'type-crash', '3.11']
title = 'ctypes unittest crashes with libffi 3.4.2'
updated_at = <Date 2021-11-21.00:16:41.330>
user = 'https://bugs.python.org/zan'bugs.python.org fields:
activity = <Date 2021-11-21.00:16:41.330>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2021-11-21.00:16:41.337>
closer = 'ned.deily'
components = ['ctypes']
creation = <Date 2021-07-03.01:27:44.161>
creator = 'zan'
dependencies = []
files = []
hgrepos = []
issue_num = 44556
keywords = []
message_count = 5.0
messages = ['396885', '406626', '406627', '406631', '406695']
nosy_count = 7.0
nosy_names = ['ned.deily', 'Ryan May', 'isuruf', 'jakirkham', 'h-vetinari', 'erykoff', 'zan']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '45350'
type = 'crash'
url = 'https://bugs.python.org/issue44556'
versions = ['Python 3.8', 'Python 3.11']
Activity
zan commentedon Jul 3, 2021
Running the unittests after upgrading libffi to latest version 3.4.2 and recompiling python, leads to several tests in Lib/ctypes/test/ to crash with Abort: trap 6. This does not happen with version 3.3 of libffi.
Steps to reproduce:
git clone https://github.com/libffi/libffi
cd libffi
autoreconf -fvi
export CC=/usr/bin/clang
./configure
make
sudo make install # installs into /usr/local
git clone https://github.com/python/CPython
cd CPython
export CC=/usr/bin/clang
export CXX=/usr/bin/clang++
export CPPFLAGS="-I/usr/local/include"
export LDFLAGS="-L/usr/local/lib"
./configure --with-system-ffi
make
./python Lib/ctypes/test/main.py -v
...
test_callbacks (ctypes.test.test_as_parameter.AsParamPropertyWrapperTestCase) ... Abort trap: 6
If I repeat the above steps with 'git checkout v3.3' in libffi, all tests in ctypes complete successfully.
Tested with :
jakirkham commentedon Nov 19, 2021
We ran into the same issue in conda-forge ( conda-forge/python-feedstock#522 ).
The problem is Apple also supplies their own
libffi. However if the build scripts in CPython fail to find the user providedlibffi, they end up pulling the headers from Apple'slibffi, but the linker will link to the user providedlibffi. IOW mashing these two incompatiblelibffi's together. As result one gets crashes like the one illustrated in this bug.In conda-forge, we are resolving this by forcing our
pkg-configto be used to ensure we pick up the headers from ourlibffias well as the libraries. Other users may be able to workaround this issue by explicitly settingLIBFFI_INCLUDE_DIR.That said, it would be preferable to have a clear way to specify the
libffiused and ensure that Apple's one doesn't get accidentally pulled in. If this exists and we are just missing these details, some pointers to this effect would be very helpful.erykoff commentedon Nov 19, 2021
To add a bit more color, I don't think you can explicitly set LIBFFI_INCLUDE_DIR, it's set by the configure script explicitly. The problem is that some time between python 3.9 and 3.10 the configure script has (at least in our system) failed to figure out the $PKG_CONFIG variable correctly. It should be set to the executable with the full path name, as is done with 3.9, but now it is set to just the executable name. When that happens the library location check fails.
isuruf commentedon Nov 20, 2021
Duplicate of https://bugs.python.org/issue45350
ned-deily commentedon Nov 21, 2021
Yes, this does appear to be a result of the problem described in bpo-45350. As noted there, the problem was caused by running autoreconf to regerenerate the ./configure script in an environment where pkg-config wasn't installed which caused some pkg-config code to be removed from ./configure. And, unfortunately, the ./configure script included with the v3.10.0 release still had that problem. You can copy an updated version from the current 3.10.0 branch or wait for 3.10.1 or you can try running autoreconf yourself in the source directory.