Repository navigation
llvm-gcc-4.2 miscompiles Python (XCode 4.1 on Mac OS 10.7) #57450
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ned-deily'
closed_at = <Date 2012-09-05.15:19:35.865>
created_at = <Date 2011-10-21.19:49:19.595>
labels = ['OS-mac', 'build']
title = 'llvm-gcc-4.2 miscompiles Python (XCode 4.1 on Mac OS 10.7)'
updated_at = <Date 2012-09-05.15:19:35.864>
user = 'https://bugs.python.org/OlegPlakhotnyuk'bugs.python.org fields:
activity = <Date 2012-09-05.15:19:35.864>
actor = 'ronaldoussoren'
assignee = 'ned.deily'
closed = True
closed_date = <Date 2012-09-05.15:19:35.865>
closer = 'ronaldoussoren'
components = ['Build', 'macOS']
creation = <Date 2011-10-21.19:49:19.595>
creator = 'Oleg.Plakhotnyuk'
dependencies = []
files = ['23490', '24206', '26001', '26004']
hgrepos = []
issue_num = 13241
keywords = ['patch', 'needs review']
message_count = 49.0
messages = ['146120', '146121', '146122', '146123', '146124', '146136', '146137', '146223', '146233', '146234', '146240', '146279', '146433', '146444', '146517', '146519', '146521', '147006', '147014', '147018', '147265', '147267', '147275', '147279', '147974', '149410', '150278', '150992', '150994', '151006', '151007', '151093', '151099', '151116', '152634', '153981', '154004', '160064', '160066', '160081', '160099', '160353', '162090', '162093', '162728', '162742', '169831', '169833', '169875']
nosy_count = 19.0
nosy_names = ['gvanrossum', 'barry', 'ronaldoussoren', 'vstinner', 'ned.deily', 'trent', 'ezio.melotti', 'thomaspinckney3', 'eric.araujo', 'michael.foord', 'maker', 'Yury.Selivanov', 'Julian', 'Anthony.Kong', 'hynek', 'Nam.Nguyen', 'Oleg.Plakhotnyuk', 'xapple', 'Anton.Korobeynikov']
pr_nums = []
priority = 'high'
resolution = 'wont fix'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue13241'
versions = ['Python 3.3']
Activity
OlegPlakhotnyuk commentedon Oct 21, 2011
Trying to build latest development revision:
localhost:repo family$ hg summary
parent: 73044:a985d733b3a3 tip
bpo-12753: Add support for Unicode name aliases and named sequences.
branch: default
commit: (clean)
update: (current)
Constantly getting this error:
./python.exe -SE -m sysconfig --generate-posix-vars
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Assertion failed: (compact->utf8_length == 0), function _PyUnicode_CheckConsistency, file Objects/unicodeobject.c, line 399.
make: *** [sysconfig] Abort trap
However, tag v3.2.2 seems to build successfully.
OS: Mac OS X 10.6.8
GCC: 4.2.1
If you have troubles reproducing this issue, feel free to ask any information I can provide that can help.
Thank you in advance!
OlegPlakhotnyuk commentedon Oct 21, 2011
Oh yes.
I am building with this command line:
./configure --with-pydebug && make -j2
vstinner commentedon Oct 21, 2011
It is a compiler bug in GCC 4.2.1 with -O3. Try to compile attached unicode.c program with gcc -O3. The correct result is:
With gcc 4.2.1 and -O3, you get "is compact ascii? 1".
vstinner commentedon Oct 21, 2011
The bug has been fixed in GCC, try a more recent version using ./configure CC=gccX.Y.
ezio-melotti commentedon Oct 21, 2011
It should also work without --with-pydebug, or with a different compiler.
ned-deily commentedon Oct 21, 2011
I believe this was a bug specific to the llvm-gcc compiler shipped with Xcode 4.1 and possibly Xcode 4.0. I do not have Xcode 4 on OS X 10.6 but haypo's test does fail with the llvm-gcc 4.2.1 that comes with Xcode 4.1 on 10.7. Can you confirm exactly which compiler you are using, i.e. the complete --version output from the compiler used in your python build? If you have not installed Xcode 4.2, one solution is to use CC=gcc-4.2 which uses the plain gcc-4.2, not llvm-gcc-4.2. For 10.6, I recommend sticking with Xcode 3 if at all possible for the time being.
ezio-melotti commentedon Oct 21, 2011
See also http://mail.python.org/pipermail/python-dev/2011-September/113731.html
OlegPlakhotnyuk commentedon Oct 23, 2011
Thank you all for help. I finally managed to build it. :-)
So here are complete steps I've taken to build python after OS reinstall:
Should I post this somewhere for others who may face similar problems?
ned-deily commentedon Oct 23, 2011
In general, I would not recommend using anything other than an Apple-supplied tool chain for building anything on Mac OS X. To help us better understand the issues, can you say exactly which compilers you tried originally, i.e. the complete --version output from the compiler(s) used in your failing Python builds?
Because of the current somewhat chaotic OS X compiler situation, I am in the process of performing a controlled test of building and testing various current versions of Python with the available Xcode compilers for 10.7 and 10.6.
OlegPlakhotnyuk commentedon Oct 23, 2011
Ah, of course.
Initially I've tried to compile using this:
i686-apple-darwin10-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)
XCode version 4.2, build 4C199
29 remaining items
ned-deily commentedon Feb 22, 2012
No. The new version of Xcode (Xcode 4.3, released last week) referred to in that post does *not* make gcc-4.2 available again. Like Xcode 4.2, it only includes clang and llvm-gcc; 4.3 includes updated versions of each of the two. gcc-4.2 was removed starting with 4.2 and there is no indication that Apple intends to re-instate it. Besides the compiler updates, Xoode 4.3 has been significantly repackaged, including the ability to download and install its components in smaller sub-packages, instead of the monolithic large downloads of recent past releases. However, Xcode 4.3 has moved the install location of components like the SDKs (which are used by Python universal builds) from /Developer to within the new Xcode application bundle (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer); this change will require additional changes to the Python build and Distutils/packaging infrastructure to support properly. For now, the easiest workaround when installing Xcode 4.3 and removing previous versions is to create a symlink from the app bundle Developer directory above to /Developer. I will be testing the new compiler versions shortly and working on more general support for 4.3.
tpn commentedon May 6, 2012
Hi Ned,
On a brand new OS X Lion install with the latest XCode (4.3.2) and command line tools*, the following worked:
./configure --with-pydebug CC=clang MACOSX_DEPLOYMENT_TARGET=10.7
That is, everything built cleanly, and all tests ran without failures:
336 tests OK.
2 tests altered the execution environment:
test_packaging test_site
26 tests skipped:
test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_gnu
test_devpoll test_epoll test_gdb test_largefile test_lzma
test_msilib test_ossaudiodev test_smtpnet test_socketserver
test_startfile test_timeout test_tk test_ttk_guionly
test_urllib2net test_urllibnet test_winreg test_winsound
test_xmlrpc_net test_zipfile64
3 skips unexpected on darwin:
test_lzma test_tk test_ttk_guionly
I presume the aim is to be able to build correctly (on all UNIX platforms) with just ./configure --with-pydebug, right?
How's your configure*-toolchain-fu?
Once we fix this I can add the build slave.
Side bar: note that this was a vanilla 10.6->10.7 install. The XCode 4.3.2 install on 10.7 was the first XCode ever installed on the box; there's no /Developer or any of the old gcc* cruft.
[*]: Went to https://developer.apple.com/downloads/index.action, logged in with my Apple ID, then downloaded 'Commandline Tools for XCode - Late March 2012' -- dmg was 'cltools_lion_latemarch12.dmg'.
mmaker commentedon May 6, 2012
Works on my machine too.
ned-deily commentedon May 6, 2012
Trent, yes, now that the Xcode 4 situation has settled down a bit, clang is the compiler of choice for OS X 10.7 with Xcode 4.3 although there are still some open questions. I intend to update configure in the near future for all active branches to provide more appropriate defaults for 10.6 and 10.7 and for universal builds with the various SDK locations.
I'm missing the context for this. Can't you override with environment variables or on the configure command itself?
tpn commentedon May 6, 2012
Yeah I uh, ....seemed to have deleted the introductory sentence I wrote that said I was doing some prep work before adding an OS X 10.7 build slave.
Are you referring to when I set up the build slave? i.e. tweak the local environment the build slave account runs under so that
./configure --with-pydebugworks out of the box?I don't think that's really a precedent we want to set (having build slave owners hack the local environment to work around build issues).
thomaspinckney3 commentedon May 10, 2012
FWIW, clang from Xcode 4.3.2 build 4E2002 w/ command line tools built everything fine for me too (i.e., ./configure CC=clang).
LM-SJN-00377886:cpython tom$ uname -a
Darwin LM-SJN-00377886 11.3.0 Darwin Kernel Version 11.3.0: Thu Jan 12 18:47:41 PST 2012; root:xnu-1699.24.23~1/RELEASE_X86_64 x86_64
LM-SJN-00377886:cpython tom$ /usr/bin/clang --version
Apple clang version 3.1 (tags/Apple/clang-318.0.58) (based on LLVM 3.1svn)
Target: x86_64-apple-darwin11.3.0
Thread model: posix
LM-SJN-00377886:cpython tom$ hg summary
parent: 76839:bb30116024ac tip
Minor fix for test_multiprocessing
branch: default
commit: (clean)
update: (current)
hynek commentedon Jun 1, 2012
Ned, any progress/problems?
I've closed two dupes already in the last few days and we're still in the alphas. We really should get this fixed before beta1 or the tracker will be flooded.
Let me know if you need something. :)
ned-deily commentedon Jun 1, 2012
Hynek: Thanks for the offer; I've got it. I'll have a comprehensive update to support the new multiple dev environments for 10.6 and beyond prior to b1 (and for 2.7.x and 3.2.x).
ronaldoussoren commentedon Jun 13, 2012
The attached configure.ac is a fragment of a configure script that tries to detect if AC_PROG_CC would pick llvm-gcc when clang is also available and then explicitly picks clang.
This would avoid "python3.3 won't build on OSX 10.7" bug reports.
This is not yet a patch to the real configure file because I'd like to add support for automatically picking the compiler inside Xcode.app when the user has Xcode 4 installed without the unix tools (e.g. on an new OSX 10.7 where the user has installed Xcode from the appstore without further actions).
ronaldoussoren commentedon Jun 13, 2012
The attached patch "bpo-13241-configure-version1.txt" implements a number of changes to configure (although only configure.ac is present in the patch)
Autodetect the compiler on OSX, in particular
(no Command Line Tools for Xcode)
Detect the most recent working SDK, falls back to the 10.4u SDK
when using an old version of Xcode or even '/' when that SDK isn't
present
This patch needs work though:
I have barely tested the patch, configure seems to generate a valid
Makefile on OSX 10.7 with the unix tools installed, I haven't tested
yet on older OSX versions of without the unix tools installed
Distutils/packaging also need changes: need to detect that the
running system has different compiler tools than the build system.
This is needed to be able to use the binary installers on python.org
on systems with other compiler versions and installed SDKs
ronaldoussoren commentedon Sep 4, 2012
The configure script for python 3.3 detects llvm-gcc and automatically selects clang when it is found.
That seems to be the best feasible solution for this issue. I therefore propose to close this issue.
xapple commentedon Sep 4, 2012
I agree to close the issue. I tried following the steps proposed here:
http://docs.python.org/devguide/
And typed these two commands on a OS X 10.8.1 with XCode and command line tools installed.
$ hg clone http://hg.python.org/cpython. $ run ./configure --with-pydebug && make -j2Python built perfectly.
ronaldoussoren commentedon Sep 5, 2012
I've closed the issue as 'wont fix' because llvm-gcc will miscompile 3.3 due to bugs in that compiler. CPython won't work around that compiler bug (if that's even possible).
The workaround: use clang to compile (and on older OSX releases use a different version of GCC that doesn't use the LLVM backend).