Repository navigation
possible multiple regressions on AIX #80877
Description
| BPO | 36696 |
|---|---|
| Nosy | @vstinner, @methane, @aixtools |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-04-23.10:29:59.791>
created_at = <Date 2019-04-22.06:58:37.681>
labels = ['build', '3.8']
title = 'possible multiple regressions on AIX'
updated_at = <Date 2019-04-23.10:29:59.778>
user = 'https://github.com/aixtools'bugs.python.org fields:
activity = <Date 2019-04-23.10:29:59.778>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-04-23.10:29:59.791>
closer = 'vstinner'
components = ['Build']
creation = <Date 2019-04-22.06:58:37.681>
creator = 'Michael.Felt'
dependencies = []
files = []
hgrepos = []
issue_num = 36696
keywords = ['patch']
message_count = 5.0
messages = ['340643', '340650', '340704', '340705', '340711']
nosy_count = 3.0
nosy_names = ['vstinner', 'methane', 'Michael.Felt']
pr_nums = ['12922']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue36696'
versions = ['Python 3.8']
Activity
aixtools commentedon Apr 22, 2019
My AIX bot has been very consistent - only the multiprocessing tests failing when run by bot, but 4 or 5 days ago 3 to 5 additional tests - that, afaik, had never failed before, are now failing.
These may also be compiler related specifics, or the presence (better lack of) 3rd party packages.
Anyway - summary of output:
INFO: Can't locate Tcl/Tk libs and/or headers
building '_pickle' extension
xlc_r -O2 -I/opt/include -I/opt/buildaix/include -g -I/opt/include -I/opt/buildaix/include -g -I./Include/internal -I./Include -I. -I/opt/include -I/opt/buildaix/include -I/home/buildbot/cpython-master/Include -I/home/buildbot/cpython-master -c /home/buildbot/cpython-master/Modules/_pickle.c -o build/temp.aix-7.1-3.8-pydebug/home/buildbot/cpython-master/Modules/_pickle.o -D Py_BUILD_CORE_MODULE
1506-261 (W) Suboption Py_BUILD_CORE_MODULE is not valid for option D.
"/home/buildbot/cpython-master/Modules/_pickle.c", line 8.4: 1506-205 (S) #error "Py_BUILD_CORE_BUILTIN or Py_BUILD_CORE_MODULE must be defined"
building '_json' extension
xlc_r -O2 -I/opt/include -I/opt/buildaix/include -g -I/opt/include -I/opt/buildaix/include -g -I./Include/internal -I./Include -I. -I/opt/include -I/opt/buildaix/include -I/home/buildbot/cpython-master/Include -I/home/buildbot/cpython-master -c /home/buildbot/cpython-master/Modules/_json.c -o build/temp.aix-7.1-3.8-pydebug/home/buildbot/cpython-master/Modules/_json.o -D Py_BUILD_CORE_MODULE
1506-261 (W) Suboption Py_BUILD_CORE_MODULE is not valid for option D.
"/home/buildbot/cpython-master/Modules/_json.c", line 8.4: 1506-205 (S) #error "Py_BUILD_CORE_BUILTIN or Py_BUILD_CORE_MODULE must be defined"
"./Include/internal/pycore_accu.h", line 13.4: 1506-205 (S) #error "this header requires Py_BUILD_CORE define"
building '_testinternalcapi' extension
xlc_r -O2 -I/opt/include -I/opt/buildaix/include -g -I/opt/include -I/opt/buildaix/include -g -I./Include/internal -I./Include -I. -I/opt/include -I/opt/buildaix/include -I/home/buildbot/cpython-master/Include -I/home/buildbot/cpython-master -c /home/buildbot/cpython-master/Modules/_testinternalcapi.c -o build/temp.aix-7.1-3.8-pydebug/home/buildbot/cpython-master/Modules/_testinternalcapi.o -D Py_BUILD_CORE_MODULE
1506-261 (W) Suboption Py_BUILD_CORE_MODULE is not valid for option D.
"/home/buildbot/cpython-master/Modules/_testinternalcapi.c", line 6.4: 1506-205 (S) #error "Py_BUILD_CORE_BUILTIN or Py_BUILD_CORE_MODULE must be defined"
"./Include/internal/pycore_coreconfig.h", line 8.4: 1506-205 (S) #error "this header requires Py_BUILD_CORE define"
Python build finished successfully!
The necessary bits to build these optional modules were not found:
_curses_panel _gdbm _tkinter
ossaudiodev readline spwd
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
time
Failed to build these modules:
_json _pickle _testinternalcapi
running build_scripts copying and adjusting /home/buildbot/cpython-master/Tools/scripts/pydoc3 -> build/scripts-3.8 copying and adjusting /home/buildbot/cpython-master/Tools/scripts/idle3 -> build/scripts-3.8 copying and adjusting /home/buildbot/cpython-master/Tools/scripts/2to3 -> build/scripts-3.8 changing mode of build/scripts-3.8/pydoc3 from 644 to 755 changing mode of build/scripts-3.8/idle3 from 644 to 755 changing mode of build/scripts-3.8/2to3 from 644 to 755 renaming build/scripts-3.8/pydoc3 to build/scripts-3.8/pydoc3.8 renaming build/scripts-3.8/idle3 to build/scripts-3.8/idle3.8 renaming build/scripts-3.8/2to3 to build/scripts-3.8/2to3-3.8 ./python -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform ./python ./Tools/scripts/run_tests.py /home/buildbot/cpython-master/python -u -W default -bb -E -m test -r -w -j 0 -u all,-largefile,-audio,-gui == CPython 3.8.0a3+ (heads/master:3e986de0d6, Apr 21 2019, 17:04:13) [C] ... running: test_venv (3 min 38 sec), test_decimal (2 min 27 sec), test_ftplib (30 sec 50 ms), test_zipfile (1 min 43 sec) 0:04:00 [ 40/420/1] test_ftplib failed -- running: test_venv (3 min 53 sec), test_decimal (2 min 42 sec), test_zipfile (1 min 58 sec) test test_ftplib failed -- Traceback (most recent call last): File "/home/buildbot/cpython-master/Lib/test/test_ftplib.py", line 605, in test_storlines self.client.storlines('stor', f) File "/home/buildbot/cpython-master/Lib/ftplib.py", line 526, in storlines conn.unwrap() File "/home/buildbot/cpython-master/Lib/ssl.py", line 1094, in unwrap s = self._sslobj.shutdown() socket.timeout: The read operation timed out 0:04:05 [ 41/420/1] test_locale passed -- running: test_venv (3 min 58 sec), test_decimal (2 min 47 sec), test_zipfile (2 min 3 sec) ... 0:12:14 [110/420/2] test_embed failed -- running: test_multiprocessing_forkserver (1 min 25 sec), test_compileall (2 min 9 sec) test test_embed failed -- multiple errors occurred; run in verbose mode for details running: test_multiprocessing_forkserver (1 min 55 sec), test_faulthandler (30 sec 42 ms), test_compile (56 sec 571 ms), test_compileall (2 min 39 sec) 0:12:55 [111/420/2] test_faulthandler passed (36 sec 350 ms) -- running: test_multiprocessing_forkserver (2 min 6 sec), test_compile (1 min 7 sec), test_compileall (2 min 50 sec) 0:13:46 [125/420/3] test_json failed -- running: test_multiprocessing_forkserver (2 min 57 sec) test test_json crashed -- Traceback (most recent call last): File "/home/buildbot/cpython-master/Lib/test/libregrtest/runtest.py", line 166, in runtest_inner the_module = importlib.import_module(abstest) File "/home/buildbot/cpython-master/Lib/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 777, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/buildbot/cpython-master/Lib/test/test_json/__init__.py", line 12, in <module> cjson.JSONDecodeError = cjson.decoder.JSONDecodeError = json.JSONDecodeError AttributeError: 'NoneType' object has no attribute 'JSONDecodeError' ... 0:24:49 [212/420/5] test_pydoc failed -- running: test_pyclbr (1 min 2 sec), test_subprocess (11 min 1 sec) test test_pydoc crashed -- Traceback (most recent call last): File "/home/buildbot/cpython-master/Lib/test/libregrtest/runtest.py", line 166, in runtest_inner the_module = importlib.import_module(abstest) File "/home/buildbot/cpython-master/Lib/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 777, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/buildbot/cpython-master/Lib/test/test_pydoc.py", line 9, in <module> import _pickle ModuleNotFoundError: No module named '_pickle' ... 0:24:59 [214/420/6] test_pyclbr failed -- running: test_subprocess (11 min 12 sec) *** Pickler test test_pyclbr failed -- Traceback (most recent call last): File "/home/buildbot/cpython-master/Lib/test/test_pyclbr.py", line 227, in test_others cm('pickle', ignore=('partial',)) File "/home/buildbot/cpython-master/Lib/test/test_pyclbr.py", line 144, in checkModule self.assertHaskey(dict, name, ignore) File "/home/buildbot/cpython-master/Lib/test/test_pyclbr.py", line 48, in assertHaskey self.assertIn(key, obj) AssertionError: 'Pickler' not found in {'partial': <pyclbr.Class object at 0x306a1990>, 'PickleError': <pyclbr.Class object at 0x30834f30>, 'PicklingError': <pyclbr.Class object at 0x30834e40>, 'UnpicklingError': <pyclbr.Class object at 0x30834f00>, '_Stop': <pyclbr.Class object at 0x30834f60>, '_Framer': <pyclbr.Class object at 0x30834f90> , '_Unframer': <pyclbr.Class object at 0x30839150>, '_getattribute': <pyclbr.Function object at 0x308392a0>, 'whichmodule': <pyclbr.Function object at 0x30839300>, 'encode _long': <pyclbr.Function object at 0x30839330>, 'decode_long': <pyclbr.Function object at 0x30839360>, '_Pickler': <pyclbr.Class object at 0x308392d0>, '_Unpickler': <pycl br.Class object at 0x308398d0>, '_dump': <pyclbr.Function object at 0x30840750>, '_dumps': <pyclbr.Function object at 0x30840780>, '_load': <pyclbr.Function object at 0x30 8407b0>, '_loads': <pyclbr.Function object at 0x308407e0>, '_test': <pyclbr.Function object at 0x30840810>} ... 1:08:14 [398/420/8] test_inspect failed -- running: test_tools (10 min 44 sec), test_multiprocessing_spawn (11 min 35 sec), test_io (5 min 23 sec) test test_inspect crashed -- Traceback (most recent call last): File "/home/buildbot/cpython-master/Lib/test/libregrtest/runtest.py", line 166, in runtest_inner the_module = importlib.import_module(abstest) File "/home/buildbot/cpython-master/Lib/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 777, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/buildbot/cpython-master/Lib/test/test_inspect.py", line 11, in <module> import _pickle ModuleNotFoundError: No module named '_pickle' ... 388 tests OK.9 tests failed:
test_embed test_ftplib test_inspect test_json
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn test_pyclbr test_pydoc
methane commentedon Apr 22, 2019
Maybe, XLC doesn't support -D name. -Dname should be used instead.
aixtools commentedon Apr 23, 2019
On 22/04/2019 14:15, Inada Naoki wrote:
Excellent hint: the diff between bot run 1013 and run 1014 reveals:
# profiler (_lsprof is for cProfile.py)
self.add(Extension('_lsprof', ['_lsprof.c', 'rotatingtree.c']))
So, some simple editing: and the diff with master (of yesterday):
# profiler (_lsprof is for cProfile.py)
self.add(Extension('_lsprof', ['_lsprof.c', 'rotatingtree.c']))
@@ -816,7 +816,7 @@ class PyBuildExt(build_ext):
# Python Internal C API test module
self.add(Extension('_testinternalcapi', ['_testinternalcapi.c'],
- extra_compile_args=['-D Py_BUILD_CORE_MODULE']))
+ extra_compile_args=['-DPy_BUILD_CORE_MODULE']))
# Python PEP-3118 (buffer protocol) test module
self.add(Extension('_testbuffer', ['_testbuffer.c']))
And now the modules build as expected.
Should I open PR - or is this better fixed in the original PR? What
keeps things cleaner?
vstinner commentedon Apr 23, 2019
I wrote PR 12922.
I really hate -Ddefine syntax, but it seems like we have to use it just for AIX :-( I tried to use "-D Py_BUILD_CORE_..." in Modules/Setup, but Module/makesetup is also stupid and doesn't support this syntax :-( I was too lazy to fix Module/makesetup.
vstinner commentedon Apr 23, 2019
New changeset 5749134 by Victor Stinner in branch 'master':
bpo-36635, bpo-36696: Fix setup.py on AIX (GH-12922)
5749134