Repository navigation
EBADF error on x86 Tiger 3.x buildbot #74411
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2017-05-04.14:52:07.049>
created_at = <Date 2017-05-02.08:12:52.478>
labels = ['OS-mac', '3.7', 'tests']
title = 'EBADF error on x86 Tiger 3.x buildbot'
updated_at = <Date 2021-12-13.12:52:34.373>
user = 'https://github.com/vstinner'bugs.python.org fields:
activity = <Date 2021-12-13.12:52:34.373>
actor = 'corona10'
assignee = 'none'
closed = True
closed_date = <Date 2017-05-04.14:52:07.049>
closer = 'vstinner'
components = ['macOS', 'Tests']
creation = <Date 2017-05-02.08:12:52.478>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 30225
keywords = []
message_count = 9.0
messages = ['292725', '292878', '292935', '292940', '292942', '292943', '292974', '292975', '292984']
nosy_count = 5.0
nosy_names = ['db3l', 'ronaldoussoren', 'vstinner', 'ned.deily', 'corona10']
pr_nums = ['1443', '1449', '1450', '30082']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue30225'
versions = ['Python 3.5', 'Python 3.6', 'Python 3.7']
Activity
vstinner commentedon May 2, 2017
test_http_body_pipe() of test_urllib2 uses a subprocess, no idea if it's related:
See also my issue bpo-29915 "Drop Mac OS X Tiger support in Python 3.7?" :-)
http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/597/steps/test/logs/stdio
./python.exe -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform
/bin/sh: line 1: pybuildbot.identify: command not found
./python.exe ./Tools/scripts/run_tests.py -j 1 -u all -W --slowest --timeout=900 -j2
make: [buildbottest] Error 127 (ignored)
== CPython 3.7.0a0 (heads/master:5d7a8d0, May 1 2017, 12:22:06) [GCC 4.0.1 (Apple Computer, Inc. build 5367)]
== Darwin-8.11.1-i386-32bit little-endian
== hash algorithm: siphash24 32bit
== cwd: /Users/db3l/buildarea/3.x.bolen-tiger/build/build/test_python_12700
== encodings: locale=UTF-8, FS=utf-8
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=1, verbose=0, bytes_warning=2, quiet=0, hash_randomization=1, isolated=0)
Using random seed 2778758
Run tests in parallel using 2 child processes
0:00:12 [ 1/405] test_deque passed
0:00:35 [ 2/405] test_trace passed (34 sec)
0:00:56 [ 3/405] test_capi passed -- running: test_socket (44 sec)
0:00:57 [ 4/405] test_pkgimport passed -- running: test_socket (45 sec)
0:01:02 [ 5/405] test_socket passed (49 sec)
0:01:07 [ 6/405] test_imaplib passed
0:01:18 [ 7/405] test_urllib2 passed
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Current thread 0xa000d000 (most recent call first):
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Current thread 0xa000d000 (most recent call first):
0:01:20 [ 8/405] test_getargs2 passed
0:01:24 [ 9/405] test_calendar passed
0:01:25 [ 10/405] test_unpack_ex passed
vstinner commentedon May 3, 2017
@david Bolen: Would you mind to take a look? I don't have access to your x86 Tiger 3.x buildbot.
db3l commentedon May 3, 2017
Hmm, I wonder if this is another race condition similar to bpo-8458?
I think that was thought to be related to the subprocess exiting quickly, in which case the question might be why that might happen more so than the actual descriptor error.
BTW, Victor, in case it would help, your account (vstinner, ssh key haypo@marge) still exists on this buildbot from past testing in 2011. The only thing I'd need to re-enable is the port forwarding on my firewall.
db3l commentedon May 3, 2017
In running the test under a local build, the issue is very repeatable, but I believe it's actually due to slow process startup rather than a quick exit.
That is, adding a brief sleep after process creation and just before the Request() call seems to fix the problem. I'm guessing the buildbot is sluggish enough that it just takes a bit longer for the process to start and be ready to be used.
Whether or not that's purely a machine or test problem, or whether it means that perhaps subprocess.Popen() is returning sooner than it should with a process that isn't ready yet is unclear. (Or even if subprocess can tell)
With the machine fairly idle, it looks like the minimum workable delay is about 200ms - probably something a bit longer would be safer under load, if looking for a quick workaround.
vstinner commentedon May 3, 2017
initstdio() is supposed to handle EBADF: see issue bpo-24891.
vstinner commentedon May 3, 2017
New changeset 1c4670e by Victor Stinner in branch 'master':
bpo-30225: Fix is_valid_fd() on macOS Tiger (bpo-1443)
1c4670e
vstinner commentedon May 4, 2017
New changeset 31906b4 by Victor Stinner in branch '3.6':
bpo-30225: Fix is_valid_fd() on macOS Tiger (bpo-1443) (bpo-1449)
31906b4
vstinner commentedon May 4, 2017
New changeset 8a1c710 by Victor Stinner in branch '3.5':
bpo-30225: Fix is_valid_fd() on macOS Tiger (bpo-1443) (bpo-1450)
8a1c710
vstinner commentedon May 4, 2017
Ok, the race condition causing EBADF is now handled correctly in 3.5, 3.6 and master (3.7).