You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Build does not work with closed stdin on NetBSD #77565
is_valid_fd.c: is_valid_fd() from Python 3.6.5 (Python/pylifecycle.c)
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2019-10-22.23:30:02.974>created_at=<Date2018-04-29.10:16:35.561>labels= ['type-feature', 'build']
title='Build does not work with closed stdin on NetBSD'updated_at=<Date2019-10-24.10:39:12.784>user='https://github.com/MartinHusemann'
I cannot reproduce the problem with Python 3.6.5, building pyephem and with the command:
( sleep 15; python setup.py build ) >foo.log 2>&1
MartinHusemann commented on May 19, 2018
MartinHusemannmannequin
MannequinAuthor
More actions
You need to exit the parent shell, to get the original stdin revoke(2)'d.
That is: the Ctrl-D in the original descritpion is not line noise. Sorry, should have been more explicit (or used "exit" or something instead)
leot commented on May 19, 2018
leotmannequin
Mannequin
More actions
A simpler way to reproduce that (probably this problem is not limited to
building but to sys module):
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Current thread 0x00007226e0ab3800 (most recent call first):
leot commented on May 19, 2018
leotmannequin
Mannequin
More actions
And here the backtrace of the corresponding core (this is on
NetBSD/amd64 8.99.15 with lang/python36 package of today pkgsrc-current):
% gdb -core python3.6.core `which python3.6`
Reading symbols from /usr/pkg/bin/python3.6...done.
[New process 1]
Core was generated by `python3.6'.
Program terminated with signal SIGABRT, Aborted.
#0 0x000076255932924a in _lwp_kill () from /usr/lib/libc.so.12
(gdb) bt
#0 0x000076255932924a in _lwp_kill () from /usr/lib/libc.so.12
#1 0x0000762559328e65 in abort () at /usr/src/lib/libc/stdlib/abort.c:74
#2 0x000076255a749d33 in Py_FatalError (msg=msg@entry=0x76255a7dea20 "Py_Initialize: can't initialize sys standard streams")
at [Python/pylifecycle.c:1462](https://github.com/python/cpython/blob/main/Python/pylifecycle.c#L1462)
#3 0x000076255a74af4c in _Py_InitializeEx_Private (install_sigs=1, install_importlib=1) at [Python/pylifecycle.c:450](https://github.com/python/cpython/blob/main/Python/pylifecycle.c#L450)
#4 0x000076255a764902 in Py_Main (argc=argc@entry=3, argv=argv@entry=0x76255ab01060) at [Modules/main.c:700](https://github.com/python/cpython/blob/main/Modules/main.c#L700)
#5 0x0000000000400e2f in main (argc=3, argv=<optimized out>) at ./Programs/python.c:69
(gdb) f 4
#4 0x000076255a764902 in Py_Main (argc=argc@entry=3, argv=argv@entry=0x76255ab01060) at [Modules/main.c:700](https://github.com/python/cpython/blob/main/Modules/main.c#L700)
700 Py_Initialize();
(gdb) f 3
#3 0x000076255a74af4c in _Py_InitializeEx_Private (install_sigs=1, install_importlib=1) at [Python/pylifecycle.c:450](https://github.com/python/cpython/blob/main/Python/pylifecycle.c#L450)
450 Py_FatalError(
(gdb) list
445 if (_PyTraceMalloc_Init() < 0)
446 Py_FatalError("Py_Initialize: can't initialize tracemalloc");
447
448 initmain(interp); /* Module __main__ */
449 if (initstdio() < 0)
450 Py_FatalError(
451 "Py_Initialize: can't initialize sys standard streams");
452
453 /* Initialize warnings. */
454 if (PySys_HasWarnOptions()) {
Please let me know if any further information is needed and I will try
to help!
Thank you!
leot commented on May 19, 2018
leotmannequin
Mannequin
More actions
After testing is_valid_fd() (from Python/pylifecycle.c) separately
(an `is_valid_fd.c' file will be attached to ease reproduction) I think
that also NetBSD is affected by bpo-30225.
Using dup(2) (at is currently done in NetBSD):
% cc -o ivf is_valid_fd.c
% sleep 5 && ./ivf > /tmp/log & exit
[... the terminal is closed via ^D ...]
% cat /tmp/log
0: 1
1: 1
2: 1
Using fstat(2):
% cc -DUSE_FSTAT -o ivf is_valid_fd.c
% sleep 5 && ./ivf > /tmp/log & exit
[... the terminal is closed via ^D ...]
% cat /tmp/log
0: 0
1: 1
2: 0
The possible attached patch seems to fix the problem reported by Martin.
I'm quite that this issue has been fixed by bpo-30225 which is part Python 3.7.4, Python 3.8.0 and newer. I mark the issue as duplicate of bpo-30225. Please reopen the bug if these versions still fail on NetBSD.
bpo-30225: Fix is_valid_fd() on macOS Tiger (bpo-1443)
is_valid_fd() now uses fstat() instead of dup() on macOS to return 0
on a pipe when the other side of the pipe is closed. fstat() fails
with EBADF in that case, whereas dup() succeed.
changed the title [-]Build does not work with closed stdin[/-][+]Build does not work with closed stdin on NetBSD[/+]on Oct 22, 2019
leot commented on Oct 24, 2019
leotmannequin
Mannequin
More actions
Hello Victor,
I can confirm that I can no longer reproduce this problem with Python
3.7.5 on NetBSD and according a double-check to Python/pylifecycle.c:is_valid_fd() now fstat() is always used on
NetBSD.
Activity
MartinHusemann commentedon Apr 29, 2018
When building python extensions in the background w/o stdin (and stderr and stdout redirected to a log file), the invocation of setup.py fails.
Normal build in a shell:
Example from pyexpat:
But if invoked as
and then watching /tmp/log from another session:
Fatal Python error: Py_Initialize: can't initialize sys standard streams
OSError: [Errno 9] Bad file descriptor
Current thread 0x000075b4e4cee800 (most recent call first):
[1] Abort trap (core dumped) /usr/pkg/bin/pyt...
xdegaye commentedon May 4, 2018
I cannot reproduce the problem with Python 3.6.5, building pyephem and with the command:
( sleep 15; python setup.py build ) >foo.log 2>&1
MartinHusemann commentedon May 19, 2018
You need to exit the parent shell, to get the original stdin revoke(2)'d.
That is: the Ctrl-D in the original descritpion is not line noise. Sorry, should have been more explicit (or used "exit" or something instead)
leot commentedon May 19, 2018
A simpler way to reproduce that (probably this problem is not limited to
building but to sys module):
And in `/tmp/log':
leot commentedon May 19, 2018
And here the backtrace of the corresponding core (this is on
NetBSD/amd64 8.99.15 with lang/python36 package of today pkgsrc-current):
Please let me know if any further information is needed and I will try
to help!
Thank you!
leot commentedon May 19, 2018
After testing is_valid_fd() (from Python/pylifecycle.c) separately
(an `is_valid_fd.c' file will be attached to ease reproduction) I think
that also NetBSD is affected by bpo-30225.
Using dup(2) (at is currently done in NetBSD):
Using fstat(2):
The possible attached patch seems to fix the problem reported by Martin.
vstinner commentedon Oct 22, 2019
I'm quite that this issue has been fixed by bpo-30225 which is part Python 3.7.4, Python 3.8.0 and newer. I mark the issue as duplicate of bpo-30225. Please reopen the bug if these versions still fail on NetBSD.
commit 1c4670e
Author: Victor Stinner <[email protected]>
Date: Thu May 4 00:45:56 2017 +0200
[-]Build does not work with closed stdin[/-][+]Build does not work with closed stdin on NetBSD[/+]leot commentedon Oct 24, 2019
Hello Victor,
I can confirm that I can no longer reproduce this problem with Python
3.7.5 on NetBSD and according a double-check to
Python/pylifecycle.c:is_valid_fd() now fstat() is always used on
NetBSD.
Thanks for the update and for looking at it!
vstinner commentedon Oct 24, 2019
Great!