Repository navigation
test_socket failures on Mac OS X #57167
Description
| BPO | 12958 |
|---|---|
| Nosy | @brettcannon, @ronaldoussoren, @ncoghlan, @ned-deily, @bitdancer, @voidspace |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ncoghlan'
closed_at = <Date 2011-09-19.10:26:46.561>
created_at = <Date 2011-09-11.05:40:43.125>
labels = ['type-bug', 'library']
title = 'test_socket failures on Mac OS X'
updated_at = <Date 2014-02-01.10:21:12.638>
user = 'https://github.com/ncoghlan'bugs.python.org fields:
activity = <Date 2014-02-01.10:21:12.638>
actor = 'ned.deily'
assignee = 'ncoghlan'
closed = True
closed_date = <Date 2011-09-19.10:26:46.561>
closer = 'python-dev'
components = ['Library (Lib)']
creation = <Date 2011-09-11.05:40:43.125>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = ['67']
issue_num = 12958
keywords = []
message_count = 14.0
messages = ['143854', '143860', '143863', '144241', '144248', '144257', '144264', '159694', '187635', '187637', '187642', '187643', '187646', '209884']
nosy_count = 8.0
nosy_names = ['brett.cannon', 'ronaldoussoren', 'ncoghlan', 'baikie', 'ned.deily', 'r.david.murray', 'michael.foord', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue12958'
versions = ['Python 3.3', 'Python 3.4']
Activity
ncoghlan commentedon Sep 11, 2011
Several of the new socket tests introduced in bpo-6560 are failing on the Mac OS X buildbots. These appear to be due to platform problems rather than anything in the test suite or the new sendmsg/recvmsg code, but it would be good if a developer on Mac OS X could take a closer look at them.
To avoid polluting the buildbot results, the affected tests will be flagged as expected failures when run on Mac OS X (this isn't in the repo yet, but once it is, the tests will be marked with "anticipate_failure" in the test_socket code).
ncoghlan commentedon Sep 11, 2011
First attempt didn't quite work - the FD passing tests somehow seem to be reporting both 'ERROR' *and* 'expected failure', which is causing the test overall to remain red.
http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%202%20custom/builds/41/steps/test/logs/stdio
ncoghlan commentedon Sep 11, 2011
Ah, I believe I see why the expected failure isn't working properly - the failing testFDPass* tests are causing the subsequent tear down code to also fail.
ncoghlan commentedon Sep 18, 2011
OK, I've now looked into *why* the socket tests are throwing errors in tearDown, and it has to do with the way the threaded client/server tests in test_socket are set up.
Specifically, ThreadableTest uses tearDown to reraise any exception raised in the client thread, and these are therefore outside the scope of the "expectedFailure" suppression in unittest.
Now that I've tracked this down, it would be fairly straightforward to fix this specifically within test_socket.ThreadableTest by appropriately adjusting the definition of ThreadableTest.clientRun to discard exceptions encountered in tests flagged as expected failures.
However, I'm wondering if that's the right thing to do. Perhaps it would make more sense to change unittest itself so that "expectedFailure" also suppresses tearDown errors. It doesn't seem all that unusual for a known failing test to also cause problems for the tearDown code.
Added Michael to the nosy list to ask for his advice/opinion. In the meantime, I'll work on a patch that adjusts ThreadableTest directly.
voidspace commentedon Sep 18, 2011
See bpo-10548. There is some resistance to expectedFailure masking errors in setUp/tearDown as these aren't the place where you would normally expect the expected failure...
ncoghlan commentedon Sep 18, 2011
OK, I'll just deal with the problem directly in test_socket then.
It looks like my latest attempt (suppressing unittest._ExpectedFailure in test_socket.ThreadableTest.clientRun) did the trick, so I'll push the updated tests some time this evening:
http://www.python.org/dev/buildbot/all/builders/AMD64%20Snow%20Leopard%202%20custom/builds/44/steps/test/logs/stdio
python-dev commentedon Sep 19, 2011
New changeset a4e4facad164 by Nick Coghlan in branch 'default':
Close bpo-12958 by flagging expected failures in test_socket on Mac OS X
http://hg.python.org/cpython/rev/a4e4facad164
python-dev commentedon Apr 30, 2012
New changeset e64bec91ac91 by Richard Oudkerk in branch 'default':
Issue bpo-14669: Skip multiprocessing connection pickling test on MacOSX
http://hg.python.org/cpython/rev/e64bec91ac91
ronaldoussoren commentedon Apr 23, 2013
Not sure what changed, but these tests are reported as failures when I run 'make test' and not as expected failures.
bitdancer commentedon Apr 23, 2013
Ronald: in 3.4 only? If so I wonder if the subtests patch had an unintended side effect...I'm pretty sure it touched the expected failure machinery somehow.
ronaldoussoren commentedon Apr 23, 2013
Yes, only on 3.4:
======================================================================
ERROR: testFDPassSeparate (test.test_socket.RecvmsgSCMRightsStreamTest)
----------------------------------------------------------------------
(And simular for the other 3 marked socket tests)
ncoghlan commentedon Apr 23, 2013
It's entirely possible the hack in test_socket (where it catches unittest._ExpectedFailure to suppress the issues in the tearDown code after the test fails) were invalidated by the subtest changes.
ronaldoussoren commentedon Apr 23, 2013
FWIW I've filed a bug report about the behavior of sendmsg with Apple, the RADAR number is 13716133.
ned-deily commentedon Feb 1, 2014
FYI, the OS X platform bugs causing the three "send timeout" test failures were fixed in OS X 10.7 causing those test cases to have previously silently skipped "unexpected successes". See bpo-20474 for more details.