Repository navigation
test_asyncio: relax timings even more #64535
Description
| BPO | 20336 |
|---|---|
| Nosy | @gvanrossum, @jcea, @ncoghlan, @vstinner, @skrah, @1st1 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2014-06-06.13:15:10.964>
created_at = <Date 2014-01-21.22:29:51.627>
labels = ['invalid', 'type-bug', 'tests', 'expert-asyncio']
title = 'test_asyncio: relax timings even more'
updated_at = <Date 2014-06-06.13:38:03.524>
user = 'https://github.com/skrah'bugs.python.org fields:
activity = <Date 2014-06-06.13:38:03.524>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2014-06-06.13:15:10.964>
closer = 'skrah'
components = ['Tests', 'asyncio']
creation = <Date 2014-01-21.22:29:51.627>
creator = 'skrah'
dependencies = []
files = []
hgrepos = []
issue_num = 20336
keywords = ['buildbot']
message_count = 10.0
messages = ['208714', '208745', '208906', '208909', '208911', '208912', '208913', '208939', '219882', '219883']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'jcea', 'ncoghlan', 'vstinner', 'skrah', 'yselivanov']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20336'
versions = ['Python 3.4']
Activity
skrah commentedon Jan 21, 2014
It seems that the relaxed timings in some tests are still not
sufficient:
http://buildbot.python.org/all/builders/AMD64%20OpenIndiana%203.x/builds/7417/steps/test/logs/stdio
======================================================================
FAIL: test_time_and_call_at (test.test_asyncio.test_base_events.BaseEventLoopTests)
----------------------------------------------------------------------
gvanrossum commentedon Jan 22, 2014
I won't have time to fix this (and bardly to even review a fix) but this seems a straightforward fix and you can go ahead without me -- maybe you can get someone else with some asyncio experience to review your patch (e.g. Antoine or Victor).
skrah commentedon Jan 23, 2014
I'm seeing other unrelated timeouts on the OpenIndiana machines.
Jesús, are the machines perhaps operating under an unusually high load?
vstinner commentedon Jan 23, 2014
I read somewhere that the host of these virtual machines is *highly* loaded (load > 6). It's more a buildbot configuration issue than a test_asyncio issue.
skrah commentedon Jan 23, 2014
I see. That's astonishing, because usually the builds are ultra-fast
on those machines.
vstinner commentedon Jan 23, 2014
Tests failing because of timings is really annoying. It would be nice to add something in test.support to configure timings. A function can maybe wait until the system load is lower than a threshold, or we should add a parameter to configure a tolerance between the perfect timing and the expected timing.
The tolerance can be a simple factor, a floating point number >= 1.0.
For example, replace:
0.09 <= t1-t0 <= 0.9, t1-t0
with
0.1 - 0.01 * factor <= t1-t0 <= 0.1 + 0.8 * factor
or a new test function:
check_timing(t1-t0, 0.1, 0.01, 0.8)
skrah commentedon Jan 23, 2014
+1 for doing something about timing failures. Perhaps also a
decorator @skip_if_load_higher_than(x).
[Nick, I'm adding you because our current test suite could lead
to "commit impossible" situations with Zuul.]
ncoghlan commentedon Jan 23, 2014
I have a draft PEP for the Zuul idea, and yes, it notes the increased consequences of declaring a buildbot stable when it sometimes isn't. There's a reason OpenStack treats third party test results as advisory only rather than as gating criteria :P
skrah commentedon Jun 6, 2014
Let's open a new issue for system load detection. This one is not asyncio specific.
vstinner commentedon Jun 6, 2014
I opened issues bpo-20910 and bpo-20964 for example.