Repository navigation
Clean up default test output #910
Description
The current test output is a little messy. Consider this test run:
mypy $ ./runtests.py unit-test -a "*MultiAnd*"
PARALLEL 8
SUMMARY 12 tasks selected
0 test cases run, all passed.; running {#0, #1, #2, #3, #4, #5, #6, #7}
*** OK ***
0 test cases run, all passed.; running {#0, #1, #2, #3, #4, #5, #7, #8}
*** OK ***
0 test cases run, all passed.; running {#0, #1, #2, #3, #4, #5, #8, #9}
*** OK ***
0 test cases run, all passed.
*** OK ***
0 test cases run, all passed.; running {#0, #3, #4, #5, #8, #9, #10, #11}
*** OK ***
0 test cases run, all passed.
*** OK ***
0 test cases run, all passed.
*** OK ***
0 test cases run, all passed.; running {#0, #3, #5, #9, #10, #11}
*** OK ***
0 test cases run, all passed.; running {#0, #3, #10, #11}
*** OK ***
0 test cases run, all passed.; running {#3, #10, #11}
*** OK ***
0 test cases run, all passed.0; running {#3, #11}
*** OK ***
2 test cases run, all passed.0; running {#3}
*** OK ***
SUMMARY all 12 tasks passed
I'd prefer something like this instead (the intermediate subsets {...} are useful as well and should not be removed, but they should be cleaned up properly):
mypy $ ./runtests.py unit-test -a "*MultiAnd*"
PARALLEL 8
SUMMARY 12 tasks selected
SUMMARY all 12 tasks passed, all 2 test cases passed
*** OK ***
Activity
o11c commentedon Oct 12, 2015
It might be as simple as just passing
['-q']as the defaultarglist.JukkaL commentedon Oct 12, 2015
-qdoesn't quite do what I want:Displaying the
{#0, #4, ...}messages is fine as long as they don't corrupt the actual test output.o11c commentedon Oct 12, 2015
I'm talking about myunit's
-q, not runtest's-q, e.g../runtests.py -a -q unit-test. This gives absolutely no intermingling for me.Total number across all suites is what #908 and the whole "collect first" then "lookup by name" is working towards (there's no reason those need to be done in the same process!)
JukkaL commentedon Oct 12, 2015
Ah okay I was confused. In that case I'd love to hear more about "collect first" and "lookup by name" and how they can give us cleaner test output :-)
o11c commentedon Oct 12, 2015
Basically, instead of running ~400 tasks, run all ~2500 individual testcases as direct children of the driver.
The complication comes from the fact that we probably don't want to be spawning a separate python interpreter for every one of those, so we need some sort of pipe with control commands. (Thus why it is out of scope for #908 for now, though the parallelism cost-benefit analysis will likely be beneficial for pythoneval).
ilevkivskyi commentedon Jul 20, 2017
Is this still relevant? I like the current test output, it is quite terse.