A parameterized Swift Testing test case reports a different duration on each backend, and the difference is currently hidden by the durations allow-list entry.
Measured
On CI's TestResults.xcresult (downloaded from the failing run of #476 and reproduced locally), exactly one leaf test disagrees:
SwiftTestingSuite/parameterizedAddition(value:) legacy 0.001268s modern 0.000423s
Legacy is ~3× modern, and the test has three argument sets. Every other leaf — XCTest cases, the non-parameterized Swift Testing cases, both other fixtures — is bit-identical.
The same measurement on a faster machine gives 0.0 on both backends, which is why it did not appear during #476's local development and only surfaced in CI.
Root cause
This is the arguments form of a problem the migration design already solved for repetitions.
Design answer 8 (docs/superpowers/specs/2026-08-10-xcresulttool-legacy-migration-design.md):
8 | Test-case duration sums repetitions? | Yes — keep today's behaviour | The modern Test Case node reports its own duration, which is not the sum … Summing in the renderer makes both backends agree by construction.
TestCase.duration is iterations.reduce(0) { $0 + $1.duration }, so repetitions converge by construction. Answer 6 added ParsedTestCase.arguments at the same time and noted it was "unexercised by fixtures" — so nobody applied answer 8's treatment to it. It is exercised now: legacy merges a parameterized case's argument executions into iterations and therefore sums them; the modern reader reports the parameterized node's own durationInSeconds and does not sum its Arguments children.
Candidate fix
Have the modern reader sum the per-argument execution durations for a parameterized case, if xcresulttool's Arguments nodes expose them — the answer-8 move, converging the two backends by construction rather than by mask.
If the format does not expose per-argument durations, the alternative is the other direction the design prefers: hold legacy down to modern's capability, i.e. stop summing for the arguments case. Either way the divergence goes away instead of being masked.
Why it needs an issue rather than living under the mask
The spec is explicit that a mask is the worse of two outcomes:
Direction of the win: each answer either removes a field from the port or an entry from the differential allow-list. Deliberately holding the legacy backend down to the modern backend's capability makes the two agree, and an unmasked diff proves more than a masked one. The cost is one-way and honest … rather than a permanent asymmetry hidden behind a mask.
and differential-allowlist.json says an entry means "accepting a permanent difference between the backends — a design decision, not a way to quiet a failing test." No entry is being added here — this divergence already falls under the existing durations rule, and has since the parameterized fixture landed — but #476's summary header is the first place it aggregates into a single number, which is what made it visible. Filing it so the mask is a known debt with an owner rather than a rug.
When this is fixed
Two things come out, both named in the code:
- the exclusion in
DifferentialTests.testSummaryHeaderNumbersAgreeAcrossBackends, which today skips this one identifier from an otherwise exact per-leaf duration comparison;
- the paragraph in
Sources/XCTestHTMLReportCore/Classes/Models/RunSummary.swift explaining why the header's total is written parenthesised.
The header's duration can then be written in any shape, because it would no longer need the durations rule to cover it.
Found while implementing #439's summary header (#476).
A parameterized Swift Testing test case reports a different duration on each backend, and the difference is currently hidden by the
durationsallow-list entry.Measured
On CI's
TestResults.xcresult(downloaded from the failing run of #476 and reproduced locally), exactly one leaf test disagrees:Legacy is ~3× modern, and the test has three argument sets. Every other leaf — XCTest cases, the non-parameterized Swift Testing cases, both other fixtures — is bit-identical.
The same measurement on a faster machine gives 0.0 on both backends, which is why it did not appear during #476's local development and only surfaced in CI.
Root cause
This is the arguments form of a problem the migration design already solved for repetitions.
Design answer 8 (
docs/superpowers/specs/2026-08-10-xcresulttool-legacy-migration-design.md):TestCase.durationisiterations.reduce(0) { $0 + $1.duration }, so repetitions converge by construction. Answer 6 addedParsedTestCase.argumentsat the same time and noted it was "unexercised by fixtures" — so nobody applied answer 8's treatment to it. It is exercised now: legacy merges a parameterized case's argument executions into iterations and therefore sums them; the modern reader reports the parameterized node's owndurationInSecondsand does not sum itsArgumentschildren.Candidate fix
Have the modern reader sum the per-argument execution durations for a parameterized case, if
xcresulttool'sArgumentsnodes expose them — the answer-8 move, converging the two backends by construction rather than by mask.If the format does not expose per-argument durations, the alternative is the other direction the design prefers: hold legacy down to modern's capability, i.e. stop summing for the arguments case. Either way the divergence goes away instead of being masked.
Why it needs an issue rather than living under the mask
The spec is explicit that a mask is the worse of two outcomes:
and
differential-allowlist.jsonsays an entry means "accepting a permanent difference between the backends — a design decision, not a way to quiet a failing test." No entry is being added here — this divergence already falls under the existingdurationsrule, and has since the parameterized fixture landed — but #476's summary header is the first place it aggregates into a single number, which is what made it visible. Filing it so the mask is a known debt with an owner rather than a rug.When this is fixed
Two things come out, both named in the code:
DifferentialTests.testSummaryHeaderNumbersAgreeAcrossBackends, which today skips this one identifier from an otherwise exact per-leaf duration comparison;Sources/XCTestHTMLReportCore/Classes/Models/RunSummary.swiftexplaining why the header's total is written parenthesised.The header's duration can then be written in any shape, because it would no longer need the
durationsrule to cover it.Found while implementing #439's summary header (#476).