Make the tests tree read like Xcode's outline (refs #439) - #483
Conversation
Second of three serial PRs implementing Option A. A1 gave the report a summary header; this one restyles everything below it except the filter pills, which are A3's. Every row in the tree — suite, test case, iteration, activity, attachment — becomes one flex line: disclosure triangle, status badge, name, and the duration in a right-hand column of its own. What that replaces is a pair of floats clearing a fixed 52px of left padding on every `<p>` whether or not the row had a badge to put there, which is why the duration could only ever be more text after the name. The status glyph becomes Xcode's rounded badge instead of #459's diamond. The same six shapes survive — that was the property #459 bought, that status is never colour alone and no state draws a blank cell — and suite rows join them, which #459 could not do while the badge was a float and a heading meant reworking the gutter. The glyph stays a knock-out rather than the mockup's white tick: a hole's contrast against its fill is by construction the status token's contrast against the row, which #439 already sized to clear 3:1 in both themes, where white on the mockup's green is 3.13:1. Nesting is finally visible. Depth is applied to the child container and never written into the markup as a number: the legacy backend interposes two wrapper levels the modern one does not, so a depth attribute would differ between the two renders in every row of the tree, and the `wrapperGroups` mask unwraps the element and would leave the numbers behind. `differential-allowlist.json` is untouched. An expanded test gets the mockup's timeline treatment minus the one part 4.0 has no data for: the panel is inset and its gutter is ruled, and there are no per-activity elapsed offsets, because `ParsedActivity` has carried no timestamps since the port and inventing them would be fiction. Three things found while shooting this and fixed here: - Below 700px the test list had no `min-height: 0`, so it grew to hold every row instead of scrolling inside its share. `body` sets `overflow: hidden`, so on a long run the last test in the report could not be brought on screen at all. Pre-existing, and worse since A1 spent height above the tree. - The scrolling list was never keyboard-reachable — axe's `scrollable-region-focusable`, serious. The gate stayed green only because the synthetic fixture fell seven pixels short of overflowing. - On a selected row the status badge was a 1.36:1 stain on the accent. It now takes the selection's own text colour, at 5.71:1. The Logs tab reclaims the summary band, which describes a test run and says nothing about a log. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughThe report templates now generate a responsive flex-based test tree with structured rows, status badges, durations, activity panels, bounded previews, keyboard focus, and Logs/Tests summary visibility. Snapshots, attachment masking, accessibility checks, and responsive browser tests reflect the new markup. ChangesReport UI redesign
Estimated code review effort: 4 (Complex) | ~60 minutes Merge Risk: ⚪ Minimal · up to The current changes update test-report presentation and interaction behavior, and no concrete merge-blocking correctness, security, availability, or readiness risk remains in the supplied evidence. Possibly related issues
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
#439) Six review findings on #483. None was a blocker; all six are addressed here. The full-width row bleed was inert. `box-shadow: -100vw 0 0 <colour>` is the border box translated whole, so the offset that clears the indentation also drags the box's right edge that far left: at -100vw the entire shadow landed at x <= 0, outside the viewport and outside the scroll container clipping it, and painted nothing. Rows hovered and selected from the indent rightwards, which is the opposite of what deviation 13 claimed. Shipped rather than deleted, because the mockup is unambiguous — its failure tint and its expanded-test band both run to the card's edge, under the disclosure column rather than starting after it. Replaced with a strip painted by `::before` and anchored `right: 100%`, so it begins where the row begins and runs left for as far as it is told, with no arithmetic that a deep enough tree or a narrow enough pane can outrun. `background-color: inherit` means hover, selection and a suite heading's own ground each reach the gutter by the same declaration that paints the row. Pixel-verified at 1440 and 375, both themes, hover and selection, on the synthetic fixture and on a real 3-deep render: every gutter pixel now matches the row's computed background, and reverting to the shadow turns all sixteen readings back to bare page colour. Nothing widens: leftward overflow is unreachable in a left-to-right scroll container, and the 375px probe holds. The axe gate was worse than "live by accident". `scrollable-region-focusable` analyses a region only if it overflows, and this fixture's layout does not settle on `load`: four `img.screenshot-tail` in the tree are `loading="lazy"` and point into a `.xcresult` that is not there, and a deferred image box is 2px until its load is attempted and 20px once the broken-image placeholder replaces it. So the tree measured 373px or 426px from the same file — 0px of overflow or 53px — and under a parallel run it was the wrong one about a fifth of the time, with axe analysing a tree that had no scrollable region at all. `a11y.spec.ts` now settles the images and then asserts the overflow. Both halves are needed: waiting alone leaves the seven-pixel blindness, asserting alone turns a real race into a flaky test. Mutation-checked both ways — stripping the `tabindex` reddens the gate, and giving the pane room to fit fails the precondition with the measurement in the message. The `attachmentDisplayNames` masker is narrowed on both counts. `(?:left )?` is gone: the masker only ever sees two renders of the same build, so an anchor matching a pre-A2 report bought a cross-revision property nothing consumes. The replacement is anchored on the `row-name` wrapper and takes only the text inside it — the entry declares that display *names* diverge, and the wrapper markup was not part of that claim. Same nine matches as before on both goldens, but the wrapper survives into the compared text: 9 dropped before, 0 now. Reverting the anchor still turns `testMaskedRendersAreIdenticalAcrossBackends` red, so the edit remains necessary rather than a laundering of the differential. Two comments that shipped inside every generated report were false. The dark theme's said the activities panel is darker than the page and that this inverts the light theme; it is lighter here and darker there, which is one step off the page in whichever direction the theme has. A1's said nothing below the header was restyled — A2 restyled the tree. 153 tests / 3 skipped / 0 failures on both reader legs, DifferentialTests 7/7 on both, visual 16/16 across twenty consecutive runs, swiftformat clean, swiftlint 35 violations 0 serious.
Refs #439. Second of three serial PRs implementing Option A — Xcode-native, on top
of A1's summary header (#476). This one is the tree and its rows: everything below
the header except the filter pills, which are A3's along with #460.
What lands
duration in a right-hand column of its own. It replaces a pair of floats clearing a
fixed 52px of left padding on every
<p>in the tree, badge or no badge, which is whythe duration could only ever be more text after the name.
suite rows get one too — C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439) #459 left headings out because the badge was a float and
giving one to a heading meant reworking the gutter. Nothing new is computed for it:
TestGroup.statusalready folds its children and already writes the class on the row.whole tree was flat, so
Selected tests,SampleAppUITests.xctestandFirstSuiteall began at the same pixel.
gutter, no per-row hairlines, and failure steps on their own tint.
accent the rest of the sheet uses for "you can click this".
slab — at 200px tall and unbounded in width it was the tallest thing in the tree by a
factor of eight, and
.screenshot-tailis emitted between rows, so it did that tothe tree rather than to the panel.
--color-row-hover,--color-bg-activities,--color-fail-tint,--color-tree-guide, plus a repointed--color-preview-border)and two non-colour ones (
--tree-indent,--row-bleed). Every colour the tree paintscomes from the token layer; the dark block overrides values only.
their colour from a token and
-isingle-file reports keep working byte for byte.Three found-fixes
These were not in the brief. The first two were found while shooting the 375px
screenshots and are pre-existing; the third is a defect the restyle would have inherited.
The ≤700px scroll lock. Below 700px
#containeris a column, so the test list issized on the block axis by a flex item's automatic minimum — its content's height.
With no
min-height: 0the list grew to hold every row instead of scrolling insideits share: measured at 375×500, a 376px pane hanging 192px below the viewport.
bodysets
overflow: hidden, so that overflow was not merely unscrolled but unreachable— on a long run the last test in the report could not be brought on screen at all.
Pre-existing (the column layout arrived with C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439) #459) and made materially worse by A1,
which spends real height above the tree.
The regression test scrolls with the wheel, not
scrollIntoView: a programmaticscroll moves an
overflow: hiddenbox perfectly well, which is exactly why thissurvived — the row was reachable by script and unreachable by the reader. Verified by
mutation: with the declaration removed the test fails, with it restored it passes.
scrollable-region-focusable(axe, serious). The scrolling list has no focusabledescendant, so a keyboard user could not scroll it. This has always been true of any
real report; the gate stayed green only because the synthetic fixture's rows fell
seven pixels short of overflowing the pane, so the rule never fired on the one
page CI checks.
tabindex="0"plus a:focus-visiblering fixes it: Page Up/Down,Home and End now scroll the list, and the arrow keys keep going to the existing row
navigation, which scrolls the same region by moving the selection.
The invisible badge on a selected row. The status glyph kept its status hue on the
accent fill — passed measured 1.36:1 light and 2.07:1 dark, i.e. no badge at
all on precisely the row the reader was looking at. It now takes the selection's own
text colour and lets the accent knock through the glyph: 5.71 / 4.82. The outcome
stays readable because the shape carries it, which is the C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439) #459 property that status
is never colour alone.
The Logs tab reclaims the summary band
The header describes the test run — an outcome ring, a failure digest, per-device bars
— and none of it says anything about the log the Logs tab shows, so it stands down while
the log is up, the way Xcode gives a different report the whole column. One body class
and one
display: none, deliberately not a height animation: the band is up to half theviewport and the log is an iframe, so anything that resized it over several frames would
reflow and repaint the frame the whole way down. One class, one reflow, no jank.
Asserted at both 1440 and 375, including that it comes back, and verified by mutation.
Boundary: that is the minimal version, and it is where A2 stops. The fuller
shell/tabs/sidebar rethink — whether the three-pane frame survives at all, where the tabs
live, what happens to the device rail — is A3's, together with the filters and #460.
Why nothing here can move the differential
DifferentialTestsrenders every fixture through both readers and requires byte equalityafter masking the four declared losses. No allow-list entry was added or changed;
differential-allowlist.jsonis not in this diff.Indentation is CSS, not markup. Depth is applied to the child container and never
written into the markup as a number. That is not a stylistic preference. The legacy
backend interposes two wrapper levels (
Selected tests,<target>.xctest) that themodern backend does not, so a depth attribute would differ between the two renders in
every row of the tree — and
KnownLossMasker'swrapperGroupsrule unwraps theelement, which would leave the numbers behind with no allow-list entry able to cover
them. Nested padding indents by construction and costs the differential nothing.
The duration keeps its masked shape.
[[TITLE]]and([[DURATION]])are separateelements now so the duration can be a column, but the rendered text is unchanged:
whitespace between the two spans keeps
text()readingname (1.23s).(N.NNs)isliterally what the
durationsrule matches, so every duration in the tree still inheritsa loss the allow-list already declares — the same discipline A1 applied to the header
total.
wrapperGroupsreads a suite heading's text to recognise the legacy-only wrapperlevels, and
CoreTestslooks tests up byhasPrefixon the same reading; both still seewhat they saw.
Contrast
Every pairing computed before it was used. Text floor 4.5:1, non-text (WCAG 1.4.11) floor
3:1.
visual/tests/tokens.spec.tsenforces the text column automatically from the livecascade in both themes and passes.
Text (floor 4.5:1) — light / dark
.row-name(primary).row-duration,.row-note(muted)Status badges and row icons (non-text, floor 3:1) — light / dark
--status-passed--status-failed--status-skipped/--status-unknown--status-expected--status-mixed.drop-down-icon(muted)Tightest badge: 3.69:1 (passed, on a hovered row, light), against a 3:1 floor. Because
the glyph is a knock-out, that number is also what the tick inside the badge is worth
against the fill around it — which is the whole reason for preferring a hole to the
mockup's white tick, whose own figure is 3.13:1.
Single-ground pairings
--color-on-accenton--color-accent--status-failedon--color-fail-tint.preview-icon—--color-accent-texton--color-bg-activitiesTightest text pairing in the whole tree: 4.74:1, against a 4.5 floor.
Deliberately below 3:1
--color-tree-guide(1.28 light / 1.44 dark, against the panel it is drawn on) and--color-preview-border(1.51 / 1.60). Both are decorative under 1.4.11: the timelinerule restates the nesting that indentation already states, and the frame outlines an
image that is itself the content. Drawing either at 3:1 would put a hard bar down the
middle of every expanded test — the same reasoning A1 recorded for
--color-summary-border.--color-preview-borderwas#021A40before this, a near-blacknavy and the only colour in the light theme belonging to no family; the dark theme already
had a normal border value for it.
Deviations from the mockup, and why
Knock-out glyphs, not the mockup's white ones. See above — it inherits the
already-vetted token-vs-surface contrast instead of introducing new unmeasured
pairings. Identical picture on a white row, a measured one everywhere else.
The corrected palette, not the mockup's. Status colours are C's contrast-vetted
tokens, not the raw Apple values — the same call A1 recorded as its deviation 2. No
status token changed in this PR.
No per-activity elapsed offsets. They are the mockup's answer to the missing
per-activity durations, and 4.0 cannot give them:
ParsedActivitycarries a title, afailure flag, attachments and sub-activities —
finishand the timestamps left withthe legacy reader (the port's decision 2). Nothing is faked; the gutter carries the
structure the offsets would have shared.
No source-location chip. The mockup pulls
FirstSuite.swift:69out into amonospace chip beside the message. Our file and line are inside the activity title
string and the two backends format that string differently — the standing
failureTitlePrefixknown loss — so splitting it out is model work, not a restyle.Suite rows draw no disclosure triangle. Collapsing a group is a script change:
toggle()looks foriterations-/activities-/attachments-ids a group has noneof, and wrapping its children in one would break
hideSummaryGroupsIfNeeded, whichwalks a group's direct
.test-summarychildren. That is A3's to rewire, so A2 showsno control it cannot honour. The column is held open with
visibility: hiddenso asuite's badge still lines up with its children's.
The blue test tile is gone from tree rows. The mockup has no equivalent, the badge
beside it already said "this row is a test", and it cost 20px of a 375px row to repeat
a fact. Nothing selects it;
--icon-testand its four rules go with it.Expected-failure is a filled amber badge (the mockup's treatment) rather than
C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439) #459's outlined one;
unknownstays outlined. The filled/outlined split now meanssettled versus not — a test that failed exactly as it declared it would is settled,
and unknown is the only state that means "we could not tell". All six glyphs stay
distinct, which is the C-refresh part 2/2 — structure: responsive collapse, SVG icons, all six status glyphs, type-derived attachment labels (refs #439) #459 property this had to preserve.
The status glyph tokens are shared, so the header's digest and summary icons and
the title icon inherit the new badge silhouette. Deliberate: a diamond in the header
and a badge in the tree would be two iconographies on one page. No A1 code is touched.
Attachments keep the preview pane. The mockup plays media inline under the
activity; the report has a working attachment pane and rewiring it is not a restyle.
The rows themselves get the mockup's treatment.
The indentation step is 10px below 700px, not 16. Sixteen is right at 1440 and
profligate on a 375px screen four levels deep, where the legacy backend's two wrapper
levels alone would eat 32px before the first real suite.
The duration reads
(13.71s), not the mockup's13.71 s. The parentheses arethe shape
KnownLossMasker'sdurationsrule normalises — dropping them would leavea declared known loss uncovered in every row of the tree. Same call A1 made for the
header total, for the same reason.
The activities panel is indented and its gutter is ruled. The mockup bounds it
with a full-bleed inset band and top/bottom hairlines only, because its elapsed-offset
column already anchors the region on the left. Without that column the panel had no
left-hand anchor at all, so the indent and the 2px rule take its place.
Row backgrounds bleed past the indentation. Rows are indented by their
container's padding, so a row's own box starts at the indent;
--row-bleedcarries thebackground back out into the gutter that indentation spent, where
.testsclips it.The mockup does this — its failure tint and its expanded-test band both run to the
card's edge, under the disclosure column rather than starting after it. Painted by a
::beforestrip anchoredright: 100%, taking its colour from the row withbackground-color: inherit. Leftward overflow is unreachable in a left-to-rightscroll container, so nothing widens the page — covered by the 375px probe. Activity
and attachment rows deliberately keep the
0default: they sit inside the activitiespanel, and a row bleeding past it would erase the very thing that says "this belongs
to the test above".
Corrected after review. This first shipped as
box-shadow: -100vw 0 0 <colour>,which painted nothing: an outer shadow is the border box translated whole, so the
offset that clears the indentation also drags the box's right edge off the left of the
viewport. Now pixel-verified at 1440 and 375, both themes, hover and selection, on the
synthetic fixture and a real 3-deep render.
Test-pin changes
KnownLossMasker,attachmentDisplayNames— the anchor regex dropsleftfromthe type-icon span. A2 drops that float class from the attachment type icon, and without
the edit the rule silently stops matching and the differential goes red; reverting it
still turns
testMaskedRendersAreIdenticalAcrossBackendsred, which is what makes theedit necessary rather than convenient. Deleted outright rather than made optional
(review finding 4): the masker only ever sees two renders of the same build, so an
anchor that also matched a pre-A2 report bought a cross-revision property nothing in the
repo consumes. The replacement is anchored on the
row-namewrapper and takes only thetext inside it, not the whole line (review finding 5), so the wrapper markup stays
compared — same nine matches on both goldens, but nine
row-namewrappers were beingdropped from the comparison before and none are now. The allow-list entry is
unchanged; this is the rule that implements it, in test code.
visual/tests/a11y.spec.ts— the axe gate now settles the fixture's lazy images andthen asserts that the tests pane overflows before analysing (review finding 3).
scrollable-region-focusableonly fires on a region that overflows, and this fixturedid not settle on
load: fourimg.screenshot-tailareloading="lazy"and point intoa
.xcresultthat is not there, and a deferred image box is 2px until its load isattempted and 20px once the broken-image placeholder replaces it. The tree therefore
measured 373px or 426px from the same file — 0px of overflow or 53px — and under a
parallel run it was the wrong one about a fifth of the time, with axe analysing a tree
that had no scrollable region at all. Both halves are needed: waiting alone leaves the
seven-pixel blindness the fix was for, asserting alone turns a real race into a flaky
test. Mutation-checked both ways — stripping the
tabindexreddens the gate, giving thepane room to fit fails the precondition.
visual/tests/behaviour.spec.ts— three new browser tests, all three verified bymutation (removing the fix fails the test; restoring it passes):
Snapshots/index.htmlandindex-inline.htmlrefreshed. The diff is thestylesheet and the row templates and nothing else; the only markup deletions are the
test-iconspans, theleftclasses, and theStatus/Testscolumn labels.No assertion was weakened or deleted anywhere.
Verification
swift test, both CI legs:XCHR_RESULT_READER=auto153 tests, 3 skipped, 0failures;
=modern153 tests, 3 skipped, 0 failures. Run repeatedly rather thanonce — see the note below.
DifferentialTestsgreen on both legs, against locally generatedTestResults,RetryResultsandSanityResults;differential-allowlist.jsonuntouchedvisual/— 16/16, including axe-core (no critical or serious) and the automated4.5:1 / 3:1 contrast gates in both themes
every test and activity expanded: the page does not scroll sideways, the tree does not
scroll sideways, and nothing crosses the viewport edge outside an intentional scroll
container
A local-only flake, named so it is not mistaken for this diff
Three separate local runs failed one test each and then passed on identical source:
PayloadExportFaultTests.testInlineExportDoesNotLeaveItsTempFileBehind,ReproducibilityTests.testMintedIdentifiersNeedNoEscapingInHTMLOrJavaScript("report wasdegraded"), and — in an earlier round —
ModernPayloadStoreTests. All three are the samesymptom:
xcresulttool exportwrites every payload to a sharedNSTemporaryDirectory()/<id>path, and this machine has ten worktrees of this repo whosesuites collide there. It is already a recorded property of this checkout, not a discovery.
It cannot be this diff. The five files touched are template strings, a test-only regex,
two goldens and a Playwright spec; none of them is on the payload-export path
(
ResultFile,ModernPayloadStore,FaultCollector) and none of them is reachable fromit.
The evidence, stated as it actually came out rather than rounded in my favour: on the
final commit, auto is 5 clean of 7 and modern 4 clean of 4, each failure a
different test and none reproducing. Detached
mainon the same machine ran the auto leg3 times clean, so it did not reproduce there either — three runs is not enough to call
that a difference, and the sample is too small to conclude anything from it. What settles
it is CI, which gives each job its own runner and where this class of collision cannot
occur: both legs are green on this PR —
test (auto)8m33s,test (modern)4m48s,alongside
visual,dump, bothbuildmatrix legs,swift,shellandactions. Tenof ten.
Screenshots
Real report (
TestResults.xcresult, 21 tests / 6 failures / 2 expected failures) and thesynthetic fixture, which is the only one that exercises expected-failure, skipped and
mixed at once. Thirty-two PNGs at 1440 and 375, light and dark, in
orca-artifacts/track3-a2-tree/: full page, tree-only crop, a crop with a row selectedand another hovered (which is where the badge fix is visible), and the Logs tab. The same
four crops of the mockup's tree card sit beside them (
mockup-tree-*.png) so thecomparison is a picture rather than a description.
Visual review sheet — mockup against shipped at both widths and both themes, the
before/after pair, the full contrast tables and every deviation with its reasoning:
https://claude.ai/code/artifact/1ad12f0e-233e-4447-931a-2f028923b345
Summary by CodeRabbit
New Features
Bug Fixes