Skip to content

Reports are not reproducible: fresh UUIDs are minted on every run #411

Description

@tylervick

The same binary run twice against the same .xcresult produces different HTML.

run A: selectDevice('6CBE1E1A-E712-42AD-BAA8-A179C1823B56')
run B: selectDevice('E2105793-8D30-4944-AF86-2BBC4017C91F')

Byte counts are identical (UUIDs are fixed width); the content differs.

Cause

Five call sites mint a fresh UUID() at parse time, used as DOM element ids and JS handles:

  • Models/TestSummary.swift:51 — uuid = UUID().uuidString
  • Models/TargetDevice.swift:21 — uniqueIdentifier = UUID().uuidString
  • Models/Test.swift:58 and :184 — let uuid = UUID().uuidString
  • Models/Iteration.swift:12 — let uuid = UUID().uuidString

Why it matters

Rendering is unaffected — these are internal handles, not user-visible. But it means:

  • Two reports from the same bundle cannot be diffed to see what actually changed
  • Reports cannot be content-addressed or cached by hash
  • Any future golden-file testing is impossible without normalising them first

TargetDevice is the clearest case: record.identifier is already read from the bundle on the line above, and then a random uniqueIdentifier is generated alongside it.

Suggested fix

Derive these identifiers deterministically from bundle content rather than randomly — the device identifier, the test's full name, the iteration index. Anything stable across runs for the same input.

Found while verifying that a SwiftFormat reformat had not changed generated output (#410). It hadn't; this had been true all along.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions