Skip to content

Create the --output directory up front instead of failing late on index.html (fixes #446) - #471

Merged
tylervick merged 1 commit into
mainfrom
tylervick/output-dir-446
Aug 14, 2026
Merged

tylervick merged 1 commit into
mainfrom
tylervick/output-dir-446

Conversation

@tylervick

@tylervick tylervick commented Aug 14, 2026 •

Copy link
Copy Markdown
Member

Fixes #446.

The bug

$ xchtmlreport TestResults.xcresult --output /some/missing/dir/report
... minutes of parsing, rendering and attachment exporting ...
Error: An error has occured while creating the report
Error: The file “index.html” doesn’t exist.
$ echo $?
1

Neither line names the real problem. The failure is the final
html.write(toFile:) in run(), so it arrives after all the work is done —
and in linking mode after payloads have already been exported into the bundle.
index.html is a red herring: the file that doesn't exist is the directory
above it. This is what made the first scheduled toolchain-drift run (#444) look
like toolchain drift.

The fix

Create the output directory, intermediates included, before anything else
happens — before the Summary is built, before parsing, rendering or
exporting. Create-by-default is what every CLI this one sits next to does, and
it is a no-op on the overwhelmingly common path where the directory already
exists.

When the directory genuinely cannot be created — no write permission on the
parent, a file in the way — that is a usage error, not a degraded report:
it throws ValidationError, so it exits 64 like the tool's other usage
errors, never 3, which is reserved for collected faults. The message names the
path and the reason, and arrives before any work:

$ xchtmlreport TestResults.xcresult --output /read-only-dir/report
Error: Could not create output directory /read-only-dir/report: You don’t have
permission to save the file “report” in the folder “read-only-dir”.
Usage: xchtmlreport [<options>] [<results> ...]
$ echo $?
64

The An error has occured while creating the report string (typo included) is
untouched — this fix makes that path unreachable for a missing output
directory, and rewording it is a separate concern.

The #445 workaround comes out

.github/workflows/toolchain-drift.yml gained a mkdir -p "$RUNNER_TEMP/drift-report"
in #445 specifically to stop this bug from being misreported as drift. It is
removed here, with a comment saying why it must not come back: pointing the
drift job at an uncreated path is now part of what the step asserts, so a
regression in the fix would show up as a failed scheduled run rather than
being masked.

(test.yml's mkdir -p "$RUNNER_TEMP/report" is left alone — it came from
#457's render step, not from this bug.)

Tests

Both written before the fix and watched fail against the old behaviour:

  • CliTests.testNonexistentNestedOutputDirectoryIsCreated — renders to a
    nonexistent nested path, asserts exit 0 and that index.html is there.
    Before the fix: exit 1, no index.html.
  • CliTests.testUnwritableOutputDirectoryFailsFastNamingThePath — a 0o555
    parent, asserts exit 64, that stderr names the path, and — the fail-fast
    half — that Building HTML never ran and that stderr no longer blames
    index.html. Before the fix: exit 1, all four assertions failed. Skipped
    when running as root, which ignores directory permissions.

Docs

README.md's exit-code section documented the old behaviour verbatim ("Exit 1
covers failures to write the output — for example -o pointing at a directory
that does not exist"). Corrected. The --output help text now says the
directory is created if missing.

Summary by CodeRabbit

  • New Features
    • The HTML report command now automatically creates missing output directories, including nested paths.
  • Bug Fixes
    • Invalid or unwritable output paths now fail quickly with exit code 64 and a path-specific diagnostic.
    • Output-writing failures continue to use exit code 1.
  • Documentation
    • Updated command-line documentation to clarify output-directory creation and exit-code behavior.

…ex.html (fixes #446)

`xchtmlreport <bundle> --output /missing/dir` parsed the bundle, exported
attachments and rendered the whole report, then died at the final
`html.write(toFile:)` with

    Error: An error has occured while creating the report
    Error: The file "index.html" doesn't exist.

Neither line names the real problem, and both arrive after all the work is
done — in linking mode, after payloads have already been exported into the
bundle. This is what made the first scheduled toolchain-drift run (#444) look
like toolchain drift.

Create the output directory, intermediates included, before the `Summary` is
built. Create-by-default is what every CLI this one sits next to does, and it
is a no-op on the common path where the directory already exists. A directory
that genuinely cannot be created — no write permission on the parent, a file
in the way — is a usage error, not a degraded report: it throws
`ValidationError`, so it exits 64 like the tool's other usage errors rather
than 3, and the message names the path and the reason before any work starts.

The `An error has occured` string is untouched; this fix simply makes that
path unreachable for a missing output directory.

Drop the `mkdir -p` that #445 added to the toolchain-drift workflow as a
workaround for this bug. Pointing that job at an uncreated path is now part of
what the step asserts, so restoring the mkdir would mask a regression.

README's exit-code section documented the old behaviour verbatim ("Exit 1
covers failures to write the output — for example `-o` pointing at a directory
that does not exist"); corrected, and `--output`'s help text now says the
directory is created if missing.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@tylervick tylervick added this to the 4.0 milestone Aug 14, 2026
@coderabbitai

coderabbitai Bot commented Aug 14, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: c66a5414-6ec6-415e-9925-be9510fd06fa

📥 Commits

Reviewing files that changed from the base of the PR and between 0f21dd2 and 21b0491.

📒 Files selected for processing (4)
  • .github/workflows/toolchain-drift.yml
  • README.md
  • Sources/XCTestHTMLReport/XCTestHtmlReport.swift
  • Tests/XCTestHTMLReportTests/CliTests.swift

📝 Walkthrough

Walkthrough

The CLI now creates missing output directories before report generation. It reports directory creation failures as argument errors with exit code 64. Tests, documentation, and the toolchain workflow cover this behavior.

Changes

Output directory handling

Layer / File(s) Summary
CLI output directory creation
Sources/XCTestHTMLReport/XCTestHtmlReport.swift
The output option documents automatic directory creation. run() creates intermediate directories before rendering and converts creation failures into ValidationErrors.
Output directory test coverage
Tests/XCTestHTMLReportTests/CliTests.swift
Tests verify nested directory creation, generated index.html, unwritable-path exit code 64, and path-specific diagnostics.
Documentation and workflow integration
README.md, .github/workflows/toolchain-drift.yml
The README separates directory-creation errors from output-write failures. The workflow passes an uncreated output path to the CLI.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 21b04

The PR creates missing output directories up front and reports uncreatable paths as usage errors; it is merge-ready after normal checks and review, with no actionable merge-blocking risk remaining.

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly states that the --output directory is created early and references the fixed issue.
Linked Issues check ✅ Passed The CLI now creates nested --output directories before report work and fails clearly when creation fails, satisfying issue #446.
Out of Scope Changes check ✅ Passed The workflow, documentation, help text, implementation, and tests all support the --output directory fix.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch tylervick/output-dir-446

Comment @coderabbitai help to get the list of available commands.

@tylervick
tylervick merged commit d8764fe into main Aug 14, 2026
10 checks passed
@tylervick

Copy link
Copy Markdown
Member Author

Post-merge note from review: --output "" exits 64 (an improvement — pre-fix it silently wrote the report into CWD) but the error message blames the CWD basename instead of naming the empty path (XCTestHtmlReport.swift:240-242). Low severity, recorded here rather than as an issue.

@tylervick
tylervick deleted the tylervick/output-dir-446 branch August 14, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

--output pointing at a nonexistent directory fails late with a misleading error

1 participant