Skip to content

Write attachment test reports to a temporary directory - #420

Open
sunnnn2005 wants to merge 2 commits into
XCTestHTMLReport:mainfrom
sunnnn2005:agent/use-temporary-report-output
Open

sunnnn2005 wants to merge 2 commits into
XCTestHTMLReport:mainfrom
sunnnn2005:agent/use-temporary-report-output

Conversation

@sunnnn2005

@sunnnn2005 sunnnn2005 commented Aug 10, 2026 •

Copy link
Copy Markdown

Summary

  • write attachment-test reports to a unique temporary directory
  • remove the temporary output after each assertion completes
  • keep generated index.html files out of the package test resources

Root cause

The attachment tests passed -r without -o, so xchtmlreport used the .xcresult parent directory as its output directory. That left Tests/XCTestHTMLReportTests/Resources/index.html untracked after running swift test.

Validation

  • swift test — 23 tests passed, 1 existing Xcode 26 skip
  • swift test --filter CliTests — 7 tests passed
  • SwiftFormat lint — no formatting violations
  • verified swift test leaves no Resources/index.html

Fixes #389

Summary by CodeRabbit

  • Tests
    • Improved attachment-report test isolation by using unique temporary output directories.
    • Added automatic cleanup of temporary test files after execution.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9845c08d-394f-4453-b12a-83139f8af9e3

📥 Commits

Reviewing files that changed from the base of the PR and between 8cf974e and ba5d9d0.

📒 Files selected for processing (1)
  • Tests/XCTestHTMLReportTests/CliTests.swift

📝 Walkthrough

Walkthrough

CLI tests now write generated reports to unique temporary directories and remove those directories after execution. Retry, attachment, and lenient-mode tests no longer use the default output location.

Changes

CLI test output isolation

Layer / File(s) Summary
Temporary output directories for CLI tests
Tests/XCTestHTMLReportTests/CliTests.swift
The tests create unique temporary directories, pass them through -o, and clean them up during teardown.

Estimated code review effort: 2 (Simple) | ~5 minutes

Suggested reviewers: tylervick

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed The tests use unique temporary output directories and clean them up, which satisfies issue #389 and prevents generated Resources/index.html files.
Out of Scope Changes check ✅ Passed The changes are limited to isolating report-producing CLI tests in temporary directories and cleaning up their output.
Title check ✅ Passed The title clearly describes the primary change: writing attachment test reports to a temporary directory.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@sunnnn2005
sunnnn2005 marked this pull request as ready for review August 10, 2026 00:21
@sunnnn2005

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Already reviewed.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Tests/XCTestHTMLReportTests/CliTests.swift`:
- Line 68: Update testRetryBundleWithSharedPayloadRefsExitsZero and
testLenientFlagIsAccepted to invoke xchtmlreportCmd with an isolated temporary
output directory, matching the defaultArgs setup used by assertAttachmentsExist.
Ensure each test creates and cleans up its temporary output location; if
default-output behavior must remain covered, execute it from an isolated
temporary bundle parent.
- Line 68: Update ResultFile.exportPayload so relocated report attachments are
self-contained: either copy assets beneath outputDirectory or emit URLs relative
to outputDirectory instead of TestResults.xcresult. In the CLI test using
defaultArgs, resolve emitted URLs against outputDirectory and assert the
resulting assets exist at those locations, covering links from
outputDirectory/index.html.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9bc2923e-b34f-4c48-b4f1-af18bbdd7f0d

📥 Commits

Reviewing files that changed from the base of the PR and between ee40463 and 8cf974e.

📒 Files selected for processing (1)
  • Tests/XCTestHTMLReportTests/CliTests.swift

Comment thread Tests/XCTestHTMLReportTests/CliTests.swift

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

swift test leaves an untracked Resources/index.html

1 participant