Skip to content

fix(tests): remove temporary fixture roots after each test #21

Closed

Description

@showxu

Summary

Several test helpers create UUID-named fixture directories but their callers never remove them. A single successful watch test leaves three roots, including a source Git repository and a cached clone. Repeated local or CI runs accumulate files independently of .build.

Code evidence at 5d4987b:

  • Tests/CoreTests/CoreTestSupport.swift:5 allocates a new directory under FileManager.default.temporaryDirectory without a lifecycle owner.
  • Tests/CoreTests/WatchServiceBehaviorTests.swift:194 allocates project, home, and remote roots and finishes at :243 without cleanup.
  • Tests/CoreTests/CLISurfaceTests.swift:1009 has a second allocation helper; the test at :560 also has no cleanup.
  • Tests/CoreTests/CLISurfaceTests.swift:8 is an example that already uses deferred cleanup; this report does not claim every test leaks.

Reproduction

Executed at 5d4987b. The retained files are counted after test completion and before wrapper cleanup. Run without concurrent test processes so the before/after inventory is attributable to this test.

Run in Bash after cd "<checkout>" && swift build, replacing <checkout> with the absolute path of the checkout at this revision. Use a fresh reproduction directory. The commands isolate HOME, agent configuration, and XDG_*, and remove their fixture root on exit.

set -eu
checkout="<checkout>"
skill_bin="$checkout/.build/debug/skill"
repro_root="$checkout/.build/issue-repro-fixture-cleanup"
mkdir -p "$repro_root/home" "$repro_root/tmp" "$repro_root/project"
trap 'rm -rf "$repro_root"' EXIT
isolated() {
  env -i PATH="$PATH" HOME="$repro_root/home" \
    CODEX_HOME="$repro_root/home/.codex" \
    CLAUDE_CONFIG_DIR="$repro_root/home/.claude" \
    XDG_CONFIG_HOME="$repro_root/home/.config" \
    XDG_STATE_HOME="$repro_root/home/.local/state" \
    XDG_CACHE_HOME="$repro_root/home/.cache" TMPDIR="$repro_root/tmp" \
    GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null \
    GIT_AUTHOR_NAME=Fixture [email protected] \
    GIT_COMMITTER_NAME=Fixture [email protected] \
    "$@"
}
skill() { isolated "$skill_bin" "$@"; }
cd "$repro_root/project"


cd "$checkout"
# Foundation may choose the OS temporary directory independently of TMPDIR.
fixture_base="$(isolated swift -module-cache-path "$repro_root/module-cache" -e 'import Foundation; print(FileManager.default.temporaryDirectory.path)')"
# Run without other test processes; track only the fixture roots created here.
isolated python3 - "$fixture_base" "$repro_root" <<'CODE'
import json, os, shutil, subprocess, sys
from pathlib import Path
base=Path(sys.argv[1])
root=Path(sys.argv[2])
prefixes=['skill-cli-tests','skill-cli-cli-tests']
before={p for name in prefixes for p in (base/name).glob('*')}
try:

    child=dict(os.environ, GIT_CONFIG_COUNT='1', GIT_CONFIG_KEY_0='init.defaultBranch', GIT_CONFIG_VALUE_0='master')
    result=subprocess.run(['swift','test','--filter','refreshesRemoteWatchedHeadCheckout'], env=child)
    print('test exit:', result.returncode)
    after={p for name in prefixes for p in (base/name).glob('*')}
    retained=after-before
    print('retained fixture roots:', len(retained))
    print('retained Git repositories:', sum(len(list(p.rglob('.git'))) for p in retained))

finally:
    after={p for name in prefixes for p in (base/name).glob('*')}
    for p in after-before:
        shutil.rmtree(p)
CODE

Observed output (relevant lines; temporary paths shortened to <temp>):

test exit: 0
retained fixture roots: 3
retained Git repositories: 2

Expected Behavior

Give temporary fixtures a scoped lifecycle and remove their owned roots on both success and failure. Make failure-artifact retention explicit if it is useful.

Maintainer decision: choose whether failed fixtures are retained through an opt-in diagnostic mode; default test runs should have a defined cleanup policy.

Completion checks:

  • The named test leaves no new fixture roots after a normal run.
  • Verify cleanup on a thrown error and with parallel tests; delete only each test's own roots.
  • Audit the other helper families and callers rather than fixing only this example.
  • Run swift test and compare the dedicated fixture-root inventory before/after.

Actual Behavior

After the passing test returns, three fixture roots and two .git directories remain. The reproduction wrapper counts these before removing them itself. On the tested macOS runtime, Foundation's temporary directory is not redirected by setting TMPDIR.

Duplicate check: no matching open/closed issue or PR. PR #12 cleans up some newer fixtures; older helper callers still retain theirs.

Environment

  • Source revision: 5d4987b (5d4987b9e1f94e8b9f85124299f05d5aa4f45f07).
  • CLI version: 0.2.1; built successfully with swift build.
  • Swift: Apple Swift 6.4 (swiftlang-6.4.0.34.1, clang-2100.3.34.1).
  • OS: macOS 27.0.1 (26A434).
  • Architecture: arm64.
  • Git: 2.42.0. Validation date: 2026-10-17.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions