Skip to content

fix(tests): isolate inherited skill configuration in CLI subprocesses #20

Closed

Description

@showxu

Summary

CLI tests can write state outside their per-test home because runSkills copies the parent process environment. A user-scope test that overrides only HOME still inherits XDG_STATE_HOME. Several agent/config/cache overrides can also remain inherited, making tests depend on or modify the caller's configuration.

Code evidence at 5d4987b:

  • Tests/CoreTests/CLISurfaceTests.swift:981 starts from the full parent environment and overlays supplied values.
  • Tests/CoreTests/CLISurfaceTests.swift:560 runs a user-scope add with only a HOME override at :572.
  • Tests/CoreTests/CLISurfaceTests.swift:1017 clears CODEX_HOME and CLAUDE_CONFIG_DIR and sets XDG_CONFIG_HOME for some tests, but omits XDG_STATE_HOME and XDG_CACHE_HOME; other call sites use only HOME.
  • Sources/Core/InstallLock.swift:110 prioritizes inherited XDG_STATE_HOME; Sources/Core/AgentRegistry.swift:88, :167, :173, and :184 consult cache/agent/config overrides.

Reproduction

Executed at 5d4987b. The executed case demonstrates XDG state leakage. Agent-specific override inheritance is established by the helper and registry code, not by a claimed run against real agent directories. The wrapper removes newly created OS fixture roots; run without concurrent test processes.

Run in Bash after cd "<checkout>" && swift build, replacing <checkout> with the absolute path of the checkout at this revision. Use a fresh reproduction directory. The commands isolate HOME, agent configuration, and XDG_*, and remove their fixture root on exit.

set -eu
checkout="<checkout>"
skill_bin="$checkout/.build/debug/skill"
repro_root="$checkout/.build/issue-repro-cli-environment"
mkdir -p "$repro_root/home" "$repro_root/tmp" "$repro_root/project"
trap 'rm -rf "$repro_root"' EXIT
isolated() {
  env -i PATH="$PATH" HOME="$repro_root/home" \
    CODEX_HOME="$repro_root/home/.codex" \
    CLAUDE_CONFIG_DIR="$repro_root/home/.claude" \
    XDG_CONFIG_HOME="$repro_root/home/.config" \
    XDG_STATE_HOME="$repro_root/home/.local/state" \
    XDG_CACHE_HOME="$repro_root/home/.cache" TMPDIR="$repro_root/tmp" \
    GIT_CONFIG_NOSYSTEM=1 GIT_CONFIG_GLOBAL=/dev/null \
    GIT_AUTHOR_NAME=Fixture [email protected] \
    GIT_COMMITTER_NAME=Fixture [email protected] \
    "$@"
}
skill() { isolated "$skill_bin" "$@"; }
cd "$repro_root/project"


cd "$checkout"
# Foundation may choose the OS temporary directory independently of TMPDIR.
fixture_base="$(isolated swift -module-cache-path "$repro_root/module-cache" -e 'import Foundation; print(FileManager.default.temporaryDirectory.path)')"
# Run without other test processes; track only the fixture roots created here.
isolated python3 - "$fixture_base" "$repro_root" <<'CODE'
import json, os, shutil, subprocess, sys
from pathlib import Path
base=Path(sys.argv[1])
root=Path(sys.argv[2])
prefixes=['skill-cli-tests','skill-cli-cli-tests']
before={p for name in prefixes for p in (base/name).glob('*')}
try:

    result=subprocess.run(['swift','test','--filter','installsUserScopedGeminiSkillInUniversalHome'])
    print('test exit:', result.returncode)
    p=root/'home/.local/state/skills/skills.resolved'
    print('resolved file written to inherited XDG_STATE_HOME:', p.exists())
    for pin in json.loads(p.read_text())['pins']:
        for skill in pin['skills']:
            for install in skill['installations']:
                print('skill:', skill['name'], 'installation in separate fixture root:', str(base/'skill-cli-cli-tests') in install['path'])

finally:
    after={p for name in prefixes for p in (base/name).glob('*')}
    for p in after-before:
        shutil.rmtree(p)
CODE

Observed output (relevant lines; temporary paths shortened to <temp>):

test exit: 0
resolved file written to inherited XDG_STATE_HOME: True
skill: gemini-demo installation in separate fixture root: True

Expected Behavior

Have the CLI subprocess helper construct a controlled environment by default, routing every state/cache/config location into the test's owned root. Tests that exercise override behavior should opt into explicit fixture paths.

Completion checks:

  • Set parent CODEX_HOME, CLAUDE_CONFIG_DIR, and all relevant XDG_* variables to separate sentinel directories; CLI tests leave them unchanged.
  • User-scope installs write their resolved state under the fixture's own root.
  • Coverage includes callers currently passing only HOME and callers using isolatedCLIEnvironment.
  • Required toolchain/PATH values remain available; run swift test.

Actual Behavior

The user-scope Gemini test passes while placing skills.resolved in the inherited state directory, outside the home created for that test. Its installation path points to a separate fixture root. All paths used by this reproduction are disposable; the demonstrated escape is from the per-test root, not from the outer reproduction environment.

Related: #10 concerns product watch scope, not test subprocess environments. PR #12 uses the partial isolation helper in newer tests; it does not close this gap. No duplicate found.

Environment

  • Source revision: 5d4987b (5d4987b9e1f94e8b9f85124299f05d5aa4f45f07).
  • CLI version: 0.2.1; built successfully with swift build.
  • Swift: Apple Swift 6.4 (swiftlang-6.4.0.34.1, clang-2100.3.34.1).
  • OS: macOS 27.0.1 (26A434).
  • Architecture: arm64.
  • Git: 2.42.0. Validation date: 2026-10-17.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions