Skip to content

test(framework): a real session per tool, on Windows and on Linux #707

Open

Description

@blafourcade

Outcome

Prove that every AIDD telemetry-capable host can complete one real authenticated session on
native Windows and Linux, then journal and report that session through the installed AIDD
plugin.

Evidence already settled

Concern Evidence
Windows storage location %APPDATA%\\aidd when set; an existing .config journal remains authoritative — cli/src/infrastructure/home-dir.ts and cli/src/contexts/telemetry/infrastructure/telemetry-sink-adapter.ts
Linux execution The normal CLI suite runs on ubuntu-latest
Windows execution cli / Windows installs the packed CLI and exercises setup, journalling, diagnosis, unit, integration and e2e paths
Windows privacy POSIX modes carry no useful restriction; the default sink applies per-user ACLs with icacls to the directory and each day file
Cross-platform defects The Windows job has already found defects that macOS could not, including the executable-mode assertion and hook reporting fixed by #749

Captured payloads and stand-in binaries prove the plumbing, but not that a real host emits what
the reader expects. cli/scripts/smoke-real.sh currently proves a real OpenCode journal on the
machine where it is run; it is not a native Windows/Linux matrix.

Scope

  • Includes: Claude Code, Codex, GitHub Copilot, Cursor and OpenCode, the five profiles that
    declare a telemetryJournalHost.
  • Includes: native Windows and Linux execution using each host's own binary and authentication.
  • Includes: AIDD installation, telemetry activation, one minimal real turn, journal inspection
    and aidd telemetry report verification.
  • Excludes: macOS, captured-payload simulations, hosts without telemetry integration, and a
    permanent credentialed CI gate.
  • Temporary manually dispatched jobs may be used. They must not retain credentials, prompts,
    transcripts, session identifiers or machine paths as artifacts.

Done When

Host Windows Linux
Claude Code [ ] [ ]
Codex [ ] [ ]
GitHub Copilot [ ] [ ]
Cursor [ ] [ ]
OpenCode [ ] [ ]

Each checked cell records all of the following:

  • OS image and host-binary version;
  • successful installation and telemetry activation through the built AIDD CLI;
  • one completed real host session;
  • a journal session_start carrying the expected host;
  • a successful report containing that session;
  • sanitized evidence containing no secret, prompt, transcript, session identifier or personal
    path.

A discovered product mismatch becomes a dedicated defect. Its matrix cell remains unchecked
until the fix is validated with another real session.

Completion Evidence

  • One repository measurement document containing the ten-cell matrix and sanitized results.
  • Links to the manual or temporary workflow runs used for Windows and Linux.
  • The final document states the exact unsupported or unmeasurable case if an upstream host makes
    a matrix cell impossible.

Activity

  1. 8 remaining items

  2. added theissue type on Sep 3, 2026
  3. changed the title [-]test(framework): nothing here has ever run on Windows or Linux[/-] [+]test(framework): a real session per tool, on Windows and on Linux[/+] on Sep 3, 2026
  4. moved this from Ideation to Todo in AIDD Roadmapon Sep 10, 2026
  5. blafourcade commented on Oct 15, 2026

    @blafourcade
    ContributorAuthor

    Status after #993: CI now runs the real claude against a local fake API (no key, no quota) on Linux, macOS and Windows (scripts/check-claude-tokens-end-to-end.cjs). Reported tokens equal billed tokens on all three. The attended flow (task asked, declaration answered by the hook, task kept across /clear) runs through a pseudo-terminal on Linux and macOS. Still open: the attended flow on Windows; under ConPTY on the runner, Claude never got past its terminal setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Fields

    Priority

    High

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions