Skip to content

feat(framework): read what a session cost from the files the tool already wrote #685

Closed

Description

@blafourcade

Outcome

The cost of a session is readable from the files the tool already wrote, with no process running and nothing exported.

Why this exists

Decided in #684. The receiver stays as an opt-in path for an exact billed amount; this becomes the default one. What it removes is the failure mode where a project is switched on, exports correctly, and stores nothing because nobody was listening.

What is already measured

Captured from files on disk 2026-08-20, no session run:

Tool Where Shape Granularity
Claude Code ~/.claude/projects/*/*.jsonl, reachable from every hook payload as transcript_path message.usage: input_tokens, cache_creation_input_tokens, cache_read_input_tokens, output_tokens, plus model, sessionId, requestId, isSidechain per assistant message
Codex ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl, reachable as transcript_path {"type":"event_msg","payload":{"type":"token_count","info":{"total_token_usage":{…},"last_token_usage":{…}}}} cumulative and last, per event
OpenCode ~/.local/share/opencode/opencode.db, fixed path, SQLite message.data.tokens = {total, input, output, reasoning, cache:{read, write}}, plus modelID and providerID per request

Not covered, and both for reasons no amount of work here fixes:

  • Copilot writes outputTokens per turn and nothing else; input, cache and reasoning appear once, in the session.shutdown aggregate. No per-request input figure exists, so no per-step breakdown can be built from it. Its cost field is denominated in premium requests, not dollars — pinned at 0.33 while totalNanoAiu ranges across three billion.
  • Cursor writes no token count anywhere. Its transcript has thirteen distinct JSON paths and not one is token-related. Its OTLP export is an Enterprise team setting nobody here can enable. It is uncovered by both routes.

Scope

  • Includes: one reader per covered tool, behind one contract, following the shape refactor(framework): one file per tool in the journal hook, not five tables #683 defines for the hook rather than five tables keyed by tool.
  • Includes: joining what is read to the run journal by session identifier, the same key the export path already joins on.
  • Includes: fixtures captured from real sessions, and a test that fails when a format moves. These are internal, undocumented formats free to change under us — that is the cost of this route and the mitigation has to be built in, not promised.
  • Excludes: the price table. feat(framework): a price table for the tools that export no amount #654 owns turning tokens into an amount.
  • Excludes: the receiver, which stays as it is.
  • Excludes: Copilot and Cursor, for the reasons above. Naming them as uncovered is in scope; covering them is not.

Done When

  • A session on a covered tool yields its token counts and model with no process running and nothing exported.
  • Which tools are covered, and which are not, is visible to a user rather than inferred from an empty result.
  • A format change in any tool fails a test rather than silently producing a wrong figure.
  • Reading is never on a session's critical path and never blocks or slows one.
  • Nothing read is stored beyond what the sink's allowlist already permits — the local path chooses what to read, so there is nothing to redact after the fact.

Relations

Field Value
parent #631
depends_on #684
related #629, #654, #647, #683, #676

Activity

  1. moved this from Ideation to Todo in AIDD Roadmapon Aug 20, 2026
  2. moved this from Todo to Done in AIDD Roadmapon Aug 20, 2026
  3. blafourcade commented on Aug 20, 2026

    @blafourcade
    ContributorAuthor

    Delivered and committed, not merged. 8303a6c9 plus its plan in e8db90c5, on blafourcade/brainstorm-telemetry.

    Gate: 2388 CLI tests, tsc, biome and the layering check all exit 0. package.json untouched — no dependency, no engine floor.

    What shipped beyond the ticket body. Four facts measured against real files, each of which a plausible implementation would have got wrong without a test noticing:

    • One Claude Code API call can log several transcript lines. A thinking block and its tool_use share one requestId, one message.id and one identical usage. Four requestIds out of five in the sampled transcript carry more than one line; one carries three. Mapping per line inflates every figure two- to threefold, and a synthetic one-line-per-call fixture passes.
    • Codex's input_tokens includes cached_input_tokens, where Claude Code's excludes it. Confirmed by arithmetic on four consecutive events: total_tokens = input + output, not input + cache + output. Without subtracting, the same column means two different things depending on the tool.
    • Codex's token_count event carries no model and no request id. They live on the turn_context that precedes the run, so records are per turn.
    • A rollout's session_meta carries both id and session_id, and the hook sees id. On a fresh session they agree; on a resumed one session_id holds the parent thread. 124 of the 330 rollouts on the measuring machine are resumed sessions, so a reader keyed on the wrong field breaks for 38% of them while passing every test written against a fresh one.

    One design point worth keeping. A reader cannot stamp its own provenance: LocalCostCandidateRecord omits the field, so it structurally cannot claim to be an export it is not. That is a type doing what a rule would otherwise have to.

    Coverage, stated rather than implied. Claude Code, Codex and OpenCode are read. Copilot writes one counter per turn and nothing else, so no per-request input figure exists for it. Cursor writes none at all, and its export is an enterprise setting nobody here can enable. Both declare that, with the reason, and a covered tool carries any caveat on its own figures as data — OpenCode's says its figures cannot yet be joined to a run journal entry, because no captured payload establishes that a hook sees its session id. That belongs to #676.

    Next on the sequence in #631: #654, then #629.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    High

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions