Skip to content

feat(telemetry): a prompt is an axis, and the only one complete by construction - #757

Merged
blafourcade merged 2 commits into
nextfrom
feat/a-prompt-is-an-axis
Sep 4, 2026
Merged

blafourcade merged 2 commits into
nextfrom
feat/a-prompt-is-an-axis

Conversation

@blafourcade

Copy link
Copy Markdown
Contributor

What

Adds by_prompt to the cost report: one row per prompt that caused work, dated by the earliest moment measured inside it, plus one row for records that named none.

Why this one

Every other breakdown can read low for a reason about the capture rather than about the work — a skill the host never named, a run journal that was never written, an identity nobody declared. This one groups on prompt_id, which the transcript reader already resolves for every usage line it stores by walking each file's own parentUuid chain back to the turn that caused the work. Nothing new is captured for it.

Measured, on the built binary

Sandboxed copy of one real session (HOME and AIDD_TELEMETRY_DIR both under a scratch directory):

by_prompt  rows  12  requests 1073   named 12, dated 12, remainder rows 0
by_agent   rows   4  requests 1073
by_step    rows   7  requests 1073   (96% of tokens unattributed)
totals.requests 1073                 cost_report_version 13

1073 of 1073 records carry a prompt_id, its 972 subagent records included, across 12 distinct prompts. The axis reconciles to the same total as every other breakdown with nothing in a remainder — on the same session where the step axis cannot name 96% of the tokens.

Decisions

Decision Choice Why
Absent key its own row, keyed on a Symbol a prompt id is opaque, so no string is safe to reserve. Same reasoning as NO_AGENT and NO_KNOWN_MODEL
started_at earliest moment in the prompt an id alone is unreadable; the moment is what a person greps for in their own transcript
The remainder row last, never ranked a bucket drawn from many turns has no size comparable to a single turn's. by_flow places its own remainder the same way
Printed rows top 10, then a count and --json the first axis whose cardinality is unbounded. by_day suppresses every row above its cap because a partial series is a lie about continuity; a partial ranking is not, and it says how many it withheld. The envelope carries them all

Envelope 12 → 13

A new top-level breakdown, so a consumer summing every breakdown's requests against totals.requests has one more to include. Every consumer updated in this commit: the contract document, the cost skill's own pinned version and axis lists, the envelope fixture, and the two e2e version pins.

Two gaps closed on the way

  • The skill's --axis enumeration and its question table both omitted agent, which had shipped two versions earlier — a person asking which subagent spent the tokens was told the question had no axis while the binary had been answering it. A guard now reads ARTEFACT_AXES from the module itself and asserts the two lists are equal, in both directions.
  • The e2e that runs every command the skill names held its own copy of the axis list in a placeholder regex, so a skill could not name a new axis without that regex being edited in the same breath. It now expands any <a|b|c> by its shape.

Cost

Bundle 588.4 → 590.6 KB. Budget raised 590 → 593, with the measurement recorded beside the previous raises.

Gates

  • cd cli && pnpm test — 305 files, 3396 tests, green (build included)
  • node --test "scripts/__tests__/**/*.test.js" — 366 pass
  • pnpm knip:production, pnpm jscpd, npx biome ci, npx tsc --noEmit — green
  • every pre-commit check run directly (json, skill frontmatter, argument hints, markdown links, cli layering); the three generators rewrite nothing
  • every guard ships with a mutation that fails it, and only it

🤖 Generated with Claude Code

https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp

…nstruction

Every breakdown the report carries can read low for a reason about the capture
rather than about the work: a skill the host never named, a run journal that was
never written, an identity nobody declared. `by_prompt` groups on `prompt_id`,
which the transcript reader already resolves for every usage line it stores by
walking each file's own `parentUuid` chain back to the turn that caused the work.

Measured on the built binary against a sandboxed copy of one real session:
1073 of 1073 records carry a `prompt_id`, its 972 subagent records included,
across 12 distinct prompts. The axis reconciles to the same total as every other
breakdown with nothing left in a remainder, on the same session where the step
axis cannot name 96% of the tokens.

A row carries `started_at`, the earliest moment measured inside that prompt,
because an id alone is opaque and the moment is what a person greps for in their
own transcript. The row for records that named no prompt carries neither field,
and is placed last rather than ranked among the prompts: a bucket drawn from many
turns has no size comparable to a single turn's. The text rendering prints the ten
largest and says how many it withheld, since this is the first axis whose
cardinality is unbounded; the envelope always carries them all.

Envelope 12 -> 13, every consumer updated in this commit.

Two gaps closed on the way. The skill's `--axis` enumeration and its
question table both omitted `agent`, which had shipped two versions earlier, so a
person asking which subagent spent the tokens was told the question had no axis
while the binary had been answering it. A guard now reads `ARTEFACT_AXES` from the
module itself and asserts the two lists are equal in both directions. And the e2e
that runs every command the skill names held its own copy of the axis list in a
placeholder regex, so a skill could not name a new axis without that regex being
edited in the same breath; it now expands any `<a|b|c>` by its shape.

Bundle 588.4 -> 590.6 KB, budget raised 590 -> 593 with the measurement recorded
beside it.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
@blafourcade
blafourcade requested a review from a team as a code owner September 4, 2026 15:59
The axis guard added with `by_prompt` matched the question table with a `\n\n`
terminator, which finds nothing in a checkout whose endings are CRLF - so a suite
green everywhere else failed on Windows alone, naming the first axis it looked
for. Measured: the same regex returns true on the POSIX file and false on the
identical file with CRLF.

The reader normalizes now, once, for every string assertion in this file rather
than only for the one that caught it. Its guard writes a CRLF file to disk and
reads it back through that reader, so a normalization removed from the reader
fails - a string normalized inside the assertion would have passed however the
reader behaved.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VWNxk63AGKkqE8HRqHLjGp
@blafourcade
blafourcade merged commit 18ed39b into next Sep 4, 2026
25 of 26 checks passed
@aidd-bot aidd-bot Bot mentioned this pull request Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant