Skip to content

refactor(nix): restore a single flake and simplify CI shells - #1254

Merged
ryoppippi merged 8 commits into
mainfrom
refactor/unify-dev-flake
Jun 10, 2026
Merged

refactor(nix): restore a single flake and simplify CI shells#1254
ryoppippi merged 8 commits into
mainfrom
refactor/unify-dev-flake

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Jun 10, 2026

Copy link
Copy Markdown
Member

Summary

Reverts the dev/root flake split from #1243 and simplifies the CI shell setup. Now that CI runs on Blacksmith with sticky-disk Nix store caching (#1249), the evaluation cost the split was avoiding is no longer noticeable, so the simpler single-flake setup wins.

What Changed

  • Merged dev/flake.nix back into the root flake.nix: treefmt-nix, git-hooks, and agent-skills inputs and their modules live in the root flake again. Input pins are carried over verbatim from dev/flake.lock, so no revisions change.
  • Removed devShells.ci and the just install / just flake-check recipes. Workflows enter the default dev shell with plain nix develop --command, whose shellHook installs pnpm dependencies, syncs agent skills, and installs git hooks.
  • The shellHook now runs pnpm install --frozen-lockfile unconditionally (an up-to-date install takes ~1s) instead of the lockfile-mtime check.
  • Dropped the unused warm-dev-shell input from the setup-nix action and restored the direct nix flake check --print-build-logs step in the lint job.
  • .envrc uses use flake again; just fmt is back to nix fmt.
  • Dropped the root node_modules sticky disk: a warm disk saved ~20s of pnpm install while mounting cost 5-45s per job (and occasionally failed), every job committed a snapshot even when nothing ran pnpm, and the per-GB-billed disk grew to tens of gigabytes despite node_modules holding ~500 MB of files. The Nix store sticky disk stays.

Testing

  • nix flake lock (no-op), nix flake show (single default dev shell, treefmt formatter restored)
  • nix flake check — all 7 checks pass
  • nix develop --command true — shellHook installs deps, syncs skills, installs hooks
  • just fmt — 0 changes; actionlint/zizmor pass on the edited workflows

Summary by CodeRabbit

  • Chores

    • Consolidated development into the top-level flake; removed the separate dev flake and added integrations for agent-skills, git-hooks, and tree formatting.
    • Simplified the dev shell to a single default shell and made dependency installation unconditional.
  • CI/CD

    • Updated workflows and actions to invoke the simplified Nix commands (removed per-dev flake target) and removed the optional warm-dev-shell behavior.
    • Adjusted macOS cache key inputs and repinned the Nix sticky-disk action.
  • Env

    • Dev-shell reload/file-watch now tracks root flake inputs for reloads.

Fold dev/flake.nix back into flake.nix so the repository ships a single
flake again: treefmt-nix, git-hooks, and agent-skills inputs return to
the root flake, and the development modules (treefmt, git-hooks,
dev-shell, agent-skills) are imported alongside the package modules.

The dev/ flake split (#1243) existed to keep package-only CI builds from
evaluating development inputs, trading a simpler setup for speed. With
the Blacksmith sticky-disk Nix store cache (#1249) the extra evaluation
cost no longer matters, so the simpler single-flake layout wins.

The pins for the restored inputs (agent-skills, git-hooks, treefmt-nix
and their transitive deps) are carried over verbatim from dev/flake.lock,
so no input revisions change. The dev shell is reached with plain
'nix develop' / 'use flake' again, and all './dev#' flake references
move back to the root flake.
Remove devShells.ci and the just install / flake-check recipes that
existed to keep CI shells free of side effects. Every workflow now
enters the default dev shell with plain 'nix develop --command', whose
shellHook installs pnpm dependencies, syncs agent skills, and installs
git hooks.

The shellHook now runs 'pnpm install --frozen-lockfile' unconditionally
instead of comparing lockfile mtimes: an up-to-date install takes about
a second, and the conditional occasionally kept stale node_modules
around after branch switches.

Also drop the warm-dev-shell input from the setup-nix action (nothing
passed it any more) and restore the direct 'nix flake check
--print-build-logs' step in the lint job. The separate ci shell and
explicit install steps were a #1243-era optimisation to avoid paying
the shellHook cost in CI; with sticky-disk caching the cost is noise
and the extra indirection just made the setup harder to follow.
@coderabbitai

coderabbitai Bot commented Jun 10, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 574af84a-5583-4ecd-951b-ba8dc7cd3184

📥 Commits

Reviewing files that changed from the base of the PR and between 877804c and 9d6f1a2.

📒 Files selected for processing (1)
  • .github/actions/setup-linux-blacksmith-sticky-disk/action.yaml

📝 Walkthrough

Walkthrough

Consolidate the dev flake into the root flake, inline and simplify the dev shell, add flake inputs/modules, update justfile tasks and Nix commands, and change CI/actions to use root flake invocations and updated cache/hash inputs.

Changes

Nix Flake Consolidation

Layer / File(s) Summary
Root flake inputs and module wiring
flake.nix
Three new flake inputs (agent-skills, git-hooks, treefmt-nix) are added and configured to follow nixpkgs. The flake-parts imports are updated to include the corresponding flake modules and new local Nix modules while removing the prior touchup/formatter handling.
Dev shell consolidation
nix/dev-shell.nix
The separate devShells.ci is removed and devShells.default is refactored to inline devShellPackages directly into buildInputs. The shellHook is simplified to unconditionally run pnpm install --frozen-lockfile while preserving agent skills sync and pre-commit hooks.
Build task simplification
justfile
Recipes are consolidated: typecheck removes its install dependency, fmt now runs nix fmt, check runs typecheck then nix flake check (removing the separate flake-check aggregation), schema generation updates to nix run .#generate-schema, and models.dev pricing generation adds formatting of the regenerated JSON.
GitHub Actions cleanup
.github/actions/setup-nix/action.yaml, .github/actions/setup-macos-nix-cache/action.yaml, .github/actions/setup-linux-blacksmith-sticky-disk/action.yaml
The setup-nix action removes the warm-dev-shell input and its conditional dev environment warming step. The macOS cache action removes dev/flake.lock and dev/flake.nix from its primary key hash computation. Linux sticky-disk action updates internal steps and pins useblacksmith/stickydisk to v1.3.0 commits.
CI workflow migration
.github/workflows/ci.yaml
All Nix invocations across lint-check, test, npm-publish, and performance comment jobs are updated from nix develop ./dev#ci --command ... to nix develop --command ..., using the default dev shell.
Release and pricing workflows
.github/workflows/release.yaml, .github/workflows/update-pricing.yaml
Both workflows remove redundant just install steps and update PNPM, changelogithub, and pricing snapshot validation/regeneration steps to use nix develop --command ... directly.
Dev environment and validation updates
.envrc, nix/checks.nix, nix/treefmt.nix
The .envrc switches from loading use flake ./dev to the default flake and updates watched inputs to the root flake.nix/flake.lock. Error hints and documentation comments now reference nix run .#generate-schema.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • ccusage/ccusage#1249: Also modifies the GitHub composite Nix setup and cache actions to update how Nix flake references and hash inputs are configured.

Poem

🐰 I hopped the dev flake to the root so fine,
Tidied shells, tasks, and workflows in a line,
No ./dev#ci to chase,
Simpler builds take place,
Carrots crunch softly — clarity's mine!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'refactor(nix): restore a single flake and simplify CI shells' directly and clearly summarizes the main changes: merging dev/flake.nix back into root flake and simplifying CI workflow shells.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch refactor/unify-dev-flake

Comment @coderabbitai help to get the list of available commands and usage tips.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 10, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide 9d6f1a2 Commit Preview URL

Branch Preview URL
Jun 10 2026, 08:58 PM

The sticky disk and cache-nix-action steps only log that a mount or
restore happened, so job logs never showed whether the run started warm
or cold, or what the restored data actually was.

Add report steps after each restore that print the sticky disk key, the
mount path, disk usage, and the number of restored Nix store paths or
top-level node_modules entries, with an explicit cold-start message when
the disk or cache comes back empty.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found and verified against the latest diff

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread nix/dev-shell.nix Outdated
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
@pkg-pr-new

pkg-pr-new Bot commented Jun 10, 2026

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@1254

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@1254

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@1254

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@1254

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@1254

commit: 877804c

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 32f808bcd01a
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 1.020s 881.4ms 56.7ms 3
PR pkg.pr.new 32f808b 999.9ms 863.8ms 53.8ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 32f808b. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 955.7ms 971.7ms 0.98x 732.50 MiB 727.25 MiB 0.99x 1.05 GiB/s 1.04 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 196.5ms 195.9ms 1.00x 89.50 MiB 92.75 MiB 1.04x 5.12 GiB/s 5.14 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 984.1ms 1.02 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 894.7ms 1.13 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 176.7ms 5.70 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 136.6ms 7.37 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 44.3ms 7.8ms 5.69x 43.75 MiB 2.50 MiB 0.06x 0.03 MiB/s 0.20 MiB/s
claude session --offline --json 0.00 MiB 47.3ms 7.6ms 6.19x 45.75 MiB 2.50 MiB 0.05x 0.03 MiB/s 0.20 MiB/s
codex daily --offline --json 0.00 MiB 48.1ms 7.2ms 6.64x 43.50 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.12 MiB/s
codex session --offline --json 0.00 MiB 45.2ms 7.5ms 6.07x 43.50 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.12 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 850.3ms 888.9ms 0.96x 733.50 MiB 735.25 MiB 1.00x 1.18 GiB/s 1.13 GiB/s
codex --offline --json 1.01 GiB 178.6ms 145.0ms 1.23x 90.50 MiB 91.25 MiB 1.01x 5.64 GiB/s 6.94 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 32f808bcd01a
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 732.3ms 953.4ms 58.0ms 3
PR pkg.pr.new 32f808b 1.147s 944.1ms 58.3ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 32f808b. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 1.059s 961.3ms 1.10x 741.50 MiB 738.00 MiB 1.00x 973.29 MiB/s 1.05 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 204.7ms 202.7ms 1.01x 92.50 MiB 92.50 MiB 1.00x 4.92 GiB/s 4.97 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 953.5ms 1.06 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 919.9ms 1.09 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 183.7ms 5.48 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 138.4ms 7.27 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 44.6ms 47.0ms 0.95x 45.75 MiB 43.75 MiB 0.96x 0.03 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 44.0ms 47.9ms 0.92x 45.50 MiB 43.50 MiB 0.96x 0.04 MiB/s 0.03 MiB/s
codex daily --offline --json 0.00 MiB 48.4ms 46.6ms 1.04x 46.00 MiB 43.75 MiB 0.95x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 47.1ms 48.3ms 0.97x 43.50 MiB 45.75 MiB 1.05x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 1.022s 963.2ms 1.06x 737.25 MiB 726.50 MiB 0.99x 1008.31 MiB/s 1.05 GiB/s
codex --offline --json 1.01 GiB 186.9ms 183.8ms 1.02x 89.25 MiB - - 5.39 GiB/s 5.48 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: af47cba9743d
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 685.5ms 820.7ms 59.6ms 3
PR pkg.pr.new af47cba 981.2ms 869.0ms 53.4ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: af47cba. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 901.6ms 1.005s 0.90x 734.25 MiB 705.50 MiB 0.96x 1.12 GiB/s 1.00 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 204.6ms 212.7ms 0.96x 87.75 MiB 91.50 MiB 1.04x 4.92 GiB/s 4.73 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 949.7ms 1.06 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 972.2ms 1.04 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 199.1ms 5.06 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 145.1ms 6.94 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 44.3ms 7.4ms 6.00x 43.50 MiB 2.50 MiB 0.06x 0.03 MiB/s 0.21 MiB/s
claude session --offline --json 0.00 MiB 46.3ms 8.1ms 5.71x 46.00 MiB 2.50 MiB 0.05x 0.03 MiB/s 0.19 MiB/s
codex daily --offline --json 0.00 MiB 48.6ms 7.9ms 6.15x 43.75 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.11 MiB/s
codex session --offline --json 0.00 MiB 48.5ms 7.9ms 6.16x 43.50 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.11 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 772.0ms 851.0ms 0.91x 737.00 MiB 710.00 MiB 0.96x 1.30 GiB/s 1.18 GiB/s
codex --offline --json 1.01 GiB 196.3ms 143.4ms 1.37x 91.50 MiB 92.50 MiB 1.01x 5.13 GiB/s 7.02 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: af47cba9743d
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 923.4ms 894.3ms 55.2ms 3
PR pkg.pr.new af47cba 861.6ms 1.018s 60.3ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: af47cba. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 918.7ms 896.4ms 1.02x 732.25 MiB 734.00 MiB 1.00x 1.10 GiB/s 1.12 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 198.3ms 192.1ms 1.03x 91.25 MiB 92.50 MiB 1.01x 5.08 GiB/s 5.24 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 933.2ms 1.08 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 860.0ms 1.17 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 173.9ms 5.79 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 134.5ms 7.49 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 42.2ms 48.0ms 0.88x 43.50 MiB 43.75 MiB 1.01x 0.04 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 48.4ms 46.7ms 1.04x 43.75 MiB 43.50 MiB 0.99x 0.03 MiB/s 0.03 MiB/s
codex daily --offline --json 0.00 MiB 48.4ms 47.2ms 1.03x 43.50 MiB 45.75 MiB 1.05x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 50.8ms 49.7ms 1.02x 45.75 MiB 43.50 MiB 0.95x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 943.6ms 958.7ms 0.98x 726.50 MiB 736.50 MiB 1.01x 1.07 GiB/s 1.05 GiB/s
codex --offline --json 1.01 GiB 178.0ms 186.2ms 0.96x 90.75 MiB 92.50 MiB 1.02x 5.66 GiB/s 5.41 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 22ce94d06866
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 1.010s 1.065s 50.9ms 3
PR pkg.pr.new 22ce94d 1.075s 1.102s 53.3ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 22ce94d. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 915.8ms 921.4ms 0.99x 722.75 MiB 730.75 MiB 1.01x 1.10 GiB/s 1.09 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 198.6ms 200.5ms 0.99x 92.00 MiB 92.50 MiB 1.01x 5.07 GiB/s 5.02 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 934.8ms 1.08 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 934.5ms 1.08 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 177.5ms 5.67 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 133.7ms 7.53 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 40.9ms 41.9ms 0.98x 43.50 MiB 45.75 MiB 1.05x 0.04 MiB/s 0.04 MiB/s
claude session --offline --json 0.00 MiB 40.7ms 43.1ms 0.94x 43.50 MiB 45.75 MiB 1.05x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 42.4ms 41.8ms 1.02x 45.75 MiB 43.50 MiB 0.95x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 42.8ms 41.8ms 1.02x 45.75 MiB 46.00 MiB 1.01x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 882.1ms 863.1ms 1.02x 737.75 MiB 741.00 MiB 1.00x 1.14 GiB/s 1.17 GiB/s
codex --offline --json 1.01 GiB 175.9ms 172.1ms 1.02x - 86.00 MiB - 5.72 GiB/s 5.85 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 22ce94d06866
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 1.061s 767.3ms 46.8ms 3
PR pkg.pr.new 22ce94d 1.000s 1.152s 46.1ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 22ce94d. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 843.5ms 864.0ms 0.98x 727.00 MiB 736.00 MiB 1.01x 1.19 GiB/s 1.17 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 177.2ms 181.9ms 0.97x 89.50 MiB 89.00 MiB 0.99x 5.68 GiB/s 5.53 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 803.3ms 1.25 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 792.8ms 1.27 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 171.9ms 5.86 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 126.8ms 7.94 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 42.3ms 6.0ms 7.11x 45.75 MiB 2.50 MiB 0.05x 0.04 MiB/s 0.26 MiB/s
claude session --offline --json 0.00 MiB 43.0ms 6.1ms 7.10x 43.50 MiB 2.50 MiB 0.06x 0.04 MiB/s 0.26 MiB/s
codex daily --offline --json 0.00 MiB 42.5ms 5.8ms 7.39x 46.00 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.15 MiB/s
codex session --offline --json 0.00 MiB 42.5ms 5.7ms 7.47x 45.75 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.15 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 843.2ms 752.5ms 1.12x 740.75 MiB 730.25 MiB 0.99x 1.19 GiB/s 1.34 GiB/s
codex --offline --json 1.01 GiB 159.6ms 126.3ms 1.26x 90.75 MiB 90.50 MiB 1.00x 6.31 GiB/s 7.97 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: f742dab3c492
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 728.3ms 799.7ms 48.1ms 3
PR pkg.pr.new f742dab 903.9ms 899.5ms 47.4ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: f742dab. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 928.2ms 903.8ms 1.03x 733.50 MiB 725.00 MiB 0.99x 1.08 GiB/s 1.11 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 175.9ms 176.0ms 1.00x 89.50 MiB 88.75 MiB 0.99x 5.72 GiB/s 5.72 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 848.9ms 1.19 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 800.8ms 1.26 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 158.9ms 6.33 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 121.3ms 8.30 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 43.0ms 42.4ms 1.01x 45.75 MiB 46.00 MiB 1.01x 0.04 MiB/s 0.04 MiB/s
claude session --offline --json 0.00 MiB 43.6ms 42.7ms 1.02x 45.75 MiB 45.75 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 42.0ms 42.2ms 1.00x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 41.9ms 42.3ms 0.99x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 891.8ms 780.1ms 1.14x 717.50 MiB 739.00 MiB 1.03x 1.13 GiB/s 1.29 GiB/s
codex --offline --json 1.01 GiB 161.7ms 160.2ms 1.01x 91.75 MiB 92.00 MiB 1.00x 6.23 GiB/s 6.28 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: f742dab3c492
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 697.5ms 552.5ms 54.4ms 3
PR pkg.pr.new f742dab 882.1ms 881.0ms 51.6ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: f742dab. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 942.8ms 876.3ms 1.08x 732.25 MiB 736.50 MiB 1.01x 1.07 GiB/s 1.15 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 200.0ms 197.0ms 1.02x 91.50 MiB 98.50 MiB 1.08x 5.03 GiB/s 5.11 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 932.6ms 1.08 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 949.0ms 1.06 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 189.4ms 5.31 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 145.8ms 6.91 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 47.7ms 7.8ms 6.10x 45.75 MiB 2.50 MiB 0.05x 0.03 MiB/s 0.20 MiB/s
claude session --offline --json 0.00 MiB 46.6ms 7.5ms 6.17x 45.75 MiB 2.50 MiB 0.05x 0.03 MiB/s 0.20 MiB/s
codex daily --offline --json 0.00 MiB 46.9ms 7.5ms 6.25x 46.00 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.11 MiB/s
codex session --offline --json 0.00 MiB 47.1ms 7.5ms 6.28x 43.75 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.11 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 890.7ms 864.3ms 1.03x 733.25 MiB 730.50 MiB 1.00x 1.13 GiB/s 1.16 GiB/s
codex --offline --json 1.01 GiB 181.5ms 143.8ms 1.26x 92.00 MiB 90.25 MiB 0.98x 5.55 GiB/s 7.00 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

Read the packageManager pin with jq instead of a node -p invocation and
a case statement. The resolved version only feeds the node_modules
sticky disk cache key; the pnpm that actually runs self-resolves to the
pinned version via manage-package-manager-versions regardless of which
binary starts it.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 1 file (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic

Comment thread .github/actions/setup-linux-blacksmith-sticky-disk/action.yaml Outdated
Drop the root node_modules sticky disk and the pnpm version resolution
that only existed to build its cache key. A warm disk saved roughly
twenty seconds of pnpm install, while mounting cost five to forty-five
seconds per job (and occasionally failed outright), every job committed
a snapshot even when nothing ran pnpm, and the per-GB-billed disk grew
to tens of gigabytes despite node_modules holding about 500 MB of
files. Installing from the registry on each run is cheaper and simpler.

The Nix store sticky disk stays; that one caches multi-gigabyte build
closures that are genuinely expensive to recreate.
@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 7ebed1a04cf1
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 1.626s 735.5ms 60.8ms 3
PR pkg.pr.new 7ebed1a 906.9ms 1.083s 55.1ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 7ebed1a. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 915.9ms 1.008s 0.91x 738.50 MiB 738.50 MiB 1.00x 1.10 GiB/s 1023.01 MiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 201.2ms 208.9ms 0.96x 86.50 MiB 88.50 MiB 1.02x 5.00 GiB/s 4.82 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 904.2ms 1.11 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 959.3ms 1.05 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 180.6ms 5.57 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 138.1ms 7.29 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 49.3ms 8.0ms 6.15x 43.75 MiB 2.50 MiB 0.06x 0.03 MiB/s 0.19 MiB/s
claude session --offline --json 0.00 MiB 50.3ms 8.2ms 6.10x 45.75 MiB 2.75 MiB 0.06x 0.03 MiB/s 0.19 MiB/s
codex daily --offline --json 0.00 MiB 46.8ms 7.1ms 6.54x 45.50 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.12 MiB/s
codex session --offline --json 0.00 MiB 49.6ms 7.9ms 6.28x 43.75 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.11 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 891.7ms 848.1ms 1.05x 728.75 MiB 732.00 MiB 1.00x 1.13 GiB/s 1.19 GiB/s
codex --offline --json 1.01 GiB 182.8ms 157.1ms 1.16x 94.00 MiB 90.75 MiB 0.97x 5.51 GiB/s 6.41 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 7ebed1a04cf1
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 618.7ms 827.8ms 54.7ms 3
PR pkg.pr.new 7ebed1a 766.8ms 931.6ms 63.6ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 7ebed1a. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 1.047s 960.3ms 1.09x 740.50 MiB 739.25 MiB 1.00x 984.59 MiB/s 1.05 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 223.1ms 202.8ms 1.10x 87.00 MiB 88.50 MiB 1.02x 4.51 GiB/s 4.97 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 983.1ms 1.02 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 889.5ms 1.13 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 180.4ms 5.58 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 134.7ms 7.47 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 45.1ms 47.9ms 0.94x 45.75 MiB 45.75 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 47.1ms 46.9ms 1.00x 43.75 MiB 45.75 MiB 1.05x 0.03 MiB/s 0.03 MiB/s
codex daily --offline --json 0.00 MiB 43.4ms 48.1ms 0.90x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 46.7ms 49.0ms 0.95x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 908.9ms 872.9ms 1.04x 745.00 MiB 736.75 MiB 0.99x 1.11 GiB/s 1.15 GiB/s
codex --offline --json 1.01 GiB 186.3ms 183.0ms 1.02x 91.50 MiB 90.00 MiB 0.98x 5.40 GiB/s 5.50 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 877804c28081
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 786.5ms 1.153s 44.4ms 3
PR pkg.pr.new 877804c 1.169s 735.9ms 44.8ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 877804c. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 856.0ms 816.0ms 1.05x 733.50 MiB 730.75 MiB 1.00x 1.18 GiB/s 1.23 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 168.4ms 167.8ms 1.00x 88.50 MiB 92.00 MiB 1.04x 5.98 GiB/s 6.00 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 844.4ms 1.19 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 795.6ms 1.27 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 163.1ms 6.17 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 124.4ms 8.09 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 41.5ms 5.6ms 7.42x 45.75 MiB 2.50 MiB 0.05x 0.04 MiB/s 0.28 MiB/s
claude session --offline --json 0.00 MiB 42.1ms 5.7ms 7.37x 46.00 MiB 2.50 MiB 0.05x 0.04 MiB/s 0.27 MiB/s
codex daily --offline --json 0.00 MiB 42.4ms 5.4ms 7.87x 45.75 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.16 MiB/s
codex session --offline --json 0.00 MiB 42.2ms 5.4ms 7.81x 45.75 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.16 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 907.4ms 798.2ms 1.14x 735.00 MiB 746.25 MiB 1.02x 1.11 GiB/s 1.26 GiB/s
codex --offline --json 1.01 GiB 158.4ms 123.1ms 1.29x 90.50 MiB 90.00 MiB 0.99x 6.36 GiB/s 8.18 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 877804c28081
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 670.5ms 791.1ms 50.2ms 3
PR pkg.pr.new 877804c 1.228s 841.8ms 49.4ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 877804c. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 920.2ms 869.4ms 1.06x 736.50 MiB 728.25 MiB 0.99x 1.09 GiB/s 1.16 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 211.8ms 194.6ms 1.09x 89.50 MiB 93.50 MiB 1.04x 4.75 GiB/s 5.17 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 878.9ms 1.15 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 877.6ms 1.15 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 199.3ms 5.05 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 145.0ms 6.94 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 42.8ms 45.0ms 0.95x 44.00 MiB 45.75 MiB 1.04x 0.04 MiB/s 0.03 MiB/s
claude session --offline --json 0.00 MiB 41.2ms 43.1ms 0.96x 45.75 MiB 45.75 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 41.8ms 43.1ms 0.97x 46.00 MiB 45.75 MiB 0.99x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 41.3ms 42.4ms 0.97x 45.75 MiB 46.00 MiB 1.01x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 966.8ms 891.6ms 1.08x 723.00 MiB 733.50 MiB 1.01x 1.04 GiB/s 1.13 GiB/s
codex --offline --json 1.01 GiB 186.5ms 181.2ms 1.03x 91.50 MiB 88.50 MiB 0.97x 5.40 GiB/s 5.56 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@ryoppippi
ryoppippi merged commit 43a05b1 into main Jun 10, 2026
24 checks passed
@ryoppippi
ryoppippi deleted the refactor/unify-dev-flake branch June 10, 2026 21:04
@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 9d6f1a27f10c
Base SHA: b47a31a131b2

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 791.7ms 881.3ms 54.5ms 3
PR pkg.pr.new 9d6f1a2 1.348s 1.113s 53.0ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 9d6f1a2. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 902.7ms 987.9ms 0.91x 724.50 MiB 726.75 MiB 1.00x 1.12 GiB/s 1.02 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 202.1ms 205.4ms 0.98x 91.25 MiB 92.00 MiB 1.01x 4.98 GiB/s 4.90 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 985.5ms 1.02 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 945.1ms 1.07 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 182.6ms 5.51 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 134.9ms 7.46 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 46.3ms 7.9ms 5.89x 46.00 MiB 2.75 MiB 0.06x 0.03 MiB/s 0.20 MiB/s
claude session --offline --json 0.00 MiB 46.4ms 8.5ms 5.46x 45.50 MiB 2.50 MiB 0.05x 0.03 MiB/s 0.18 MiB/s
codex daily --offline --json 0.00 MiB 46.4ms 7.5ms 6.22x 45.75 MiB 2.50 MiB 0.05x 0.02 MiB/s 0.12 MiB/s
codex session --offline --json 0.00 MiB 49.0ms 7.7ms 6.35x 43.75 MiB 2.50 MiB 0.06x 0.02 MiB/s 0.11 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs rust/target/release/ccusage directly. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 895.7ms 824.3ms 1.09x 731.25 MiB 742.00 MiB 1.01x 1.12 GiB/s 1.22 GiB/s
codex --offline --json 1.01 GiB 184.3ms 153.2ms 1.20x 90.50 MiB 90.75 MiB 1.00x 5.46 GiB/s 6.57 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 9d6f1a27f10c
Base SHA: b47a31a131b2

This compares the PR package against the configured base package on the same CI runner.

Package runner startup

Execution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one bunx -p <url> ccusage --version run with an empty Bun install cache. Warm reuses that cache and reports the median of repeated runs.

Package SHA Execution setup Bunx temp cache Bunx warm median Warm samples
Base pkg.pr.new b47a31a131b2 813.3ms 821.4ms 46.7ms 3
PR pkg.pr.new 9d6f1a2 968.3ms 1.179s 45.3ms 3

Cached bunx execution performance

Runs the same large fixture through bunx -p <pkg.pr.new URL> ccusage after the Bun install cache has already been populated by the startup measurement. This separates cached package-runner execution from first-fetch package materialization.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base package: b47a31a131b2; PR package: 9d6f1a2. Both run through bunx -p <pkg.pr.new URL> ccusage using the warmed Bun install cache from package runner startup, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
bunx -p <pkg> ccusage claude --offline --json 1.01 GiB 810.6ms 825.0ms 0.98x 734.50 MiB 747.50 MiB 1.02x 1.24 GiB/s 1.22 GiB/s
bunx -p <pkg> ccusage codex --offline --json 1.01 GiB 163.9ms 168.3ms 0.97x 91.25 MiB 90.00 MiB 0.99x 6.14 GiB/s 5.98 GiB/s

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 778.7ms 1.29 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 807.1ms 1.25 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 158.8ms 6.34 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 121.1ms 8.31 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 44.2ms 43.2ms 1.02x 45.75 MiB 45.75 MiB 1.00x 0.03 MiB/s 0.04 MiB/s
claude session --offline --json 0.00 MiB 42.3ms 42.1ms 1.00x 46.00 MiB 45.75 MiB 0.99x 0.04 MiB/s 0.04 MiB/s
codex daily --offline --json 0.00 MiB 41.6ms 41.1ms 1.01x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s
codex session --offline --json 0.00 MiB 41.5ms 41.4ms 1.00x 45.75 MiB 45.75 MiB 1.00x 0.02 MiB/s 0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 864.5ms 785.2ms 1.10x 732.00 MiB 740.75 MiB 1.01x 1.16 GiB/s 1.28 GiB/s
codex --offline --json 1.01 GiB 164.8ms 160.0ms 1.03x 90.25 MiB 88.00 MiB 0.98x 6.11 GiB/s 6.29 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 17.30 KiB 17.30 KiB +0.00 KiB 1.00x
installed native package binary 3417.74 KiB 3417.74 KiB +0.00 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant