Skip to content

fix(pricing): avoid slow fallback for display model aliases - #1315

Merged
ryoppippi merged 2 commits into
mainfrom
codex/model-alias-pricing-lookup
Jun 14, 2026
Merged

fix(pricing): avoid slow fallback for display model aliases#1315
ryoppippi merged 2 commits into
mainfrom
codex/model-alias-pricing-lookup

Conversation

@ryoppippi

@ryoppippi ryoppippi commented Jun 14, 2026

Copy link
Copy Markdown
Member

Fixes model alias pricing lookup so display-oriented aliases do not force slow online pricing fallback when the original model is already known.

Previously, CCUSAGE_MODEL_ALIASES replaced the model before pricing/context lookup, so aliases such as claude-opus-4-8=mythos-5 tried to price mythos-5 and could block on models.dev. The lookup now prefers the original model first, then falls back to the alias target for private raw slugs that need canonical pricing.

Testing:

  • cargo fmt --manifest-path rust/Cargo.toml --all --check
  • cargo test --manifest-path rust/Cargo.toml -p ccusage pricing_lookup
  • cargo test --manifest-path rust/Cargo.toml -p ccusage model_aliases
  • pre-push hook: clippy, treefmt, gitleaks, cargo test

View with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is enabled.


Summary by cubic

Prefer the original model over its display alias when resolving pricing and context limits to avoid unnecessary online fallback and speed up lookups. Preserves canonical pricing for private raw slugs while preventing blocking calls for display-only aliases.

  • Bug Fixes

    • Changed lookup order: original model → alias target → models.dev → embedded pricing.
    • Added regression test to ensure a known model like claude-opus-4-8 resolves locally even if aliased to mythos-5.
  • Refactors

    • Renamed internal fallback variable to resolved_alias for clearer intent.

Written for commit ee69618. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • Bug Fixes
    • Improved pricing lookup precedence for models with alternative names so the original model identifier is prioritized before any alias mapping, yielding more accurate pricing and context limit results.

Resolve pricing and context limits from the original model before falling back to CCUSAGE_MODEL_ALIASES. This keeps display-oriented aliases such as claude-opus-4-8=mythos-5 from forcing an unnecessary models.dev lookup while preserving private-slug aliases that map unknown raw models to known pricing entries.

Add a regression test covering known original models with unknown display aliases.
@pullfrog

pullfrog Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

no API key found — this repo is configured to use deepseek/deepseek-v4-pro, which needs DEEPSEEK_API_KEY, but the runner has no key for it.

To fix: add the key as a GitHub Actions secret (referenced from your workflow's env: block) or as a Pullfrog secret in the console — or switch this repo to a different model (free models need no key).

Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →

Pullfrog  | Rerun failed job ➔View workflow run | via Pullfrog | Using DeepSeek Pro (free via Pullfrog for OSS) | 𝕏

@coderabbitai

coderabbitai Bot commented Jun 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7713a000-d984-4ecf-8098-7a317f73570e

📥 Commits

Reviewing files that changed from the base of the PR and between 7862c88 and ee69618.

📒 Files selected for processing (1)
  • rust/crates/ccusage/src/pricing.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • rust/crates/ccusage/src/pricing.rs

📝 Walkthrough

Walkthrough

PricingMap::find and PricingMap::context_limit in pricing.rs are refactored to separate the original model key from its resolved alias (resolved_alias). Both functions now attempt the initial lookup using the original model and only consult models_dev_pricing/embedded_models_dev_pricing via the alias when they differ. A unit test verifies this precedence.

Changes

Alias Lookup Precedence Fix

Layer / File(s) Summary
Original model key precedence in find and context_limit
rust/crates/ccusage/src/pricing.rs
Both find and context_limit now resolve the model alias up-front into resolved_alias, perform an initial lookup using the original model key, and only attempt alias lookups in the primary map when resolved_alias differs from the original model. The fallback paths to models_dev_pricing and embedded_models_dev_pricing use resolved_alias. A new unit test pricing_lookup_prefers_known_original_model_before_alias asserts that the original model entry is returned rather than the alias-resolved one.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • ccusage/ccusage#1313: Directly modifies the same alias resolution logic in PricingMap::find/context_limit, introducing the structure this PR then reorders for precedence.
  • ccusage/ccusage#1301: Changes PricingMap::find and PricingMap::context_limit alias-based fallback behavior, establishing foundational patterns refined by this PR.
  • ccusage/ccusage#1303: Adjusts alias handling for specific model entries within the same pricing lookup path affected by this PR.

Poem

A bunny hopped through model names one day,
Found the original should have first say!
Before the alias steals the show,
Check the source key — that's the way to go.
🐇 Original first, then fallback with grace,
No alias jumps ahead in this race! ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(pricing): avoid slow fallback for display model aliases' accurately and concisely summarizes the main objective of the pull request—preventing unnecessary slow online fallbacks for display-oriented model aliases.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/model-alias-pricing-lookup

Comment @coderabbitai help to get the list of available commands and usage tips.

@ryoppippi

Copy link
Copy Markdown
Member Author

@coderabbitai review
@cubic-dev-ai review

@cubic-dev-ai

cubic-dev-ai Bot commented Jun 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai review
@cubic-dev-ai review

@ryoppippi I have started the AI code review. It will take a few minutes to complete.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 14, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
ccusage-guide ee69618 Commit Preview URL

Branch Preview URL
Jun 14 2026, 06:24 PM

@coderabbitai

coderabbitai Bot commented Jun 14, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Get PR details
gh pr view 1315 --repo ccusage/ccusage --json title,body,files,additions,deletions,commits

# Get PR diff
gh pr diff 1315 --repo ccusage/ccusage

Length of output: 7864



@ryoppippi: The fix is clean and well-reasoned. Here's my review:

Review of rust/crates/ccusage/src/pricing.rs

✅ Core logic is correct

The new lookup order for find and context_limit:

  1. Original model in self → fast path for known Anthropic models
  2. Alias target in self → handles private-slug aliases
  3. Alias target in models.dev (online)
  4. Alias target in embedded models.dev

This correctly fixes the reported issue where claude-opus-4-8=mythos-5 caused an unnecessary models.dev fetch.

⚠️ Asymmetry in the fallback chain

Steps 3 and 4 (models.dev / embedded fallback) only try fallback_model (the alias target), not the original model. This means:

If the original model is only in models.dev (not in self or embedded) and its alias target is unknown there, the original model would be missed.

For example, a hypothetical non-Anthropic model aliased to something unknown wouldn't be found in models.dev by its original name.

That said, for the primary use case (Anthropic models like claude-opus-4-8 are already in embedded pricing), step 2 always succeeds before any network call, so this is an acceptable trade-off—adding the original-model lookup to steps 3/4 would double the already-expensive models.dev network call in the already-slow path.

✅ Test coverage

The regression test is well-targeted: it installs the claude-opus-4-8=mythos-5 alias, loads embedded pricing, and verifies that both find and context_limit return the embedded entry for the original model rather than falling through to an online lookup for the unknown alias.

Nit: variable naming

let alias = crate::model_aliases::resolve_model_name(model);
let fallback_model = alias.as_ref();

The name fallback_model is slightly backwards in intent—the alias target is the fallback, but the name implies it's the primary. It reads clearly enough in context though. Something like alias_model or resolved_alias would be marginally clearer, but this isn't blocking.


Summary: The fix is correct for the described use case. The asymmetry in the models.dev fallback is a reasonable performance trade-off. LGTM with the minor naming nit.

Address review feedback: the variable holds the alias's resolved target,
which is used as the fallback lookup key. Renaming makes the intent
clearer at the call sites.

Co-authored-by: Codesmith <[email protected]>
@pkg-pr-new

pkg-pr-new Bot commented Jun 14, 2026

Copy link
Copy Markdown

Open in StackBlitz

ccusage

npx https://pkg.pr.new/ccusage@1315

@ccusage/ccusage-darwin-arm64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-arm64@1315

@ccusage/ccusage-darwin-x64

npx https://pkg.pr.new/@ccusage/ccusage-darwin-x64@1315

@ccusage/ccusage-linux-arm64

npx https://pkg.pr.new/@ccusage/ccusage-linux-arm64@1315

@ccusage/ccusage-linux-x64

npx https://pkg.pr.new/@ccusage/ccusage-linux-x64@1315

@ccusage/ccusage-win32-x64

npx https://pkg.pr.new/@ccusage/ccusage-win32-x64@1315

commit: ee69618

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 7862c8817aec
Base SHA: 3b4ff2c0fa90

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 246.9ms 4.08 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 204.3ms 4.93 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 114.3ms 8.81 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 94.3ms 10.68 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 29.7ms 5.1ms 5.87x 53.50 MiB 10.20 MiB 0.19x 0.05 MiB/s 0.31 MiB/s
claude session --offline --json 0.00 MiB 29.6ms 4.8ms 6.20x 54.00 MiB 10.19 MiB 0.19x 0.05 MiB/s 0.32 MiB/s
codex daily --offline --json 0.00 MiB 29.5ms 3.7ms 8.07x 53.75 MiB 8.18 MiB 0.15x 0.03 MiB/s 0.23 MiB/s
codex session --offline --json 0.00 MiB 28.5ms 2.5ms 11.48x 53.75 MiB 8.18 MiB 0.15x 0.03 MiB/s 0.35 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 261.8ms 233.8ms 1.12x 924.33 MiB 918.32 MiB 0.99x 3.85 GiB/s 4.31 GiB/s
codex --offline --json 1.01 GiB 110.2ms 87.3ms 1.26x 389.04 MiB 403.03 MiB 1.04x 9.13 GiB/s 11.53 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB +0.00 KiB 1.00x
installed native package binary 3850.88 KiB 3851.06 KiB +0.19 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: 7862c8817aec
Base SHA: 3b4ff2c0fa90

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 254.7ms 3.95 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 218.9ms 4.60 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 115.0ms 8.75 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 87.9ms 11.45 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 29.1ms 31.1ms 0.94x 53.75 MiB 53.50 MiB 1.00x 0.05 MiB/s 0.05 MiB/s
claude session --offline --json 0.00 MiB 29.9ms 29.3ms 1.02x 53.75 MiB 54.00 MiB 1.00x 0.05 MiB/s 0.05 MiB/s
codex daily --offline --json 0.00 MiB 30.4ms 30.5ms 1.00x 53.50 MiB 53.75 MiB 1.00x 0.03 MiB/s 0.03 MiB/s
codex session --offline --json 0.00 MiB 26.6ms 29.5ms 0.90x 53.75 MiB 53.50 MiB 1.00x 0.03 MiB/s 0.03 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 295.3ms 277.7ms 1.06x 934.33 MiB 934.32 MiB 1.00x 3.41 GiB/s 3.63 GiB/s
codex --offline --json 1.01 GiB 113.9ms 112.9ms 1.01x 425.04 MiB 415.04 MiB 0.98x 8.84 GiB/s 8.91 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB +0.00 KiB 1.00x
installed native package binary 3850.88 KiB 3851.06 KiB +0.19 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
rust/crates/ccusage/src/pricing.rs (1)

1821-1835: 💤 Low value

Test validates main use case; consider edge case coverage.

The test correctly verifies that a known original model (claude-opus-4-8 in primary table) with a display alias (mythos-5) returns the original pricing, preventing unnecessary alias lookups.

For more comprehensive coverage, consider adding tests for:

  1. Private slug alias: unknown original → known alias (e.g., private-model -> gpt-5.5)
  2. Embedded-only model with alias: model only in embedded pricing with display alias

These would validate that the fallback logic works correctly across all intended use cases.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/crates/ccusage/src/pricing.rs` around lines 1821 - 1835, The test
`pricing_lookup_prefers_known_original_model_before_alias` covers the case where
a known original model has a display alias, but lacks coverage for important
edge cases. Add two new test functions: one that validates the private slug
alias use case where an unknown original model is aliased to a known model
(e.g., private-model maps to gpt-5.5), and another that validates the
embedded-only model scenario where a model exists only in the embedded pricing
table and also has a display alias. These new tests should follow the same
pattern as the existing test, setting up the appropriate model aliases and
verifying that the pricing lookup and context limit functions return correct
values for these different scenarios.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@rust/crates/ccusage/src/pricing.rs`:
- Around line 1821-1835: The test
`pricing_lookup_prefers_known_original_model_before_alias` covers the case where
a known original model has a display alias, but lacks coverage for important
edge cases. Add two new test functions: one that validates the private slug
alias use case where an unknown original model is aliased to a known model
(e.g., private-model maps to gpt-5.5), and another that validates the
embedded-only model scenario where a model exists only in the embedded pricing
table and also has a display alias. These new tests should follow the same
pattern as the existing test, setting up the appropriate model aliases and
verifying that the pricing lookup and context limit functions return correct
values for these different scenarios.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 104cba6a-eba4-44d1-9267-e762591c1e1a

📥 Commits

Reviewing files that changed from the base of the PR and between 3b4ff2c and 7862c88.

📒 Files selected for processing (1)
  • rust/crates/ccusage/src/pricing.rs

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: ee69618be62e
Base SHA: 3b4ff2c0fa90

This compares the Rust PR release binary against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 242.3ms 4.15 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 232.1ms 4.34 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 101.4ms 9.93 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 80.3ms 12.53 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 27.3ms 4.1ms 6.73x 53.50 MiB 10.20 MiB 0.19x 0.06 MiB/s 0.38 MiB/s
claude session --offline --json 0.00 MiB 26.7ms 2.3ms 11.37x 53.75 MiB 10.19 MiB 0.19x 0.06 MiB/s 0.66 MiB/s
codex daily --offline --json 0.00 MiB 23.4ms 2.1ms 11.24x 53.75 MiB 8.18 MiB 0.15x 0.04 MiB/s 0.41 MiB/s
codex session --offline --json 0.00 MiB 22.8ms 2.1ms 10.85x 53.75 MiB 8.18 MiB 0.15x 0.04 MiB/s 0.41 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published native ccusage binary from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 269.2ms 232.4ms 1.16x 938.33 MiB 944.32 MiB 1.01x 3.74 GiB/s 4.33 GiB/s
codex --offline --json 1.01 GiB 100.8ms 83.2ms 1.21x 427.05 MiB 447.03 MiB 1.05x 9.98 GiB/s 12.10 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB +0.00 KiB 1.00x
installed native package binary 3850.88 KiB 3851.06 KiB +0.19 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@github-actions

Copy link
Copy Markdown
Contributor

ccusage performance comparison

PR SHA: ee69618be62e
Base SHA: 3b4ff2c0fa90

This compares the PR package against the configured base package on the same CI runner.

Package runtime diagnostics

Compares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
All rows run --offline --json, measured by hyperfine with 0 warmups and 1 runs. This isolates wrapper overhead from the installed native optional dependency and the workspace release binary built on the runner.

Command Runtime Input Median Throughput Samples
claude --offline --json Package wrapper 1.01 GiB 246.3ms 4.09 GiB/s 1
claude --offline --json Installed native binary 1.01 GiB 218.3ms 4.61 GiB/s 1
codex --offline --json Package wrapper 1.01 GiB 101.3ms 9.94 GiB/s 1
codex --offline --json Installed native binary 1.01 GiB 93.0ms 10.83 GiB/s 1

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 2 warmups and 7 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude daily --offline --json 0.00 MiB 25.4ms 24.9ms 1.02x 53.50 MiB 53.50 MiB 1.00x 0.06 MiB/s 0.06 MiB/s
claude session --offline --json 0.00 MiB 23.5ms 23.9ms 0.99x 53.75 MiB 53.75 MiB 1.00x 0.07 MiB/s 0.06 MiB/s
codex daily --offline --json 0.00 MiB 23.8ms 23.2ms 1.03x 53.75 MiB 53.75 MiB 1.00x 0.04 MiB/s 0.04 MiB/s
codex session --offline --json 0.00 MiB 22.2ms 22.5ms 0.99x 53.50 MiB 53.75 MiB 1.00x 0.04 MiB/s 0.04 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/_work/_temp/ccusage-large-fixture (1.01 GiB, 2597 files), Codex /home/runner/_work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2597 files)
Base runs the published ccusage package from pkg.pr.new, installed before measurement; PR runs the published ccusage package from pkg.pr.new, installed before measurement. Both run --offline --json, measured by hyperfine with 0 warmups and 1 runs.
Peak RSS is measured separately with /usr/bin/time using 1 runs. Lower RSS ratios are better.

Command Input Base median PR median PR vs base Base peak RSS PR peak RSS PR/base RSS Base throughput PR throughput
claude --offline --json 1.01 GiB 269.2ms 253.7ms 1.06x 926.32 MiB 922.33 MiB 1.00x 3.74 GiB/s 3.97 GiB/s
codex --offline --json 1.01 GiB 100.7ms 110.4ms 0.91x 433.04 MiB 415.04 MiB 0.96x 10.00 GiB/s 9.12 GiB/s

Artifact size

Artifact Base PR Delta Ratio
packed ccusage-*.tgz 18.08 KiB 18.08 KiB +0.00 KiB 1.00x
installed native package binary 3850.88 KiB 3851.06 KiB +0.19 KiB 1.00x

Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

@ryoppippi
ryoppippi merged commit 92c518d into main Jun 14, 2026
33 checks passed
@ryoppippi
ryoppippi deleted the codex/model-alias-pricing-lookup branch June 14, 2026 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant