fix(pricing): pick closest-length fuzzy match in LiteLLM pricing lookup by ryoppippi · Pull Request #1018 · ccusage/ccusage

ryoppippi · 2026-05-17T11:00:40Z

Summary

Fixes #934.

When a model has no direct entry in the LiteLLM pricing map, getModelPricing() falls back to a substring-based fuzzy match. The old loop returned the first key whose name was a substring of (or contained) the input, in Map insertion order. For gpt-5.4-mini this matched gpt-5 instead of gpt-5.4, producing a ~5x cost overcharge in the Codex adapter.

This change collects all substring matches and picks the candidate whose name length is closest to the input length. So gpt-5.4-mini now resolves to gpt-5.4 (length diff 5) over gpt-5 (length diff 7). The heuristic also handles the reverse direction: when the input is shorter than candidates, the closest-length key still wins.

This is the algorithmic fix suggested as option 2 in the issue. It does not require adding new prefetched entries and does not introduce a strict-mode flag.

Changes

packages/internal/src/pricing.ts — replace first-match fuzzy loop in LiteLLMPricingFetcher.getModelPricing() with a closest-length-difference scan.
Two new in-source Vitest cases covering both directions of the fuzzy match.

Test plan

pnpm --filter @ccusage/internal test — 58 passing
pnpm run test — 497 passing, 3 skipped
pnpm typecheck — clean
pnpm run format — clean

Generated by Claude Code

Summary by cubic

Fixes pricing lookup when a model has no direct entry by choosing the closest-length fuzzy match. This prevents cases like gpt-5.4-mini mapping to gpt-5 and overcharging.

Bug Fixes
- Update LiteLLMPricingFetcher.getModelPricing() to scan all substring matches and select the key with the smallest length difference to the input.
- Add two Vitest tests to cover longer-input and longer-key scenarios.

^{Written for commit b22019d. Summary will update on new commits. Review in cubic}

Summary by CodeRabbit

Bug Fixes
- Improved model pricing lookup accuracy by implementing fuzzy matching for model names, which now selects the best match by similarity rather than the first partial match.
Tests
- Added test cases validating the new fuzzy matching behavior in pricing lookups.

When no direct pricing entry exists for a model name, the fuzzy fallback in getModelPricing() previously returned the first substring match in iteration order. For "gpt-5.4-mini" this matched "gpt-5" instead of "gpt-5.4", producing a ~5x cost overcharge. Collect all substring matches and pick the candidate whose name length is closest to the input length, so "gpt-5.4-mini" prefers "gpt-5.4" over the more generic "gpt-5" entry. Fixes #934

cloudflare-workers-and-pages · 2026-05-17T11:00:50Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	ccusage-guide	`b22019d`	Commit Preview URL Branch Preview URL	May 17 2026, 10:07 AM

coderabbitai · 2026-05-17T11:00:51Z

Caution

Review failed

Pull request was closed or merged during review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 58e88bc5-d2d4-471a-9798-ee012ea692e1

📥 Commits

Reviewing files that changed from the base of the PR and between c6337cf and b22019d.

📒 Files selected for processing (1)

packages/internal/src/pricing.ts

📝 Walkthrough

Walkthrough

The PR improves pricing lookup accuracy by changing LiteLLMPricingFetcher.getModelPricing()'s fallback matching strategy from selecting the first partial key match to selecting the pricing key closest in length to the requested model name. Two test cases validate the new fuzzy matching behavior.

Changes

Pricing fuzzy matching improvement

Layer / File(s)	Summary
Fuzzy matching by closest key length `packages/internal/src/pricing.ts`	Replaced first-partial-match selection with fuzzy match that selects the pricing key closest in length to the requested model name; caches `null` if no match is found. Test cases validate that closest-length keys are preferred over earlier-iterated partial matches.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

ryoppippi/ccusage#689: Both PRs modify LiteLLMPricingFetcher.getModelPricing() matching logic—fix(ccusage): broaden pricing lookup to cover provider variants #689 introduces heuristic best-match scoring across provider variants, and this PR adjusts fuzzy selection to choose the closest-length key.
ryoppippi/ccusage#657: Both PRs modify the internal LiteLLM pricing lookup in the same file—the retrieved PR expands provider/model prefix matching, while this PR changes the fuzzy selection logic for best pricing entry resolution.

Poem

🐰 A fuzzy fix with matching care,
Lengths now choose with fairer flair,
No more gpt-5 gets the blame,
When mini models stake their claim! 🎯

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: updating fuzzy matching logic in LiteLLM pricing lookup to select closest-length matches rather than first matches.
Linked Issues check	✅ Passed	The PR implements the preferred approach from issue `#934`: fuzzy matching now selects the candidate with closest name length rather than the first substring match, fixing the gpt-5.4-mini mispricing bug.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to fixing the fuzzy matching logic in LiteLLMPricingFetcher.getModelPricing() and adding test coverage, with no unrelated modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch claude/fix-issue-934-5dqVg

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

packages/internal/src/pricing.ts

[baseline-browser-mapping] The data in this module is over two months old. To ensure accurate Baseline data, please update: npm i baseline-browser-mapping@latest -D
tsconfig.json is not found. we cannot use type-aware rules.

Oops! Something went wrong! :(

ESLint: 9.35.0

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'eslint-plugin-format' imported from /node_modules/.pnpm/@antfu+eslint-config@4.19.0_@vue [email protected][email protected][email protected]_vit_670a2c5c75d4275eabd7bc195a173ee6/node_modules/@antfu/eslint-config/dist/index.js
at Object.getPackageJSONURL (node:internal/modules/package_json_reader:301:9)
at packageResolve (node:internal/modules/esm/resolve:764:81)
at moduleResolve (node:internal/modules/esm/resolve:855:18)
at defaultResolve (node:internal/modules/esm/resolve:988:11)
at #cachedDefaultResolve (node:internal/modules/esm/loader:697:20)
at #resolveAndMaybeBlockOnLoaderThread (node:internal/modules/esm/loader:714:38)
at ModuleLoader.resolveSync (node:internal/modules/esm/loader:746:52)
at #resolve (node:internal/modules/esm/loader:679:17)
at ModuleLoader.getOrCreateModuleJob (node:internal/modules/esm/loader:599:35)
at node:internal/modules/esm/loader:628:32

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-05-17T11:02:38Z

ccusage performance comparison

This compares the PR build against the base branch build on the same CI runner.

Committed fixture performance

Committed small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage.

Fixtures: Claude apps/ccusage/test/fixtures/claude (0.00 MiB, 2 files), Codex apps/ccusage/test/fixtures/codex (0.00 MiB, 1 files)
Runtime: package ccusage bin from apps/ccusage/package.json through bun -b, --offline --json, measured by hyperfine with 2 warmups and 7 runs.

Command	Base median	PR median	PR vs base	Base throughput	PR throughput
`claude daily --offline --json`	55.7ms	55.3ms	1.01x	0.03 MiB/s	0.03 MiB/s
`claude session --offline --json`	55.3ms	55.4ms	1.00x	0.03 MiB/s	0.03 MiB/s
`codex daily --offline --json`	55.6ms	55.0ms	1.01x	0.02 MiB/s	0.02 MiB/s
`codex session --offline --json`	55.0ms	55.0ms	1.00x	0.02 MiB/s	0.02 MiB/s

Large real-world-shaped fixture performance

Generated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures.

Fixtures: Claude /home/runner/work/_temp/ccusage-large-fixture (1.01 GiB, 2,597 files), Codex /home/runner/work/_temp/ccusage-large-codex-fixture (1.01 GiB, 2,597 files)
Runtime: package ccusage bin from apps/ccusage/package.json through bun -b, --offline --json, measured by hyperfine with 0 warmups and 1 runs.

Command	Input	Base median	PR median	PR vs base	Base throughput	PR throughput
`claude --offline --json`	1.01 GiB	1.420s	1.403s	1.01x	725.95 MiB/s	734.85 MiB/s
`codex --offline --json`	1.01 GiB	1.701s	1.758s	0.97x	606.15 MiB/s	586.43 MiB/s

Package size

Package artifact	Base	PR	Delta	Ratio
packed `ccusage-*.tgz`	67.53 KiB	67.58 KiB	+0.05 KiB	1.00x

Lower medians and smaller packed package sizes are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees.

ryoppippi merged commit 68d3ea9 into main May 17, 2026
21 of 22 checks passed

ryoppippi deleted the claude/fix-issue-934-5dqVg branch May 17, 2026 11:01

This was referenced May 25, 2026

fix(pricing): resolve Claude dot-separated model aliases #1154

Merged

fix(pricing): add Claude Opus 4.8 rates #1182

Merged

Navigation Menu

Uh oh!

Uh oh!

fix(pricing): pick closest-length fuzzy match in LiteLLM pricing lookup - #1018

fix(pricing): pick closest-length fuzzy match in LiteLLM pricing lookup#1018
ryoppippi merged 1 commit into
mainfrom
claude/fix-issue-934-5dqVg

ryoppippi commented May 17, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

cloudflare-workers-and-pages Bot commented May 17, 2026

Uh oh!

coderabbitai Bot commented May 17, 2026 •

edited

Loading

Review failed

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Poem

Uh oh!

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

ryoppippi commented May 17, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Summary by cubic

Summary by CodeRabbit

Uh oh!

cloudflare-workers-and-pages Bot commented May 17, 2026

Deploying with Cloudflare Workers

Uh oh!

coderabbitai Bot commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review failed

Walkthrough

Changes

Estimated code review effort

Possibly related PRs

Poem

Uh oh!

Uh oh!

github-actions Bot commented May 17, 2026

ccusage performance comparison

Committed fixture performance

Large real-world-shaped fixture performance

Package size

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ryoppippi commented May 17, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented May 17, 2026 •

edited

Loading