feat(internal): add support for 1M context window pricing - #651
Conversation
- Add new fields to LiteLLMModelPricingSchema for 1M context pricing: - input_cost_per_token_above_200k_tokens - output_cost_per_token_above_200k_tokens - cache_creation_input_token_cost_above_200k_tokens - cache_read_input_token_cost_above_200k_tokens - Update calculateCostFromPricing to handle tiered pricing: - Split tokens into two buckets: below and above 200k threshold - Apply normal pricing for tokens below 200k - Apply higher pricing for tokens above 200k when available - Gracefully fall back to normal pricing for models without 1M pricing - Add comprehensive tests for 1M context pricing: - Test input token pricing with 300k tokens - Test output token pricing with 250k tokens - Test cache token pricing (creation and read) - Test fallback behavior for models without 1M pricing This fixes issue #568 where ccusage was not tracking usage correctly for Claude models with 1M context windows. The implementation follows LiteLLM's pricing structure which uses separate rates for tokens above 200k.
|
Warning Rate limit exceeded@ryoppippi has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 7 minutes and 19 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (2)
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds optional tiered-pricing fields and two-bucket cost calculation for a 200,000-token threshold across input/output and cache token types; extends the pricing schema and exported types. Also documents tiered-pricing notes (including 128k Gemini fields) in CLAUDE.md. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Caller as Caller
participant Pricing as Pricing Engine
participant Schema as Pricing Schema
Caller->>Pricing: calculateCostFromPricing(tokens, pricing)
Pricing->>Schema: read pricing fields (base and *_above_200k_tokens, *_above_128k_tokens)
Note over Pricing: CONTEXT_THRESHOLD = 200000 (200k)\nGemini 128k fields present in schema but not used in calc
rect rgb(240,245,255)
note right of Pricing: For each token class: input, output, cache_create, cache_read
Pricing->>Pricing: determine tokensBelow = min(total, 200k)
Pricing->>Pricing: tokensAbove = max(0, total - 200k)
alt above-rate defined
Pricing->>Pricing: cost = tokensBelow*baseRate + tokensAbove*aboveRate
else base-rate defined
Pricing->>Pricing: cost = total*baseRate
else
Pricing->>Pricing: cost = 0
end
end
Pricing-->>Caller: totalCost
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello @ryoppippi, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces robust support for tiered pricing models, particularly for large language models with 1M context windows like Claude 4 Sonnet. It ensures that token usage costs are accurately calculated based on different rates for tokens below and above a 200k threshold, and also fixes a critical bug related to usage tracking for these models. The changes enhance the system's ability to handle advanced pricing structures and improve cost transparency. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
commit: |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 649c073 | Sep 18 2025, 09:53 PM |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds support for tiered pricing for models with large context windows, like Claude 3.5 Sonnet. The changes to the schema and pricing calculation logic are well-implemented and accompanied by a comprehensive set of new tests. My main feedback is to refactor the calculateCostFromPricing method to reduce code duplication, which will improve maintainability.
| let cost = 0; | ||
| const CONTEXT_THRESHOLD = 200_000; | ||
|
|
||
| if (pricing.input_cost_per_token != null) { | ||
| cost += tokens.input_tokens * pricing.input_cost_per_token; | ||
| // Calculate input tokens cost | ||
| if (tokens.input_tokens > 0) { | ||
| if (tokens.input_tokens > CONTEXT_THRESHOLD && pricing.input_cost_per_token_above_200k_tokens != null) { | ||
| // Split tokens into two buckets: below and above 200k | ||
| const tokensBelow200k = Math.min(tokens.input_tokens, CONTEXT_THRESHOLD); | ||
| const tokensAbove200k = Math.max(0, tokens.input_tokens - CONTEXT_THRESHOLD); | ||
|
|
||
| if (pricing.input_cost_per_token != null) { | ||
| cost += tokensBelow200k * pricing.input_cost_per_token; | ||
| } | ||
| cost += tokensAbove200k * pricing.input_cost_per_token_above_200k_tokens; | ||
| } | ||
| else if (pricing.input_cost_per_token != null) { | ||
| cost += tokens.input_tokens * pricing.input_cost_per_token; | ||
| } | ||
| } | ||
|
|
||
| if (pricing.output_cost_per_token != null) { | ||
| cost += tokens.output_tokens * pricing.output_cost_per_token; | ||
| // Calculate output tokens cost | ||
| if (tokens.output_tokens > 0) { | ||
| if (tokens.output_tokens > CONTEXT_THRESHOLD && pricing.output_cost_per_token_above_200k_tokens != null) { | ||
| // Split tokens into two buckets: below and above 200k | ||
| const tokensBelow200k = Math.min(tokens.output_tokens, CONTEXT_THRESHOLD); | ||
| const tokensAbove200k = Math.max(0, tokens.output_tokens - CONTEXT_THRESHOLD); | ||
|
|
||
| if (pricing.output_cost_per_token != null) { | ||
| cost += tokensBelow200k * pricing.output_cost_per_token; | ||
| } | ||
| cost += tokensAbove200k * pricing.output_cost_per_token_above_200k_tokens; | ||
| } | ||
| else if (pricing.output_cost_per_token != null) { | ||
| cost += tokens.output_tokens * pricing.output_cost_per_token; | ||
| } | ||
| } | ||
|
|
||
| if ( | ||
| tokens.cache_creation_input_tokens != null | ||
| && pricing.cache_creation_input_token_cost != null | ||
| ) { | ||
| cost | ||
| += tokens.cache_creation_input_tokens | ||
| * pricing.cache_creation_input_token_cost; | ||
| // Calculate cache creation cost | ||
| if (tokens.cache_creation_input_tokens != null && tokens.cache_creation_input_tokens > 0) { | ||
| if (tokens.cache_creation_input_tokens > CONTEXT_THRESHOLD && pricing.cache_creation_input_token_cost_above_200k_tokens != null) { | ||
| const tokensBelow200k = Math.min(tokens.cache_creation_input_tokens, CONTEXT_THRESHOLD); | ||
| const tokensAbove200k = Math.max(0, tokens.cache_creation_input_tokens - CONTEXT_THRESHOLD); | ||
|
|
||
| if (pricing.cache_creation_input_token_cost != null) { | ||
| cost += tokensBelow200k * pricing.cache_creation_input_token_cost; | ||
| } | ||
| cost += tokensAbove200k * pricing.cache_creation_input_token_cost_above_200k_tokens; | ||
| } | ||
| else if (pricing.cache_creation_input_token_cost != null) { | ||
| cost += tokens.cache_creation_input_tokens * pricing.cache_creation_input_token_cost; | ||
| } | ||
| } | ||
|
|
||
| if (tokens.cache_read_input_tokens != null && pricing.cache_read_input_token_cost != null) { | ||
| cost | ||
| += tokens.cache_read_input_tokens * pricing.cache_read_input_token_cost; | ||
| // Calculate cache read cost | ||
| if (tokens.cache_read_input_tokens != null && tokens.cache_read_input_tokens > 0) { | ||
| if (tokens.cache_read_input_tokens > CONTEXT_THRESHOLD && pricing.cache_read_input_token_cost_above_200k_tokens != null) { | ||
| const tokensBelow200k = Math.min(tokens.cache_read_input_tokens, CONTEXT_THRESHOLD); | ||
| const tokensAbove200k = Math.max(0, tokens.cache_read_input_tokens - CONTEXT_THRESHOLD); | ||
|
|
||
| if (pricing.cache_read_input_token_cost != null) { | ||
| cost += tokensBelow200k * pricing.cache_read_input_token_cost; | ||
| } | ||
| cost += tokensAbove200k * pricing.cache_read_input_token_cost_above_200k_tokens; | ||
| } | ||
| else if (pricing.cache_read_input_token_cost != null) { | ||
| cost += tokens.cache_read_input_tokens * pricing.cache_read_input_token_cost; | ||
| } | ||
| } | ||
|
|
||
| return cost; |
There was a problem hiding this comment.
The cost calculation logic for input, output, cache creation, and cache read tokens is very similar and repeated four times. This duplication makes the code harder to read and maintain. You can significantly improve this by extracting the common logic into a single helper function defined within this method. This will make the main function a series of calls to the helper, which is much cleaner.
const CONTEXT_THRESHOLD = 200_000;
const calculateTieredCost = (
totalTokens: number | undefined,
basePrice: number | undefined,
tieredPrice: number | undefined,
): number => {
if (totalTokens == null || totalTokens <= 0) {
return 0;
}
if (totalTokens > CONTEXT_THRESHOLD && tieredPrice != null) {
const tokensBelowThreshold = CONTEXT_THRESHOLD;
const tokensAboveThreshold = totalTokens - CONTEXT_THRESHOLD;
let tieredCost = tokensAboveThreshold * tieredPrice;
if (basePrice != null) {
tieredCost += tokensBelowThreshold * basePrice;
}
return tieredCost;
}
if (basePrice != null) {
return totalTokens * basePrice;
}
return 0;
};
const inputCost = calculateTieredCost(
tokens.input_tokens,
pricing.input_cost_per_token,
pricing.input_cost_per_token_above_200k_tokens,
);
const outputCost = calculateTieredCost(
tokens.output_tokens,
pricing.output_cost_per_token,
pricing.output_cost_per_token_above_200k_tokens,
);
const cacheCreationCost = calculateTieredCost(
tokens.cache_creation_input_tokens,
pricing.cache_creation_input_token_cost,
pricing.cache_creation_input_token_cost_above_200k_tokens,
);
const cacheReadCost = calculateTieredCost(
tokens.cache_read_input_tokens,
pricing.cache_read_input_token_cost,
pricing.cache_read_input_token_cost_above_200k_tokens,
);
return inputCost + outputCost + cacheCreationCost + cacheReadCost;Refactored the duplicated cost calculation logic into a single `calculateTieredCost` helper function to improve code readability and maintainability. This addresses the review feedback from @gemini-code-assist. The logic remains identical, just DRYed up: - Single helper handles all tiered pricing calculations - Each token type now uses the same consistent logic - Easier to maintain and understand
Added thorough edge case tests to ensure the calculateTieredCost helper function handles all scenarios correctly: - Exactly 200k tokens (boundary test) - 200,001 tokens (tiered pricing for 1 token) - Zero tokens - Undefined/null token values - Models with only tiered pricing (no base price) These tests ensure the refactored tiered pricing logic is robust and handles all edge cases properly.
Consolidated the individual tests for input/output/cache tokens into a single comprehensive test that covers all token types. This reduces test duplication while maintaining the same coverage. Changes: - Merged 3 separate tests for different token types into 1 comprehensive test - Simplified edge case tests by removing redundant undefined token test - Renamed edge case test to better reflect its focus on boundary conditions The test suite remains at 6 tests but is now more maintainable and focused.
Updated test names to be more descriptive about what they're actually testing: - Specify exact token counts being tested (300k, 250k, etc.) - Clarify boundary conditions (200k, 200,001, 0) - Show expected behavior in test name This makes it easier to understand what each test covers without reading the implementation.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (3)
packages/internal/src/pricing.ts (3)
220-220: Consider hoisting CONTEXT_THRESHOLD to a module‑level constKeeps the value single‑sourced and reusable by helpers/tests.
239-254: Avoid repetition: extract a small tiered-cost helperSame pattern repeats 4x; a local helper will reduce drift and ease future threshold changes.
403-429: Add tests for cache fallback behaviorPlease add a test where cache_creation/read rates are omitted and we fall back to input rates (including >200k). This guards the new fallback logic.
Suggested test to append:
+ it('falls back to input rates when cache pricing is missing (incl. >200k)', async () => { + using fetcher = new LiteLLMPricingFetcher({ + offline: true, + offlineLoader: async () => ({ + 'claude-4-sonnet-20250514': { + input_cost_per_token: 3e-6, + input_cost_per_token_above_200k_tokens: 6e-6 + // no cache_creation_input_token_cost + // no cache_read_input_token_cost + // no *_above_200k_tokens for cache + }, + }), + }); + + const cost = await Result.unwrap(fetcher.calculateCostFromTokens({ + input_tokens: 0, + output_tokens: 0, + cache_creation_input_tokens: 250_000, // 200k @ base, 50k @ above + cache_read_input_tokens: 300_000, // 200k @ base, 100k @ above + }, 'claude-4-sonnet-20250514')); + + const expected = + (200_000 * 3e-6) + (50_000 * 6e-6) + + (200_000 * 3e-6) + (100_000 * 6e-6); + expect(cost).toBeCloseTo(expected); + });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
packages/internal/src/pricing.ts(3 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.ts: Use tab indentation and double quotes (ESLint formatting)
Do not use console.log; only allow where explicitly disabled via eslint-disable
Always use Node.js path utilities for file paths for cross-platform compatibility
Use .ts extensions for local file imports (e.g., import { foo } from './utils.ts')
Prefer @praha/byethrow Result type over traditional try-catch for functional error handling
Use Result.try() to wrap operations that may throw (e.g., JSON parsing)
Use Result.isFailure() for checking errors instead of negating isSuccess()
Use early return on failures (e.g., if (Result.isFailure(r)) continue) instead of ternary patterns
For async operations, create a wrapper using Result.try() and call it
Keep traditional try-catch only for complex file I/O or legacy code that’s hard to refactor
Always use Result.isFailure() and Result.isSuccess() type guards for clarity
Variables use camelCase naming
Types use PascalCase naming
Constants can use UPPER_SNAKE_CASE
Only export constants, functions, and types that are actually used by other modules
Do not export internal/private constants that are only used within the same file
Before exporting a constant, verify it is referenced by other modules
Use Vitest globals (describe, it, expect) without imports in test blocks
Never use await import() dynamic imports anywhere in the codebase
Never use dynamic imports inside Vitest test blocks
Use fs-fixture createFixture() for mock Claude data directories in tests
All tests must use current Claude 4 models (not Claude 3)
Test coverage should include both Sonnet and Opus models
Model names in tests must exactly match LiteLLM pricing database entries
Use logger.ts instead of console.log for logging
Files:
packages/internal/src/pricing.ts
packages/internal/src/**
📄 CodeRabbit inference engine (packages/internal/CLAUDE.md)
Place new shared utility files under src/
Files:
packages/internal/src/pricing.ts
packages/internal/src/**/*.ts
📄 CodeRabbit inference engine (packages/internal/CLAUDE.md)
packages/internal/src/**/*.ts: Use .ts extensions in local import specifiers
Prefer @praha/byethrow Result type over try-catch for error handling
Only export symbols that are actually used by other modules
Use Vitest in-source tests guarded by if (import.meta.vitest != null) blocks
Files:
packages/internal/src/pricing.ts
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Cost calculation per model/date: charge non-cached input, cached input (fallback to input rate if missing), and output using the specified per-million token rates
📚 Learning: 2025-09-18T16:07:16.277Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Fetch per-model pricing from LiteLLM model_prices_and_context_window.json via LiteLLMPricingFetcher using an offline cache scoped to Codex-prefixed models; handle aliases (e.g., gpt-5-codex → gpt-5) in CodexPricingSource
Applied to files:
packages/internal/src/pricing.ts
📚 Learning: 2025-09-18T16:07:16.277Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Cost calculation per model/date: charge non-cached input, cached input (fallback to input rate if missing), and output using the specified per-million token rates
Applied to files:
packages/internal/src/pricing.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Workers Builds: ccusage-guide
🔇 Additional comments (3)
packages/internal/src/pricing.ts (3)
222-237: Tiered input pricing logic — LGTMCorrect two‑bucket split with fallback to base rate when above‑200k not provided.
380-402: 1M pricing test (output > 200k) — LGTMCovers the output bucket correctly.
431-449: Fallback for models without 1M pricing — LGTM; add Opus coverage (use anthropic/claude-opus-4-20250514)Add an Opus (Claude 4) test using the LiteLLM pricing DB key anthropic/claude-opus-4-20250514 (alias claude-opus-4-20250514). Opus has a 200K context window (not 1M) — test should use 200K pricing or verify fallback behavior.
Likely an incorrect or invalid review comment.
Added detailed JSDoc comments to document the 1M context window pricing functionality: - Documented calculateCostFromPricing method with explanation of tiered pricing support - Added JSDoc for calculateTieredCost helper function with example calculation - Clarified that the 200k threshold is used for tiered pricing This improves code maintainability and helps future developers understand the tiered pricing logic for 1M context window models.
Fixed test to use the canonical 'anthropic/claude-4-sonnet-20250514' model key that matches LiteLLM's pricing database format. The previous non-canonical key 'claude-4-sonnet-20250514' would not match at runtime. Also hoisted the 200k token threshold to a module-level constant with documentation explaining its relationship to LiteLLM's schema field names. This makes the threshold value more visible to maintainers and documents the dependency on upstream schema. Changes: - Updated model key in tiered pricing test to use 'anthropic/' prefix - Extracted TIERED_PRICING_TOKEN_THRESHOLD as module constant - Added comprehensive JSDoc explaining the threshold's purpose and upstream dependency - Replaced local CONTEXT_THRESHOLD with module constant in calculateTieredCost helper All tests pass successfully with these changes.
Added comprehensive documentation about tiered pricing support in LiteLLM to help future developers understand the different pricing models and their thresholds. This addresses the complexity of supporting multiple model providers with different pricing structures. Key documentation additions: - Detailed breakdown of models WITH tiered pricing (Claude 200k, Gemini 128k) - Models WITHOUT tiered pricing (GPT/OpenAI flat rate) - Current implementation status and limitations - Critical guidelines for adding support for new models - Warning about threshold differences requiring calculation logic updates This documentation stems from the work on 1M context pricing support where it became clear that different model providers use different token thresholds for tiered pricing, and this complexity needs to be well-documented to prevent future integration issues. The current implementation only supports 200k threshold (Claude models). Adding Gemini or other models with different thresholds would require refactoring the calculateTieredCost helper function.
Added detailed warning comments to the LiteLLM pricing schema explaining the complexity of tiered pricing across different model providers. This addresses the critical knowledge gap around threshold differences that could cause calculation errors. Warning documentation includes: - Model-specific threshold values (Claude 200k, Gemini 128k, GPT flat rate) - Current implementation status for each provider - Step-by-step guidance for adding new model support - Clear indication of what requires code changes vs schema-only updates These warnings stem from implementing 1M context pricing support where the complexity of supporting multiple providers with different thresholds became apparent. The current calculateTieredCost function only handles 200k thresholds, making these warnings essential for preventing future integration bugs. Additionally clarified existing comment to specify "Claude/Anthropic" for the 200k threshold fields to distinguish from other providers.
Added schema fields for Gemini models' tiered pricing at 128k token threshold to support future implementation. These fields are documented as NOT yet implemented in calculation logic to prevent confusion. Fields added: - input_cost_per_token_above_128k_tokens - output_cost_per_token_above_128k_tokens This prepares the schema for potential Gemini model support while clearly indicating that calculation logic updates would be required. The current calculateTieredCost function only handles 200k thresholds (Claude models). Adding these fields now ensures schema completeness for LiteLLM compatibility while the warning comments added in previous commit make it clear what implementation work would be needed to support Gemini pricing calculations.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (2)
packages/internal/src/pricing.ts (2)
443-474: Standardize model key to LiteLLM canonical form in this testUse "anthropic/claude-4-sonnet-20250514" for consistency with other tests and the live DB. Avoids surprises if this test later uses live/merged fixtures.
Apply:
- 'claude-4-sonnet-20250514': { + 'anthropic/claude-4-sonnet-20250514': { @@ -}, 'claude-4-sonnet-20250514')); +}, 'anthropic/claude-4-sonnet-20250514'));
309-319: Cache creation/read: missing fallback to input rates can under-chargeIf cache_* rates are absent, cached tokens currently cost $0. We should fall back to input rates (base and >200k) to align with pricing policy.
Apply this diff:
- const cacheCreationCost = calculateTieredCost( - tokens.cache_creation_input_tokens, - pricing.cache_creation_input_token_cost, - pricing.cache_creation_input_token_cost_above_200k_tokens, - ); + const cacheCreationBase = + pricing.cache_creation_input_token_cost + ?? pricing.input_cost_per_token; + const cacheCreationAbove = + pricing.cache_creation_input_token_cost_above_200k_tokens + ?? pricing.input_cost_per_token_above_200k_tokens + ?? cacheCreationBase; + const cacheCreationCost = calculateTieredCost( + tokens.cache_creation_input_tokens, + cacheCreationBase, + cacheCreationAbove, + ); - const cacheReadCost = calculateTieredCost( - tokens.cache_read_input_tokens, - pricing.cache_read_input_token_cost, - pricing.cache_read_input_token_cost_above_200k_tokens, - ); + const cacheReadBase = + pricing.cache_read_input_token_cost + ?? pricing.input_cost_per_token; + const cacheReadAbove = + pricing.cache_read_input_token_cost_above_200k_tokens + ?? pricing.input_cost_per_token_above_200k_tokens + ?? cacheReadBase; + const cacheReadCost = calculateTieredCost( + tokens.cache_read_input_tokens, + cacheReadBase, + cacheReadAbove, + );Please add unit tests for both cache branches where cache rates are missing to prevent regressions.
🧹 Nitpick comments (4)
packages/internal/CLAUDE.md (1)
62-96: Doc is clear; add explicit note on cache-rate fallbacks to avoid drift with codePlease add that cache_creation/cache_read rates should fall back to input rates when cache-specific rates are absent (both base and >threshold). This prevents ambiguity for models missing cache fields and aligns expectations with pricing.ts behavior after the fallback fix.
Apply this minimal addition:
### ⚠️ IMPORTANT for Future Development @@ 4. **Add comprehensive tests** for boundary conditions at the threshold 5. **Document the pricing structure** in relevant CLAUDE.md files +6. If cache-specific rates are missing, fall back to the corresponding input rates (base and above-threshold) to avoid under-charging cached tokens.packages/internal/src/pricing.ts (3)
7-15: Hard-coded 200k threshold — consider per-model threshold detection (future-proofing)Current constant locks all tiered pricing to 200k. Non-blocking now, but detecting threshold from available “above_*” fields per model will make this extensible (e.g., Gemini 128k) without touching call sites.
Here’s a low-impact direction:
-const TIERED_PRICING_TOKEN_THRESHOLD = 200_000; +const DEFAULT_TIERED_THRESHOLD = 200_000;And later, derive per-model when computing costs (see helper comment below).
257-295: Consider passing an explicit threshold to the helper (non-blocking)To support models with different thresholds, let calculateTieredCost accept a threshold param. You can derive it once per model (e.g., 200k if any above_200k field present; 128k if any above_128k field present).
Example shape:
-const calculateTieredCost = (totalTokens, basePrice, tieredPrice): number => { +const calculateTieredCost = (totalTokens, basePrice, tieredPrice, threshold = DEFAULT_TIERED_THRESHOLD): number => { @@ - if (totalTokens > TIERED_PRICING_TOKEN_THRESHOLD && tieredPrice != null) { - const tokensBelowThreshold = Math.min(totalTokens, TIERED_PRICING_TOKEN_THRESHOLD); - const tokensAboveThreshold = Math.max(0, totalTokens - TIERED_PRICING_TOKEN_THRESHOLD); + if (totalTokens > threshold && tieredPrice != null) { + const tokensBelowThreshold = Math.min(totalTokens, threshold); + const tokensAboveThreshold = Math.max(0, totalTokens - threshold);
39-47: Upstream has 200k keys — keep current fields; optionally add 128k supportVerification: the upstream JSON includes these _above_200k_tokens keys: cache_creation_input_token_cost_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens, input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens. It also contains several _above_128k_tokens keys (e.g., input_cost_per_token_above_128k_tokens, output_cost_per_token_above_128k_tokens and media/character variants).
Action: no immediate change required in packages/internal/src/pricing.ts (lines 39–47). Recommended (optional): add parsing aliases or threshold detection to also handle _above_128k_tokens keys for Gemini tiered pricing, or document that 128k-tier pricing is not calculated.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
packages/internal/CLAUDE.md(1 hunks)packages/internal/src/pricing.ts(5 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
**/*.ts
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.ts: Use tab indentation and double quotes (ESLint formatting)
Do not use console.log; only allow where explicitly disabled via eslint-disable
Always use Node.js path utilities for file paths for cross-platform compatibility
Use .ts extensions for local file imports (e.g., import { foo } from './utils.ts')
Prefer @praha/byethrow Result type over traditional try-catch for functional error handling
Use Result.try() to wrap operations that may throw (e.g., JSON parsing)
Use Result.isFailure() for checking errors instead of negating isSuccess()
Use early return on failures (e.g., if (Result.isFailure(r)) continue) instead of ternary patterns
For async operations, create a wrapper using Result.try() and call it
Keep traditional try-catch only for complex file I/O or legacy code that’s hard to refactor
Always use Result.isFailure() and Result.isSuccess() type guards for clarity
Variables use camelCase naming
Types use PascalCase naming
Constants can use UPPER_SNAKE_CASE
Only export constants, functions, and types that are actually used by other modules
Do not export internal/private constants that are only used within the same file
Before exporting a constant, verify it is referenced by other modules
Use Vitest globals (describe, it, expect) without imports in test blocks
Never use await import() dynamic imports anywhere in the codebase
Never use dynamic imports inside Vitest test blocks
Use fs-fixture createFixture() for mock Claude data directories in tests
All tests must use current Claude 4 models (not Claude 3)
Test coverage should include both Sonnet and Opus models
Model names in tests must exactly match LiteLLM pricing database entries
Use logger.ts instead of console.log for logging
Files:
packages/internal/src/pricing.ts
packages/internal/src/**
📄 CodeRabbit inference engine (packages/internal/CLAUDE.md)
Place new shared utility files under src/
Files:
packages/internal/src/pricing.ts
packages/internal/src/**/*.ts
📄 CodeRabbit inference engine (packages/internal/CLAUDE.md)
packages/internal/src/**/*.ts: Use .ts extensions in local import specifiers
Prefer @praha/byethrow Result type over try-catch for error handling
Only export symbols that are actually used by other modules
Use Vitest in-source tests guarded by if (import.meta.vitest != null) blocks
Files:
packages/internal/src/pricing.ts
🧠 Learnings (4)
📚 Learning: 2025-09-18T17:43:09.223Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: CLAUDE.md:0-0
Timestamp: 2025-09-18T17:43:09.223Z
Learning: Applies to **/*.ts : Model names in tests must exactly match LiteLLM pricing database entries
Applied to files:
packages/internal/CLAUDE.mdpackages/internal/src/pricing.ts
📚 Learning: 2025-09-18T16:07:16.277Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Cost calculation per model/date: charge non-cached input, cached input (fallback to input rate if missing), and output using the specified per-million token rates
Applied to files:
packages/internal/src/pricing.ts
📚 Learning: 2025-09-18T16:07:16.277Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Use 2025-08-07 gpt-5 cached token rates: input $1.25/M, cached input $0.125/M, output $10/M
Applied to files:
packages/internal/src/pricing.ts
📚 Learning: 2025-09-18T16:07:16.277Z
Learnt from: CR
PR: ryoppippi/ccusage#0
File: apps/codex/CLAUDE.md:0-0
Timestamp: 2025-09-18T16:07:16.277Z
Learning: Fetch per-model pricing from LiteLLM model_prices_and_context_window.json via LiteLLMPricingFetcher using an offline cache scoped to Codex-prefixed models; handle aliases (e.g., gpt-5-codex → gpt-5) in CodexPricingSource
Applied to files:
packages/internal/src/pricing.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Workers Builds: ccusage-guide
🔇 Additional comments (6)
packages/internal/src/pricing.ts (6)
16-31: Schema guidance is helpfulThe inline documentation is clear and sets expectations for future contributors. No changes needed.
237-247: JSDoc accurately scopes the new behaviorNicely documents tiered pricing and covered token types.
390-421: Tiered pricing test (all token types) — LGTMCovers input/output/cache creation/read above threshold with expected splits.
423-441: Fallback to standard pricing when 1M fields absent — LGTMCorrectly charges all tokens at base rates.
476-504: Edge case (no base price) — LGTMValidates charging only the above-threshold portion when base is missing.
351-506: Add minimal Opus coverage to satisfy internal test policyNo 'anthropic/claude-4-.opus.' keys were found in the LiteLLM dataset at the provided URL; confirm the canonical LiteLLM key for Claude 4 Opus and I will draft an Opus test mirroring the existing Sonnet tiered case.
…arameter - Change TIERED_PRICING_TOKEN_THRESHOLD to DEFAULT_TIERED_THRESHOLD for clarity - Add threshold parameter to calculateTieredCost with 200k default value - Remove complex Gemini 128k detection logic to keep implementation simple - Update documentation to note cache rate fallback behavior - Clean up tests removing Gemini-specific scenarios This maintains flexibility for future models with different thresholds while keeping the current implementation straightforward and maintainable.
1 participant
Summary
Changes
Schema Update: Added new optional fields to
liteLLMModelPricingSchemafor 1M context pricing:input_cost_per_token_above_200k_tokensoutput_cost_per_token_above_200k_tokenscache_creation_input_token_cost_above_200k_tokenscache_read_input_token_cost_above_200k_tokensCost Calculation Logic: Updated
calculateCostFromPricingmethod to:Tests: Added comprehensive test coverage for:
Test Results
All tests pass successfully:
Fixes #568
Notes
This implementation follows LiteLLM's pricing structure which uses separate rates for tokens above 200k, supporting models like Claude 4 Sonnet that have 1M context windows.
Summary by CodeRabbit
New Features
Tests
Documentation