fix(codex): date fallback review model pricing - #1303
Conversation
Codex can emit the internal codex-auto-review label for review runs, but pricing that label as gpt-5.5 for every historical row overstates older usage. Resolve that label while parsing Codex logs, using a small release-date table from models.dev so each event falls back to the newest known Codex/OpenAI model available on the log date without extra I/O in the hot path. Remove the undated pricing alias for codex-auto-review, keep the raw alias handling for gpt-5.3-spark, and document the date-aware fallback behavior in the Codex guide.
|
@coderabbitai review\n@cubic-dev-ai review |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughCentralizes ChangesCodex Auto-Review Model Resolution
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
✅ Action performedReview finished.
|
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 01b690b | Commit Preview URL Branch Preview URL |
Jun 12 2026, 02:01 PM |
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-x64
commit: |
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — resolves codex-auto-review log labels to the newest known Codex/OpenAI model per event date during parsing, removes the undated pricing alias so older logs no longer price as gpt-5.5.
- Date-aware model resolution in parser — extracted duplicated model resolution into
resolve_codex_usage_modelwith acodex_log_model_fallbackthat mapscodex-auto-reviewper-event timestamp against a sorted table of(release_date, model_name)pairs. - Pricing alias removed —
"codex-auto-review" => Some("gpt-5.5")deleted frompricing_alias; the pricing module no longer resolves it globally. - New tests —
resolves_codex_auto_review_to_latest_model_for_event_date(direct turn.completed) andresolves_codex_auto_review_turn_context_for_each_event_date(context propagation); existing test renamed and inverted. - Docs update — Codex guide reflects the date-aware resolution path.
Big Pickle (free via Pullfrog for OSS) ｜ 𝕏
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
rust/crates/ccusage/src/adapter/codex/parser.rs (2)
499-512: ⚡ Quick winInconsistent default fallback models for edge cases.
When the timestamp date cannot be parsed (line 504), the code defaults to
"gpt-5.5"(the newest model). However, when the parsed date predates all known releases (line 510), it defaults to"gpt-5"(the oldest model). This inconsistency may confuse future maintainers and could cause old logs with malformed timestamps to be priced at the newest model rate. Consider defaulting both edge cases to the same conservative fallback (likely"gpt-5"), or document the rationale if the asymmetry is intentional.♻️ Proposed consistent conservative fallback
let Some(date) = codex_timestamp_date(timestamp) else { - return Some("gpt-5.5"); + return Some("gpt-5"); };Alternatively, document the rationale:
+ // If timestamp is malformed, assume it's recent and use the newest model. + // Well-formed timestamps should always parse successfully after upstream normalization. let Some(date) = codex_timestamp_date(timestamp) else { return Some("gpt-5.5"); };🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/parser.rs` around lines 499 - 512, The function codex_log_model_fallback has inconsistent edge-case defaults: when codex_timestamp_date(timestamp) fails it returns "gpt-5.5" but when the parsed date predates known releases it falls back to "gpt-5"; change the parse-failure branch to return the same conservative fallback ("gpt-5") as the unwrap_or branch so both edge cases are consistent. Locate codex_log_model_fallback and replace the return Some("gpt-5.5") in the codex_timestamp_date None branch with Some("gpt-5"), or alternatively add a short comment explaining and intentionally preserving the asymmetry if that is required. Ensure references: CODEX_AUTO_REVIEW_MODEL, codex_timestamp_date, and CODEX_AUTO_REVIEW_FALLBACK_MODELS remain unchanged.
37-46: 💤 Low valueDocument the fallback table ordering requirement.
The fallback table must remain ordered newest-to-oldest for the
find_maplogic at lines 507-510 to work correctly (it picks the first entry wheredate >= released_on). Consider adding a comment above the constant to make this constraint explicit for future maintainers.📝 Suggested documentation comment
+// Release date → model fallback table for codex-auto-review. +// IMPORTANT: Entries must be ordered newest-to-oldest by release date so that +// find_map picks the newest model available on a given log date. const CODEX_AUTO_REVIEW_FALLBACK_MODELS: [(&str, &str); 7] = [ ("2026-04-23", "gpt-5.5"),🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/parser.rs` around lines 37 - 46, Add a clear comment above the CODEX_AUTO_REVIEW_FALLBACK_MODELS constant documenting that the array must be ordered newest-to-oldest (descending by date) because the parser logic (the find_map that compares date >= released_on) relies on the first matching entry; mention that changing the order will break the selection of fallback models and that maintainers must insert new entries at the front to preserve behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/guide/codex/index.md`:
- Line 63: The doc incorrectly states that the CLI resolves the
`codex-auto-review` alias via LiteLLM pricing data; update the text to reflect
the new flow: explain that the parser maps `codex-auto-review` to a dated
fallback (i.e., the parser’s alias mapping step) before any pricing runs, and
that `pricing.rs` no longer resolves that label — make the wording explicitly
say the parser performs the alias-to-dated-fallback mapping and pricing only
uses the already-resolved model name.
---
Nitpick comments:
In `@rust/crates/ccusage/src/adapter/codex/parser.rs`:
- Around line 499-512: The function codex_log_model_fallback has inconsistent
edge-case defaults: when codex_timestamp_date(timestamp) fails it returns
"gpt-5.5" but when the parsed date predates known releases it falls back to
"gpt-5"; change the parse-failure branch to return the same conservative
fallback ("gpt-5") as the unwrap_or branch so both edge cases are consistent.
Locate codex_log_model_fallback and replace the return Some("gpt-5.5") in the
codex_timestamp_date None branch with Some("gpt-5"), or alternatively add a
short comment explaining and intentionally preserving the asymmetry if that is
required. Ensure references: CODEX_AUTO_REVIEW_MODEL, codex_timestamp_date, and
CODEX_AUTO_REVIEW_FALLBACK_MODELS remain unchanged.
- Around line 37-46: Add a clear comment above the
CODEX_AUTO_REVIEW_FALLBACK_MODELS constant documenting that the array must be
ordered newest-to-oldest (descending by date) because the parser logic (the
find_map that compares date >= released_on) relies on the first matching entry;
mention that changing the order will break the selection of fallback models and
that maintainers must insert new entries at the front to preserve behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 4ff84d5f-7469-456e-bc32-c626db819176
📒 Files selected for processing (4)
docs/guide/codex/index.mdrust/crates/ccusage/src/adapter/codex/loader.rsrust/crates/ccusage/src/adapter/codex/parser.rsrust/crates/ccusage/src/pricing.rs
Address review feedback on the date-aware codex-auto-review fallback. Malformed timestamps now use the same conservative gpt-5 fallback as dates before the known release table, avoiding accidental newest-model pricing for bad data. The Codex guide now states that the parser maps codex-auto-review before pricing, and the fallback table documents its descending date-order invariant.
|
@coderabbitai review\n\nAddressed the review feedback in 0b0d653: clarified that the Codex parser maps `codex-auto-review` before pricing, made malformed timestamps use the conservative `gpt-5` fallback, and documented the fallback table order invariant.\n\nValidation: `nix develop --command cargo test --manifest-path rust/Cargo.toml -p ccusage codex` and pre-push hooks passed. |
|
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rust/crates/ccusage/src/adapter/codex/parser.rs (1)
500-512:⚠️ Potential issue | 🟠 Major | ⚡ Quick winMalformed timestamp validation is still too permissive, so newest-model pricing can leak through.
At Line 515-524,
codex_timestamp_dateaccepts anyYYYY-MM-DD-shaped prefix (e.g.2026-99-99), and at Line 510-511 that value can still resolve to the newest fallback model. This breaks the conservative malformed-timestamp behavior (expectedgpt-5) and can misprice bad data.Proposed fix
fn codex_log_model_fallback(model: &str, timestamp: &str) -> Option<&'static str> { if model != CODEX_AUTO_REVIEW_MODEL { return None; } - let Some(date) = codex_timestamp_date(timestamp) else { + let Some(date) = codex_timestamp_date(timestamp) else { return Some("gpt-5"); }; Some( CODEX_AUTO_REVIEW_FALLBACK_MODELS .iter() .find_map(|(released_on, fallback)| (date >= *released_on).then_some(*fallback)) .unwrap_or("gpt-5"), ) } fn codex_timestamp_date(timestamp: &str) -> Option<&str> { let date = timestamp.get(..10)?; let bytes = date.as_bytes(); - (bytes.len() == 10 + let is_basic_shape = bytes.len() == 10 && bytes[0..4].iter().all(u8::is_ascii_digit) && bytes[4] == b'-' && bytes[5..7].iter().all(u8::is_ascii_digit) && bytes[7] == b'-' - && bytes[8..10].iter().all(u8::is_ascii_digit)) - .then_some(date) + && bytes[8..10].iter().all(u8::is_ascii_digit); + if !is_basic_shape { + return None; + } + let month = (bytes[5] - b'0') * 10 + (bytes[6] - b'0'); + let day = (bytes[8] - b'0') * 10 + (bytes[9] - b'0'); + ((1..=12).contains(&month) && (1..=31).contains(&day)).then_some(date) }Also applies to: 515-524
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/parser.rs` around lines 500 - 512, codex_timestamp_date is too permissive and allows invalid YYYY-MM-DD-like prefixes, letting codex_log_model_fallback pick newer models; fix by making codex_timestamp_date perform strict date parsing (e.g. use chrono::NaiveDate::parse_from_str("%Y-%m-%d") or equivalent) and return None on parse failure so codex_log_model_fallback will return the conservative Some("gpt-5"). Update codex_timestamp_date implementation and add/adjust tests for invalid dates; keep codex_log_model_fallback, CODEX_AUTO_REVIEW_MODEL and CODEX_AUTO_REVIEW_FALLBACK_MODELS usage unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@rust/crates/ccusage/src/adapter/codex/parser.rs`:
- Around line 500-512: codex_timestamp_date is too permissive and allows invalid
YYYY-MM-DD-like prefixes, letting codex_log_model_fallback pick newer models;
fix by making codex_timestamp_date perform strict date parsing (e.g. use
chrono::NaiveDate::parse_from_str("%Y-%m-%d") or equivalent) and return None on
parse failure so codex_log_model_fallback will return the conservative
Some("gpt-5"). Update codex_timestamp_date implementation and add/adjust tests
for invalid dates; keep codex_log_model_fallback, CODEX_AUTO_REVIEW_MODEL and
CODEX_AUTO_REVIEW_FALLBACK_MODELS usage unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: fc70e891-0549-44af-ae16-358246236417
📒 Files selected for processing (2)
docs/guide/codex/index.mdrust/crates/ccusage/src/adapter/codex/parser.rs
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/guide/codex/index.md
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
Resolve codex-auto-review model fallback dates from the raw log timestamp used for model selection instead of only the normalized event timestamp. This prevents malformed timestamp strings from being discarded and replaced by file mtime before the fallback decision. Validate YYYY-MM-DD components before comparing release dates, including month ranges, month lengths, and leap years. Invalid or missing dates now conservatively resolve to gpt-5. Add regression coverage for invalid month and non-leap-day Codex review logs.
|
Addressed the malformed-date review fallback issue in 8f0937e. Changes:
Validation:
|
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — calendar-level date validation, conservative malformed-timestamp fallback, and per-exec-event timestamp separation for model resolution.
- Calendar-level date validation —
codex_timestamp_datenow validates month range and day-of-month bounds including leap years, rejecting2026-99-99and2026-02-29as invalid. - Conservative fallback — malformed-timestamp fallback changed from
"gpt-5.5"to"gpt-5", consistent with the pre-table-date fallback (addresses prior CodeRabbit feedback). - Timestamp separation for exec events —
CodexExecTimestampsstruct andraw_or_normalized_*helpers isolate the model-resolution timestamp from the event timestamp, soresolve_codex_usage_modeluses the raw log timestamp for date-aware fallback. - New test —
resolves_codex_auto_review_with_invalid_event_date_to_conservative_modelvalidates the conservative fallback for impossible dates.
Big Pickle (free via Pullfrog for OSS) ｜ 𝕏
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
DiagnosisThe check suite (73751687517) on PR #1303 reported Local verificationI ran the full suite of checks that CI would execute and all pass cleanly:
No code changes were needed — the PR's code is correct and all checks pass locally. Action takenPushed the current tip to Task list (5/5 completed)
|
Replace the hand-written codex-auto-review release table in the parser with an embedded snapshot generated from the pinned models.dev input. Extend the models.dev snapshot generator to emit Codex auto-review fallback metadata alongside the existing pricing snapshot, and wire the just recipe to refresh both generated files. Keep runtime lookup fast by parsing the tiny embedded snapshot once through LazyLock, while preserving the existing date-aware fallback behavior and malformed-date conservative fallback.
|
Good catch. The previous implementation had the fallback release table hand-written in parser.rs. Updated in 5331af1:
Validation:
|
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes — Extracted the codex-auto-review fallback table from a hardcoded Rust array to a committed JSON file, auto-generated from the pinned models.dev catalog via a new TS generator.
- Extract fallback data to JSON — The hardcoded
CODEX_AUTO_REVIEW_FALLBACK_MODELSarray replaced withinclude_str!+ zero-copy serde deserialization fromcodex-auto-review-fallbacks.json, keeping the same 7 entries. - New TS generator —
nix/models-dev-gen.tsupdated to produce the fallback JSON alongside the existing pricing snapshot, filtering to GPT-5 family models and deduplicating codex/non-codex variants per decimal version. - Updated build pipeline —
nix/models-dev-pricing.nixoutputs both snapshots;justfilerecipe copies both files and formats them. - New test —
loads_codex_auto_review_fallbacks_from_models_dev_snapshotvalidates length, content, and descending date order of the embedded data.
Big Pickle (free via Pullfrog for OSS) ｜ 𝕏
The Nix cleanSource filter only whitelists specific JSON files alongside Cargo sources. The newly added codex-auto-review-fallbacks.json embedded via include_str! by the codex parser was being filtered out, causing the Linux/macOS/Windows native package builds to fail with 'No such file or directory' at compile time. Co-authored-by: Codesmith <[email protected]>
There was a problem hiding this comment.
🧹 Nitpick comments (1)
rust/crates/ccusage/src/adapter/codex/parser.rs (1)
968-979: ⚡ Quick winRelax the snapshot-size assertions in this unit test.
just gen-models-dev-pricingnow regeneratescodex-auto-review-fallbacks.json, but this test still hardcodes the current snapshot length and edge entries. The next pinned models.dev bump will failjust checkuntilparser.rsis manually edited too, even if the generated snapshot and parser logic are correct. Prefer asserting non-empty data plus ordering/shape, or compare directly against the committed JSON artifact instead.♻️ Possible simplification
let fallbacks = codex_auto_review_fallback_models(); - assert_eq!(fallbacks.len(), 7); - assert_eq!(fallbacks[0].released_on, "2026-04-23"); - assert_eq!(fallbacks[0].model, "gpt-5.5"); - assert_eq!(fallbacks[6].released_on, "2025-08-07"); - assert_eq!(fallbacks[6].model, "gpt-5"); + assert!(!fallbacks.is_empty()); + assert!(fallbacks + .iter() + .all(|fallback| !fallback.released_on.is_empty() && !fallback.model.is_empty())); assert!(fallbacks .windows(2) .all(|window| window[0].released_on > window[1].released_on));🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rust/crates/ccusage/src/adapter/codex/parser.rs` around lines 968 - 979, The test loads_codex_auto_review_fallbacks_from_models_dev_snapshot is brittle because it hardcodes snapshot length and exact edge entries; update it to validate shape and ordering instead: call codex_auto_review_fallback_models(), assert the returned Vec is non-empty, verify each entry has expected fields populated (e.g., model and released_on not empty) and that the list is strictly sorted by released_on using the existing windows check; optionally replace exact value checks with a comparison to the committed codex-auto-review-fallbacks.json artifact if you prefer exact match.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@rust/crates/ccusage/src/adapter/codex/parser.rs`:
- Around line 968-979: The test
loads_codex_auto_review_fallbacks_from_models_dev_snapshot is brittle because it
hardcodes snapshot length and exact edge entries; update it to validate shape
and ordering instead: call codex_auto_review_fallback_models(), assert the
returned Vec is non-empty, verify each entry has expected fields populated
(e.g., model and released_on not empty) and that the list is strictly sorted by
released_on using the existing windows check; optionally replace exact value
checks with a comparison to the committed codex-auto-review-fallbacks.json
artifact if you prefer exact match.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 1a088920-2eaa-47ac-b7fa-afc576171bd8
📒 Files selected for processing (7)
docs/guide/codex/index.mdjustfilenix/models-dev-gen.tsnix/models-dev-pricing.nixnix/packages.nixrust/crates/ccusage/src/adapter/codex/codex-auto-review-fallbacks.jsonrust/crates/ccusage/src/adapter/codex/parser.rs
✅ Files skipped from review due to trivial changes (2)
- nix/packages.nix
- docs/guide/codex/index.md
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 9 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Replace ';' with '&&' between the nix build and two cp commands in the gen-models-dev-pricing recipe so that a failed earlier step doesn't get silently masked by a later successful cp. Co-authored-by: Codesmith <[email protected]>
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
…s malformed The model-date resolution chain in codex_model_timestamp_from_result used raw_or_normalized_codex_timestamp, which returned any non-empty string as-is regardless of whether it parsed as a date. That short-circuited the or_else chain, so a malformed top-level `timestamp` would prevent the parser from looking at `created_at`, `created_at_camel`, nested result-field timestamps, or the file mtime fallback, and forced events through the conservative gpt-5 mapping. Validate the raw string against codex_timestamp_date before short-circuiting; fall through to normalize_codex_timestamp / normalize_value_timestamp and finally None so the rest of the chain can supply a valid date. Mirror the same change for the serde_json::Value path. Also fix the models.dev fallback generator so a gpt-5.x decimal base model is only dropped when a gpt-5.x-codex variant exists on the SAME release date. When the codex variant ships later, keeping the base entry lets events in the gap still resolve to the most recent model actually available then. Co-authored-by: Codesmith <[email protected]>
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runner startupExecution setup measures any pre-benchmark package materialization used by the execution benchmark. Bunx temp cache measures one
Cached bunx execution performanceRuns the same large fixture through Fixtures: Claude
Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
1 participant
Summary:
Testing:
Need help on this PR? Tag
/codesmithwith what you need. Autofix is enabled.Summary by cubic
Fix Codex review pricing by resolving
codex-auto-reviewto the newest available model per log date during parsing, using an embeddedmodels.devsnapshot. Invalid or missing dates fall back togpt-5, and the embedded snapshot is now included in Nix builds.models.dev, with correct base vs-codexsame-day handling; parse once at runtime (no extra I/O).just gen-models-dev-pricingnow refreshes both snapshots.created_at/createdAtand nested fields when the top-level value is malformed; validate YYYY-MM-DD, month lengths, and leap years; invalid/pre-table/missing dates map togpt-5.pricingsocodex-auto-reviewdoesn’t resolve globally or set a context limit.turn_context.just gen-models-dev-pricingfail fast by chaining build and copy with&&.Written for commit 01b690b. Summary will update on new commits.
Summary by CodeRabbit
Documentation
Behavior
codex-auto-reviewno longer maps to a fixed model and is treated as not found (no context limit); dated aliases resolve to the newest model for the event date.Tests
Chores