Bug Fixes

• See More

  command-executed: Keep whole argv-joined payload in shell unwrap
  (#77,
  7abd080)

• command-executed: Match patterns against shell-normalized commands
  (#77,
  7abd080)

• command-executed: Narrow command param to str before shell-normalizing
  (#77,
  7abd080)

• command-executed: Recognize shell wrappers by predicate, not allowlist
  (#77,
  7abd080)

• criteria: Add present predicate so asserting a switch cannot weaken a guard
  (#72,
  8574ded)

• criteria: Make cli_called guards fail loud instead of vacuously passing
  (#72,
  8574ded)

• criteria: Stop ignore_flags re-opening the guard false-PASS
  (#72,
  8574ded)

• early-stop: Address PR review — trajectory parity, reason determinism, doc restore
  (#78,
  4cf8092)

• lint: Derive CE030 criteria from the source union literal, not runtime
  (#77,
  7abd080)

• lint: Enumerate in-tree criteria by module attribute, not the union
  (#77,
  7abd080)

• lint: Scope CE030 criterion parity to in-tree criteria only
  (#77,
  7abd080)

• reports: Explicit return on every early_stop_gate_note path (CodeQL py/mixed-returns)
  (#78,
  4cf8092)

• reports: Pre-initialize the gate note so CodeQL sees it bound on every path
  (#78,
  4cf8092)

Chores

• deps-dev: Bump postcss from 8.5.18 to 8.5.23 in /evalboard
  (#75,
  cdced15)

Documentation

• Surface the Marketplace listing and make the Action quickstarts self-sufficient
  (#80,
  401245a)

• command-executed: Document shell-normalization contract + gate it (CE030)
  (#77,
  7abd080)

Features

• criteria: Add cli_called for structured invocation matching
  (#72,
  8574ded)

• criteria: Match a flag across spellings with FlagMatch.aliases
  (#72,
  8574ded)

• early-stop: Per-criterion arming via stop_early blocks on live criteria
  (#78,
  4cf8092)

Refactoring

• command-executed: Total _match_haystacks, shared window, memoized
  (#77,
  7abd080)

Bug Fixes

• litellm: Pin litellm[proxy]==1.95.0 + fastapi==0.140.0 for proxy startup
  (#76,
  f2f8580)

• litellm: Pin proxy deps (litellm 1.95.0 + fastapi 0.140.0) to fix startup crash
  (#76,
  f2f8580)

Chores

• action: Rename Marketplace listing to coder_eval, add author
  (a9c274d)

Documentation

• litellm: Surface the proxy dep-pin override vars in start script
  (#76,
  f2f8580)

Refactoring

• litellm: Address PR review — pin SSOT guard, rename, doc ripple
  (#76,
  f2f8580)

Bug Fixes

• early-stop: Address PR review — polarity-blind budget, pass_threshold displacement,
  gate-semantic split (#74,
  800ac77)

Chores

• deps: Bump aiohttp 3.14.1→3.14.3, cryptography 49.0.0→50.0.0
  (#74,
  800ac77)

Features

• early-stop: Weighted ceiling/floor bounds + decision-step budget
  (#74,
  800ac77)

Bug Fixes

• cost: A task timeout with no preserved turn is unrecorded spend, not free
  (#63,
  93c7fc0)

• cost: Book spend on the error and timeout paths, flag what is unpriced
  (#63,
  93c7fc0)

• cost: Flag every hard-killed task as a cost floor, not just the empty ones
  (#63,
  93c7fc0)

• evalboard: Honest scoped counts, and one definition of a run's scope
  (#69,
  0bdac0b)

• litellm: Gate cost_log_tags on agent capability, not route (fixes non-Claude crash)
  (#66,
  4131a2a)

• litellm: Make the orphaned-spend warning actually fire
  (#66,
  4131a2a)

• litellm: Per-attempt cost-log scoping + single run-id accessor + no-match warning
  (#66,
  4131a2a)

• litellm: Pin each open-weight model to a vetted provider set (no silent fallback)
  (#66,
  4131a2a)

• litellm: Proxy-authoritative token buckets + all-priced gate + transactional join
  (#66,
  4131a2a)

• litellm: Sanitize cost headers, reject non-finite cost, drop debug scaffolding
  (#66,
  4131a2a)

• orchestrator: Recover the in-flight turn's spend on a hard kill
  (#63,
  93c7fc0)

• pricing: Add the claude-opus-5 rate so killed turns stop booking zero
  (#63,
  93c7fc0)

• pricing: Add the five unpriced codex tiers still on OpenAI's rate card
  (#63,
  93c7fc0)

• pricing: Correct every wrong rate-card entry and close the alias gaps
  (#63,
  93c7fc0)

• pricing: Refresh the rate card and correct gemini-3-flash-preview
  (#63,
  93c7fc0)

• reports: Count errors as misses and stop losing cost on error paths
  (#63,
  93c7fc0)

• reports: Count errors as misses in one canonical pass rate
  (#63,
  93c7fc0)

Code Style

• evalboard: Drop the swatch dots and the scope caption from the header
  (#69,
  0bdac0b)

Documentation

• cost: Describe the per-turn backfill as the net it is
  (#63,
  93c7fc0)

• cost: Describe the unpriced-crash mechanism accurately and keep comments framework-general
  (#63,
  93c7fc0)

• litellm: Correct the cost contract after cutting per-message distribution
  (#66,
  4131a2a)

• litellm: Document LITELLM_COST_LOG wiring + correct the reconciliation-cost contract
  (#66,
  4131a2a)

Features

• cost: Publish one accurate total on every reporting surface
  (#63,
  93c7fc0)

• docker: Bind-mount the LiteLLM cost log so --driver docker joins actual cost
  (#66,
  4131a2a)

• evalboard: Compare every harness on the overview, and scope the whole page to one
  (#69,
  0bdac0b)

• evalboard: Compare harnesses on the overview, and identify each run
  (#69,
  0bdac0b)

• evalboard: Lift the harness scope to the page header, in vendor colors
  (#69,
  0bdac0b)

• evalboard: Make each turn's provider-call table a collapsed dropdown
  (#66,
  4131a2a)

• evalboard: Mark a partly-priced run total as a floor, not the bill
  (#63,
  93c7fc0)

• evalboard: One set of pass-rate cutoffs, and a run table that pages through all history
  (#69,
  0bdac0b)

• evalboard: Per-call cost/cache table from provider_call_costs (replaces inline)
  (#66,
  4131a2a)

• evalboard: Read the canonical pass rate and surface incomplete cost
  (#63,
  93c7fc0)

• evalboard: Say which harness, model, and framework version a run used
  (#69,
  0bdac0b)

• litellm: Actual per-call cost + cache accounting for the open-weight backend
  (#66,
  4131a2a)

Refactoring

• cost: Correct the simulator-cost bound and drop the unread variant error share
  (#63,
  93c7fc0)

• cost: Cut the commentary and drop unreachable rate-card keys
  (#63,
  93c7fc0)

• cost: Define the unpriced-row test once, and only for new runs
  (#63,
  93c7fc0)

• cost: Total_cost_usd means the whole bill everywhere
  (#63,
  93c7fc0)

• evalboard: Call the UiPath harness Delegate
  (#69,
  0bdac0b)

• litellm: Cut per-message distribution; turn-level join + per-call audit record
  (#66,
  4131a2a)

• litellm: Drop the provider field/column — unavailable on the streaming path
  (#66,
  4131a2a)

• litellm: Stream the cost log + de-duplicate the OpenRouter config comment
  (#66,
  4131a2a)

Testing

• litellm:...

Features

• agents: Extend cooperative early stop to codex and antigravity
  (b849421)

Features

• criteria: Async-primary BaseCriterion contract with CheckerMisuseError escalation
  (#60)