fix(skillopt-sleep): surface codex auth/model/version failures instead of silently scoring 0 - #92

Problem

A nightly sleep cycle could run for a long time emitting held-out 0.000 -> 0.000 (gate reject, zero edits) — indistinguishable from "nothing to learn" — when the real cause was the codex backend returning an error (expired auth / a model unsupported on a ChatGPT account / an outdated CLI) that got scored as a failed rollout. Nothing surfaced the error, so the loop silently never learned, and the per-cycle report gave no way to tell a genuine no-improvement night from a backend that never reached the model.

Fix

CodexCliBackend (_call path)

• Split _call into _call_once + a retry wrapper: transient empty/timeout calls are retried instead of silently returning "" (mirrors AzureOpenAIBackend's existing guard).
• On a non-zero exec exit, surface the reason via last_call_error and return "" rather than leaking the CLI's error text as if it were a model response.
• Fail fast (no retries) on fatal, non-recoverable errors: 401 Unauthorized, refresh_token_reused, token_expired, not supported when using Codex with a ChatGPT account, requires a newer version of Codex.

CodexCliBackend.attempt_with_tools (tool-rollout path)

• Same treatment: a failed tool rollout now sets last_call_error (timeout / exception / non-zero exit) instead of swallowing the exception — so an auth/model/version failure on the tool path is visible in diagnostics rather than a silent empty→0. Response stays empty; the CLI error text is never returned as a "response".

CliBackend.reflect — retain last_reflect_raw so a no-edits night is diagnosable (empty/non-JSON optimizer reply vs genuinely no failures).

consolidate — ConsolidationResult now carries per-task held-out detail (response_len, response_head, hard/soft, why) + reflect_raw + call_error. Observability only; the gate's accept logic and the learning algorithm are unchanged.

cycle — write a diagnostics.json per cycle so a 0.0 night self-explains; fully try/except-wrapped so it can never break the cycle.

.gitignore — ignore the .skillopt-sleep/ runtime dir.

Why it matters

Turns a silent multi-night failure into a single-glance diagnosis: a backend auth/model/version problem now shows up immediately as a populated call_error in diagnostics.json instead of masquerading as "the gate found nothing worth adopting."

How it was tested

• Unit: 5 new tests, all green; full tests/test_sleep_engine.py suite passes (52 passed, 1 skipped; the only 3 failures are pre-existing Windows path-separator assertions that also fail on main). Tests: _call retry-not-silent-zero; auth-error surfaced-not-scored + no-retry; tool-path failure surfaced-not-silent; holdout_detail capture; reflect_raw capture.
• End-to-end: a capped real-backend cycle (codex + gpt-5.5) went from the prior held-out 0.000 -> 0.000 reject to 0.000 -> 1.000 accept_new_best (2 edits staged), with a populated diagnostics.json (empty call_error, real reflect_raw, non-empty responses) — i.e. the loop produced its first real learning signal once the backend was reachable.
• Not e2e-verified: the attempt_with_tools tool-path change is covered by a unit test but was not exercised against a live failing tool-rollout end-to-end.
• Lint: adds no new ruff violations (net −1 vs main); py_compile clean.

Review

Reviewed in a fresh, independent context against the stated intent (which surfaced the tool-path gap, now fixed).

Risk: low

Offline nightly tool; the change is error-handling hardening + read-only observability, not on any request/auth/data/money path, and does not alter the gate or learning algorithm. Worst case for a consumer is an unchanged cycle plus a diagnostics.json file.

🤖 Generated with Claude Code