fix(sleep): enforce max_tokens_per_night in the nightly cycle - #317

Fixes #313

Behavior change first

The default max_tokens_per_night is 400,000, and with this change it takes effect. Attempt prompts embed the whole skill and the whole CLAUDE.md, so on the CLI backends a 40-task night with a 12 to 20 KB CLAUDE.md comes to roughly 300k to 500k estimated tokens by our count. Nights in that range will now skip their remaining steps and say so in the report. Setting "max_tokens_per_night": 0 restores the previous behavior.

If you would rather not change behavior for existing users, the alternative is a one-line change of the default to 0, which makes the budget opt-in. I am happy to switch to that, and to add a CHANGELOG.md [Unreleased] entry in either case.

Problem

skillopt_sleep/config.py sets "max_tokens_per_night": 400_000, and skillopt_sleep/budget.py has a Budget controller that is meant to stop cleanly and report what it skipped. On main nothing connects them: the key is never read outside DEFAULTS, and Budget is only constructed by the gbrain experiment harness and its unit test. A night runs every step regardless of how many tokens it has already used.

The new regression test shows it with a deterministic backend that bills 100 tokens per uncached call. With max_tokens_per_night set to 1,200, the night on main spends 2,500 tokens: after the budget is reached it still reflects, runs the skill gate trial and replays the training tasks again. With per-skill fan-out and a budget of 1, it runs the full consolidation and both groups (three reflect calls).

Root cause

• skillopt_sleep/config.py:43 defines the key, but run_sleep_cycle (skillopt_sleep/cycle.py) never reads it.
• Budget (skillopt_sleep/budget.py:15) is never created in the engine, so consolidate(), the held-out test replay and consolidate_groups() have no way to know the budget is spent.

Change

run_sleep_cycle now builds a Budget from max_tokens_per_night at the start of the night and checks it before each step that would start new model work:

1. the consolidation itself,
2. its skill edit step (reflect plus the gate trial),
3. its memory edit step (training replay, reflect, gate trial),
4. the final validation replay,
5. the held-out test replay,
6. each per-skill fan-out group.

Once the budget is reached, steps that have not started are skipped and the night records it:

• report.md and report.json get one note, for example max_tokens_per_night reached (431870 of 400000 tokens); skipped: memory edits.
• Each skip is logged as a budget skip evidence event, the cycle end event carries max_tokens_per_night and budget_skipped, and max_tokens_per_night is added to the logged cycle config.
• A fan-out group that never started is a skipped row with the reason token budget for the night was reached. A group that started and was cut short keeps its own gate result and says which step it skipped.
• If the budget is already spent before the consolidation starts (possible when a backend counts mining usage), the consolidation is skipped with gate_action="budget_exhausted", no proposal is staged, report.md shows no held-out score instead of 0.000, and the night is recorded in state.json and diagnostics.json without scores. The plain-text CLI summary still prints held-out 0.000 -> 0.000 => budget_exhausted, and report.json (and the --json output) keep the default 0.0 scores next to gate_action: budget_exhausted, the same way a night with no tasks prints zeros. I left those to keep the change small and avoid changing the SleepReport type; happy to change them if you prefer.

Why step boundaries

A step that has started always finishes. That keeps the tokens already spent useful (a skill edit that passed its gate trial is still staged) and keeps the change small, and every accept or reject still rests on a complete validation replay. Staging, state and adoption writes are never skipped. Stopping at individual calls and discarding the unfinished step would make the cap tighter; that could be a follow-up if you want it.

The cost is overshoot. In the worst case a night ends above the budget by one step:

• the first replay of the validation and training tasks,
• a skill or memory edit step: a training replay, a reflect call and a validation replay of the candidate,
• with dream_rollouts=K, the skill step runs K attempts per training task before its reflect call and validation replay,
• the final validation replay, when the budget is not yet reached as it starts,
• the held-out test replay,
• each fan-out group that starts runs its own first replay.

The budget is also checked before the final validation replay. When it is spent, the gate decides from the validation replay that already scored the final skill and memory (the baseline replay, or the trial replay of the last kept edit) instead of starting a new one, because tool replay and failed calls are not served from the response cache. The trade-off is that on a budget-stopped night a kept edit is not re-confirmed by a second replay.

Handoff backend

A handoff round counts unanswered prompts and their placeholder replies. If the budget were applied to that count, a round could skip its reflect step, finish on placeholder replies, stage a report and save state.json while still exiting 3. So when the budget is reached while prompts are pending, the budget check raises PendingCalls and the round pauses for answers like any dependent call. A round only stops on the budget once every counted call has a real answer. When the budget is not reached, or is disabled, handoff rounds behave exactly as before.

Values

Budget already treats a missing or zero limit as unlimited, and lookback_hours already uses 0 for "no limit", so 0 and null disable the budget. Negative numbers, booleans, fractional or infinite floats (including JSON Infinity) and non-numeric strings raise ValueError before any model call, like the existing split-fraction validation.

Reuse and compatibility of the internals

The change reuses Budget as it is. consolidate(), dream_consolidate() and consolidate_groups() take an optional budget_exhausted callable that defaults to None, so the experiment harnesses and other callers behave exactly as before. ConsolidationResult gets one optional trailing field, budget_skipped, which keeps positional construction working. Only tonight's spend counts (Budget.start records the backend's total when the night begins), so a backend object reused across nights is not stopped early.

A night that stops on the budget still advances the harvest checkpoint, as every other outcome does; otherwise a night whose mining alone used up the budget would mine the same sessions again every night. Skipped work is not retried later, except that tonight's tasks stay in the task archive that recall_k draws on. The docs say this.

How spend is counted

The budget uses the backend's token counter, the one the report header prints as tokens used. The header shows the backend's running total and the note shows tonight's spend; they only differ when a caller reuses one backend object across nights. This PR does not change how tokens are counted, and the counter has gaps that the new docs section lists:

• CLI backends estimate from prompt and reply length. LLM task mining and contrastive reflection (dream_rollouts above 1, rollout.py) call backend._call directly and are not counted.
• azure_openai adds the API's reported usage inside _call, and CliBackend._cached_call and reflect then add the length estimate for the same call, so replay and reflect calls count roughly twice while mining is counted once. fix(sleep): thread-safe backend cache + redact exports #251 also reworks per-attempt token accounting.
• handoff counts the prompt and answer text of the current round.
• mock reports no usage.

There is no elapsed-time budget.

Tests

New tests/test_sleep_token_budget.py (stdlib unittest, no network). The main backend is a MockBackend that bills a fixed cost per uncached attempt and per reflect call and, like the CLI backends, serves repeated attempts on unchanged documents from cache.

• Budget equal to the cost of the first replay: no reflect call, spend stays within the budget, nothing is staged, one note names the skipped steps, report.md shows it and the evidence end event lists them.
• Boundary: one more token of headroom lets the skill step run and keep its validated edit, and only the memory step is skipped.
• Dry run: the same stop happens and nothing is staged or saved.
• Budget spent before consolidation (a counter that jumps after the night's first reading): no model call, budget_exhausted, no 0.000 in the report header, no scores in state.json history or diagnostics.json.
• 0, null, "0" and 0.0 behave exactly like the default budget on an ordinary night (same calls, accepted, no note); the default 400,000 leaves that night unchanged.
• A backend that already spent tokens on an earlier night is not stopped early.
• The held-out test replay is skipped, noted and logged once the budget is spent.
• Per-skill fan-out: groups that did not start are skipped rows with the budget reason and a skip event each, no reflect runs and no group replays against its own skill document.
• -1, "lots", 0.5, True, inf and a JSON Infinity raise before any model call.
• Handoff, CLI level: skillopt-sleep run --backend handoff with max_tokens_per_night: 1 in the config file exits 3 with PROMPTS.md, writes no state.json and stages no report.
• Handoff, two rounds: the first round pauses with PendingCalls; once its prompts are answered, the second round applies the budget, skips the skill and memory edits, has nothing pending and records night 1.
• consolidate() with a spent budget makes no call and returns budget_exhausted; with a budget that is never reached it makes the same calls and decision as without one.
• consolidate_groups() skips groups that have not started and marks a group that was cut short.

Results (Ubuntu, Python 3.10 and 3.12, same commands as CI):

# the first commit's 16 tests against main's source (only the test file added)
$ python -m pytest -q tests/test_sleep_token_budget.py
16 failed, 5 passed, 4 subtests passed
# e.g. "2500 not less than or equal to 1200", "'failed' != 'consolidated'"

# this branch
$ python -m pytest -q tests/test_sleep_token_budget.py
21 passed, 9 subtests passed

$ python -m pytest -q
1735 passed, 12 skipped, 368 subtests passed

$ python -m mkdocs build --strict
INFO    -  Documentation built in 1.36 seconds

ruff check on the changed files reports only the import-order warning that skillopt_sleep/cycle.py already has on main.

Docs

• docs/sleep/README.md: new "Nightly token budget" section: what is checked, the report note, valid values, worst-case overshoot, how each backend counts spend, and that a budget-stopped night still advances the harvest checkpoint.
• plugins/README.md and plugins/claude-code/README.md: the safety lists no longer say the CLI has no token budget; they describe max_tokens_per_night and say it is not a hard cap and there is no wall-clock budget.
• plugins/openclaw/README.md: no longer says max_tokens_per_night is not enforced. The wrapper's run_sleep.py passes its config.json (800,000) to the shared cycle, so it now applies, measured with that backend's own rough estimate.

Notes for maintainers

• Base 343db229dbd5ddaf9df6b1d5540d8bcdb2604d5c, head 356f8c8096aa3b73240a1e71a2fd6548dd293788, one commit.
• feat(sleep): add adversarial candidate probes #263 touches the same dream_consolidate signature and call; whichever lands second needs a trivial merge that keeps both sets of keyword arguments. If feat(sleep): add adversarial candidate probes #263 lands, its adversarial probe replays would be another step to check the budget before.