Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: Hailey-coding/SkillOpt
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: microsoft/SkillOpt
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 16 commits
  • 66 files changed
  • 20 contributors

Commits on Oct 17, 2026

  1. fix(sleep): thread-safe backend cache + redact exports (microsoft#251)

    * fix(sleep): thread-safe backend cache + redact exports
    
    - Guard CliBackend _cache/_tokens with a lock on the opt-in parallel replay
      path (SKILLOPT_SLEEP_WORKERS>1) so a concurrent miss cannot corrupt state
      or lose the token cost metric; the model call stays outside the lock so
      parallel workers still overlap.
    - Redact report.json (redact_secrets) before staging.
    - Redact harvest --output and --json exports (_redact_deep).
    - Add tests for CliBackend caching/thread-safety.
    
    * fix(sleep): mapping-key-aware redaction at export boundaries
    
    Address maintainer review on microsoft#251:
    - Use redact_secrets (mapping-key aware) instead of _redact_deep for harvest
      --output/--json and handoff exports, so nested api_key/token mappings are
      redacted (not just bare string leaves).
    - Redact report_md before writing it alongside report.json.
    - Add boundary tests (nested api_key/token + report.md).
    
    * fix(sleep): thread-safe cache access through locked helpers
    
    Address maintainer review on microsoft#251 (thread-safety):
    - Add _cache_get/_cache_pop/_cache_pop_if locked helpers and route the Pi and
      OpenCode _cached_call overrides through them (they previously read/pop the
      cache outside the lock).
    - tokens_used() now reads _tokens under the lock.
    - Popping a failed entry is conditional (_cache_pop_if): a failed caller only
      drops its own empty value, never another worker's just-stored success.
    - Add tests: barrier-forced overlapping misses stay consistent, and pop-if
      does not delete a successful entry.
    
    * fix(sleep): call-local token accounting for parallel replay
    
    Address maintainer review on microsoft#251 (last thread-safety item):
    - Record each model call's token delta on the calling thread (thread-local),
      so parallel replay_one() charges its own cost instead of a before/after
      global total that an overlapping worker inflates.
    - replay_one reads backend.token_delta() (falling back to the text-length
      heuristic for backends that don't track tokens).
    - Add tests for call-local and thread-isolated token deltas.
    
    * fix(sleep): never cache empty results; add Pi concurrency test
    
    Address maintainer review on microsoft#251 (deepen thread-safety):
    - _cached_call no longer caches empty (transient-failure) results and prefers a
      concurrently cached success, so an empty/duplicate cannot clobber or delete
      another worker's successful entry.
    - Add a Pi subclass-level concurrency test (barrier-forced empty-vs-success)
      asserting the success survives.
    
    * fix(sleep): key-aware redaction + correct concurrent token accounting
    
    - Redefine _redact_deep to delegate to the key-aware redact_secrets walker so
      {"api_key": "x"} is scrubbed at every boundary (--json, digests/snapshot
      files, gate_trials, extra, display), not just the --json link.
    - Reset _thread_local.delta on cache hit so a later hit doesn't reuse the
      previous call's delta.
    - Charge every real call's tokens on a concurrent miss (the dedup worker used
      to be free, undercounting).
    - Add DualBackend.token_delta() so replay_one() reads the target's call cost.
    - Regressions: cache-hit delta reset, barrier-forced concurrent charge,
      DualBackend token_delta, key-aware _redact_deep.
    
    * test(sleep): raise barrier wait timeout to avoid slow-CI flakiness
    
    - The barrier-forced concurrency tests waited 5s for all workers to reach the
      barrier; under a slow/loaded CI that can break the barrier mid-test and turn
      a pass into a spurious failure. Raise the wait to 15s (no semantic change).
    
    * docs(sleep): clarify DualBackend.token_delta intent
    
    Document that token_delta() is target-only by design (replay drives the target),
    that the optimizer only appears in replay via the rare model-judge fallback
    (rule/exact/answer tasks are scored locally, 0 tokens), and that the aggregate
    tokens_used() still counts both sub-backends so the total is not undercounted.
    
    * fix(sleep): call-local accounting for attempt_with_tools + refresh redaction test
    
    - Set the thread-local delta in every attempt_with_tools override that charged
      _tokens directly (Claude CLI, OpenCode, Codex, Cursor), so replay_one() reads
      real call-local usage instead of falling back to a response-length estimate.
    - Update the stale test_redact_deep_loses_mapping_key_context to assert redaction
      (it was asserting the old leak bug).
    - Add tool-replay regressions: attempt_with_tools sets call-local delta, and the
      dual-backend path surfaces the target's delta.
    
    * refactor(sleep): centralize token accounting in _record_cost
    
    - Add CliBackend._record_cost(prompt, response) as the single path to charge an
      inference's token cost (aggregate _tokens under _lock + call-local
      _thread_local.delta), so no path under- or over-counts.
    - Route _cached_call miss, all attempt_with_tools overrides
      (Claude/OpenCode/Codex/Cursor), and reflect through it; the OpenCode error
      path keeps its prompt-only charge.
    - No behavior change: identical delta model, just centralized — removing the
      duplicated len//4 accounting that caused the microsoft#251-class bugs to recur.
    
    * fix(sleep): reset call-local delta on no-call paths + one locked accounting helper
    
    Address the review: all token charging now routes through one locked helper
    (_record_cost), and the call-local delta is reset on every no-call / early-return
    path (_reset_call_delta), so a reused worker never reports a previous call's
    token count — covering the OpenCode disabled-tool-replay early return, cache
    hits, and the start of every tool-aware path.
    
    Add regressions: barrier-forced concurrent _record_cost (no lost updates) and
    same-thread prior-call -> disabled-tool-replay delta reset.
    
    * fix(sleep): record call-local delta on real-usage (Azure/OpenCode) paths
    
    - Add _record_delta(delta) and route the Azure/OpenCode real-usage accounting
      through it, so those backends also set the call-local _thread_local.delta —
      the last accounting path that did not (replay_one() saw 0/stale and fell back
      to a length estimate for these backends).
    
    * fix(sleep): single-owner token accounting, no double charge
    
    A backend that self-reports provider usage (AzureOpenAI/AzureResponses) must not
    be charged twice: _cached_call/_reflect now skip the len//4 estimate when the
    call already recorded its own exact usage (via a _charged_in_call marker), so a
    30-token provider usage is charged once as 30, not as the ~110 estimate.
    
    - AzureOpenAI/AzureResponses _call accumulate usage across every paid attempt and
      record _record_delta(total) once; the call-local delta equals that increment.
    - OpenCodeCliBackend.attempt_with_tools error path routes prompt-only cost through
      _record_delta (no manual _tokens / _thread_local.delta update).
    - The _call str return contract is preserved, so llm_miner/rollout/slow_update
      keep working and their Azure cost stays accounted.
    - Added fake-client regressions for both Azure backends (single call, empty-retry
      + success accumulation, cache-hit no-charge) + OpenCode error-path.
    
    * refactor(sleep): thread-localize the self-charge marker
    Move _charged_in_call from a shared instance attribute to _thread_local.charged_in_call
    so parallel workers can never read another worker's marker; the marker already gets
    reset to False before every _call on the current thread.
    
    * fix(sleep): finalize accumulated usage on every exhausted-retry exit
    
    AzureOpenAI._call recorded usage_total only when last_exc was 'empty-response'; if
    the final attempt raised (after a paid empty response), it returned without
    _record_delta, so _cached_call substituted its len//4 estimate and dropped the
    known paid usage. Now always record the accumulated usage + set the marker on any
    exhausted-retry exit (usage_total 0 = genuinely no paid usage). Regression:
    empty-paid-response -> terminal-error keeps the exact 7 tokens.
    
    * fix(sleep): backward-compatible call-local accounting for tokens_used()-only backends
    
    replay_one no longer substitutes a text-length estimate when a backend implements
    only the older tokens_used() contract (no token_delta): it reports the same-thread
    before/after total difference instead, and DualBackend does the same for a target
    that lacks token_delta (snapshotted in attempt/attempt_with_tools). CliBackend and
    DualBackend-with-modern-target keep the thread-safe per-call delta, so the shared
    total is still never reduced by subtraction for concurrent CliBackend calls.
    Added existing-backend regressions (direct + DualBackend, tokens_used-only).
    
    * refactor(sleep): drop unused _cache_pop; note DualBackend snapshot not thread-safe
    
    * fix(sleep): report exact zero cost for cached replay results
    
    A replay served from the backend cache has no new model call: _cached_call() resets the per-thread delta to zero and the aggregate ledger is unchanged. replay_one() was still treating any zero as 'backend does not track tokens' and substituting a text-length estimate, so ReplayResult.tokens disagreed with the backend ledger (102 vs 0 in reproduction). With token_delta() present, the returned cost is authoritative - including a known zero cache hit. Only backends without the per-call interface fall back to before/after totals and the length estimate.
    
    * docs(sleep): spell out the parallel-replay accounting contract
    
    The cumulative-only fallback in DualBackend.token_delta() is sequential-only:
    `_target_tokens_before` is shared instance state, so two workers overlapping
    in one DualBackend overwrite each other's baseline. That is not the same
    guarantee as a target that reports a thread-local per-call delta, and it is not
    the same as a bare legacy backend (whose before/after snapshot replay_one takes
    on the worker's own thread).
    
    replay_batch()'s docstring now lists the four supported combinations, the
    DualBackend constructor note and its token_delta() fallback say which branch is
    sequential-only, and replay_one()'s comment no longer implies that any
    token_delta()-bearing backend is thread-safe.
    
    Tests: a regression for the modern path - replay_batch(workers=4) through a
    DualBackend over a target with a thread-local delta must produce exactly the
    same per-task costs as the sequential run, with the tasks spread far enough
    apart in size that the check is not vacuous.
    
    * docs(sleep): correct the parallel-replay claim for cumulative-only backends
    
    The first version of this contract filed a bare backend with only the older
    tokens_used() contract under "safe to share". It is not: replay_one differences
    a counter that the other workers are also spending, so an overlapping worker's
    tokens land in this task. Two workers against a backend where every attempt
    costs 37 report [74, 37].
    
    Only a thread-local per-call delta is safe to share; every cumulative-total
    shape over-counts when workers overlap. The docstring now says that and names
    the failure direction - budgets see more spend than actually happened - rather
    than implying that legacy backends are fine.
    
    The parallel regression did not bite either: with no real overlap, a backend
    that had lost its thread-locality still produced costs identical to the
    sequential run. The fake target now blocks on a barrier until every worker is
    in flight, so the assertion fails when that locality goes away. Verified by
    substituting a cumulative delta (test fails) and by five consecutive runs with
    the real one (test passes).
    
    * fix(sleep): 区分用量缺失与权威零,避免误记为 0
    
    provider 的 usage 为 None 时旧代码仍把 charged_in_call 置为 True,
    _cached_call 因此跳过 len//4 回退,聚合用量与 call-local 增量双双归零,
    一次真实调用被记成免费。现在把是否上报用量与上报的数值分开判断:
    usage 缺失落到长度估算,上报 0 仍按权威零计费,多轮重试中已知的部分
    用量继续保留。两个 backend(chat completions 与 responses)各自补齐
    缺失、权威零、部分重试三类回归。
    
    * fix(sleep): keep unknown usage distinct from zero on exhausted Azure retries
    
    A run of empty responses that all omit usage metadata exhausted the retry
    budget without any attempt reporting usage; the exit charged usage_total (0)
    and set charged_in_call, so unknown usage was recorded as an authoritative
    zero. Charge the accumulated provider usage only when at least one attempt
    actually reported it; otherwise leave the marker clear so _cached_call's
    len//4 estimate stands in for the unknown cost, as it does on main.
    
    Applies to AzureOpenAIBackend (Chat Completions) and AzureResponsesBackend
    (Responses); a reported zero still stays an authoritative zero, known partial
    retry usage is preserved, and cache hits remain unchanged.
    
    Regressions: exhausted-empty-without-usage and exhausted-empty-with-reported-
    zero for both API variants. Also drops an unused local in the OpenCode error
    path and tidies the new test files' imports (no behavior change).
    
    ---------
    
    Co-authored-by: WODE25500 <[email protected]>
    Co-authored-by: WODE25500 <[email protected]>
    3 people authored Oct 17, 2026
    Configuration menu
    Copy the full SHA
    50d9d90 View commit details
    Browse the repository at this point in the history
  2. fix(sleep): preserve config limits across encoding upgrades (microsof…

    …t#304)
    
    * fix(sleep): read config files as UTF-8
    
    * fix(sleep): preserve legacy config limits and reject unreadable settings
    KennyMcSimpson authored Oct 17, 2026
    Configuration menu
    Copy the full SHA
    ea6f380 View commit details
    Browse the repository at this point in the history
  3. fix(dsh): support @deepseek-ai/dsh-tools 0.2.x (microsoft#311)

    `plugins/dsh` declared `@deepseek-ai/dsh-tools` only as an optional peer at
    `^0.1.0-rc.8` while importing it directly at runtime. That range admits neither
    `0.1.1-rc.2` nor `0.2.0-rc.2` (npm does not match prereleases of a tuple the
    range never names), so under the current harness the plugin's requirement
    conflicts with the tool runtime the harness installs: @deepseek-ai/dsh
    0.2.0-rc.2 ships dsh-tools 0.2.0-rc.2.
    
    Declare it as a direct dependency with the range this plugin is tested
    against — `>=0.1.1-rc.2 <0.2.0 || >=0.2.0-rc.1 <0.3.0` — keeping the optional
    `cordis` peer, and align the package version and README prerequisites with the
    standalone dsh-skillopt package. No source change: `defineTool` is identical in
    both lines and the packed canary (40 checks) passes against dsh-tools
    0.2.0-rc.2.
    
    Co-authored-by: WODE25500 <[email protected]>
    WODE25500 and WODE25500 authored Oct 17, 2026
    Configuration menu
    Copy the full SHA
    c03c88c View commit details
    Browse the repository at this point in the history
  4. fix(envs): confine benchmark task output paths (microsoft#309)

    Co-authored-by: adongwanai <[email protected]>
    adongwanai and adongwanai authored Oct 17, 2026
    Configuration menu
    Copy the full SHA
    2d33785 View commit details
    Browse the repository at this point in the history
  5. fix(sleep): keep train and val splits non-empty (microsoft#272)

    * fix(sleep): keep train and val splits non-empty
    
    * fix: retain task exposure across nightly split reassignment
    kaluli123123 authored Oct 17, 2026
    Configuration menu
    Copy the full SHA
    ae2f766 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2026

  1. fix(sleep): read and write state.json as UTF-8 (microsoft#302)

    * fix(sleep): read and write state.json as UTF-8
    
    * fix(sleep): migrate locale-encoded state.json instead of discarding it
    
    The previous revision read and wrote state.json as UTF-8. That is right for
    new files, but not for existing ones: a state file written by an earlier
    release on a GBK or cp1252 box holds those bytes, so the UTF-8 read raised and
    the bare `except Exception: pass` turned the failure into a fresh state. The
    next save() then wrote over the original, and the night counter, the harvest
    cursors, the history and the cross-night memory were gone with no warning.
    
    Read the bytes and decode as UTF-8 first; on a UnicodeDecodeError fall back to
    the locale codec -- the one the old writer used on the machine that produced
    the file -- and let the next save() rewrite it as UTF-8, so the migration
    completes on the first write.
    
    A file that decodes under neither is now reported as StateFileError rather
    than replaced. The message names the path and says the file was left
    untouched. Continuing to return a fresh state is what makes the loss silent,
    because a reset state is indistinguishable from a first run.
    
    Tests: a legacy GBK fixture carrying an unknown field is loaded, saved and
    read back as UTF-8 with every field intact; an unreadable file raises while
    its bytes stay on disk. Both fail on the previous revision -- verified by
    running them against it -- and the locale codec is injected so the migration
    test is meaningful on a UTF-8 runner rather than passing by accident.
    
    test_sleep_state.py: 5 passed. tests/test_model_change_warning.py and
    tests/test_split_hardening_2x3.py are unchanged (4 failures there are a
    missing optional `llm_client` import, present before this change too).
    
    * fix: reject a state file whose JSON body is not an object
    
    * test: cover non-object state bodies
    
    ---------
    
    Co-authored-by: PerryLink <[email protected]>
    PerryLink and PerryLink authored Oct 18, 2026
    Configuration menu
    Copy the full SHA
    618d5d2 View commit details
    Browse the repository at this point in the history
  2. fix(sleep): enforce max_tokens_per_night in the nightly cycle (micros…

    …oft#317)
    
    * fix(sleep): enforce max_tokens_per_night in the nightly cycle
    
    The config default and the Budget controller existed, but nothing in the
    engine read the key or constructed a Budget, so a night ran every step no
    matter how many tokens it had already used.
    
    Build a Budget from max_tokens_per_night at the start of each cycle and
    check it, against the backend's token counter, before each step that would
    start new model work: the consolidation, its skill and memory edit steps,
    the held-out test replay and every per-skill fan-out group. Steps that never
    started are listed in a report note, in budget skip evidence events and in
    the cycle end event. A started step always finishes, so a gate decision
    never rests on partial evidence and nothing is cut off mid-write.
    
    A handoff round counts unanswered prompts and placeholder replies, so when
    the budget is reached while prompts are still pending the round pauses for
    answers instead of stopping. Otherwise it would stage and save a night built
    from placeholders. If the budget is already spent before consolidation, the
    night records no held-out scores instead of zeros.
    
    0 or null keeps the night unbudgeted, matching Budget's existing semantics.
    Negative, boolean, fractional, infinite or non-numeric values fail before
    any model call. Only tonight's spend counts, so a backend reused across
    nights is not stopped early.
    
    Fixes microsoft#313
    
    * fix(sleep): skip the final validation replay once the budget is spent
    
    The budget check assumed the final validation replay was free because it
    re-scores documents an earlier replay already scored. That holds for cached
    text replies, but tool replay is never cached and a failed or timed out call
    is not kept in the cache, so on those routes the final replay started new
    provider calls after both edit steps had been skipped.
    
    Check the budget before the final validation replay as well. When it is
    spent, decide from the validation replay that already scored the final skill
    and memory (the baseline replay, or the trial replay of the last kept edit)
    instead of starting a new one, and list the skipped replay in the report
    note and the budget skip events. Nights that never reach the budget still
    run the fresh final replay as before.
    
    The new tests drive ClaudeCliBackend through its real tool and text routes
    with a synthetic process standing in for the claude CLI and count every
    process it starts, for answered and timed out calls.
    Thanatos9404 authored Oct 18, 2026
    Configuration menu
    Copy the full SHA
    1a44050 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2026

  1. fix(dsh): call ShellExecutor.execute() on dsh 0.2.x, keep run() for 0…

    ….1.x (microsoft#330)
    
    dsh 0.2.x replaced ShellExecutor.run() with execute(), which returns a
    ShellExecution whose foreground outcome is handle.result(); 0.1.x has only
    run(). plugins/dsh declares @deepseek-ai/dsh-tools for BOTH generations
    (>=0.1.1-rc.2 <0.2.0 || >=0.2.0-rc.1 <0.3.0), so under 0.2.x every
    skillopt_* tool call failed with "shell.run is not a function".
    
    Feature-detect the seam instead of switching it: execute() wins when present,
    run() stays the 0.1.x path. The test covers all three shapes (0.2.x handle,
    0.1.x direct, transitional both-present) and dies on the pre-fix body with
    the same production error.
    
    Co-authored-by: WODE25500 <[email protected]>
    WODE25500 and WODE25500 authored Oct 23, 2026
    Configuration menu
    Copy the full SHA
    f550b9f View commit details
    Browse the repository at this point in the history
  2. Exclude SkillOpt-generated Codex replay sessions from harvest (micros…

    …oft#287)
    
    * Exclude Codex self-replay sessions from harvest
    
    * fix: mark Codex engine sessions with explicit provenance
    kaluli123123 authored Oct 23, 2026
    Configuration menu
    Copy the full SHA
    deeadaa View commit details
    Browse the repository at this point in the history
  3. fix(sleep): key the harvest checkpoint by project and transcript sour…

    …ce (microsoft#322)
    
    `skillopt-sleep run` stored one harvest cursor per project, whatever
    `--source` the run used. When one project is fed from several transcript
    sources, every source after the first only saw sessions newer than the
    first run's start: its older, never-harvested sessions were skipped for
    good. Each source is a separate store with its own timestamps, so a
    single shared cursor cannot be right for all of them.
    
    - state: `last_harvest_for`/`set_last_harvest` take the transcript
      source and key the cursor `<project>::<source>`. A bare `<project>`
      key written before this change is still read as the fallback (so an
      upgrade does not re-scan the whole history) and is never overwritten,
      because it is the only cursor older state files have.
    - cycle/__main__: read and advance the cursor with the run's
      `transcript_source`, in both the no-tasks branch and the normal end,
      including the handoff path.
    - docs: state that the checkpoint is per project and source.
    SU-AN-coder authored Oct 23, 2026
    Configuration menu
    Copy the full SHA
    84b36cd View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2026

  1. feat: add an offline reproducibility checker for saved evaluation art…

    …ifacts (microsoft#329)
    
    * feat: add offline reproducibility checker
    
    * fix: validate available reproducibility artifacts independently
    JianhaoYu-AI authored Oct 25, 2026
    Configuration menu
    Copy the full SHA
    0be3ae9 View commit details
    Browse the repository at this point in the history
  2. fix(sleep): harvest pasted briefs instead of dropping them as host no…

    …ise (microsoft#321)
    
    * fix(sleep): harvest pasted briefs instead of dropping them as host noise
    
    A user message that arrives wrapped in angle brackets was classified as
    system noise, so a brief pasted into the Claude Code desktop app
    (`<pasted_content id="…">…</pasted_content id="…">`) never became a task:
    the session was mined from a short follow-up instead, or dropped entirely
    when the brief was its only user turn.
    
    - unwrap `<pasted_content …>` and keep the inner text as the user's
    - drop host-injected `<system-reminder>` blocks from the intent rather
      than copying them into it
    - keep the "message is nothing but one host tag block" filter, but only
      for a closed list of host tags (command / local-command / system-reminder
      / bash / IDE / Copilot `environment_info`), so a wrapper this list does
      not know about loses its tags rather than the session's request
    - treat a session whose user turns were all filtered as engine-generated
      only when it has no assistant work
    
    Fixes microsoft#318
    
    * fix(sleep): strip only the host's leading notes, keep no-intent records out
    
    Two boundary corrections from review.
    
    * Host stripping no longer rewrites text the user wrote. `_strip_host_wrappers`
      removed every `<system-reminder>…</system-reminder>` block in the message, so
      a prompt that quoted one — inline code, a fenced XML example — lost it, and
      both the harvested request and its mined intent changed. Removal is now
      anchored: the block is dropped only while it leads the message, which is the
      form the host prepends. A quoted sample anywhere after the user's own text
      survives verbatim. The tag match now also tolerates attributes.
    
    * A transcript with no user turns is a replay again, whatever assistant work it
      holds. The zero-user/assistant-present exception admitted host-only isMeta
      records before the session cap; such a record mines nothing, so under
      `--max-sessions` it displaced a real session and left the night with no
      tasks. Reverted to the base behaviour. The pasted-brief fix does not need the
      exception: unwrapping now yields a user turn, which is what the new tests
      assert.
    
    Measured with the new tests against the pre-review head `d97592c`: 3 failed,
    136 passed, 1 skipped —
    
      test_inline_code_system_reminder_example_is_user_text
          ['How ... escape `` inside my transcript fixture?'] != [the prompt]
      test_fenced_system_reminder_example_is_user_text
          the reminder block was deleted from inside the fence
      test_host_only_transcript_does_not_take_a_session_slot
          ['host-note'] != ['real-task']   (limit=1, host-only record newer)
    
    After: 139 passed, 1 skipped, 12 subtests. Full suite: 1,809 passed, 11
    skipped, 359 subtests.
    
    * fix(sleep): unwrap only a leading pasted block, keep literal tags in a request
    
    The wrapper is the host's while it leads the message, not wherever the tag
    pair appears. A request that asks for a parser test on
    `<pasted_content>LITERAL_XML_VALUE</pasted_content>` — inline code, a fenced
    XML example, a fixture carrying attributes — lost its tags in
    digest_transcript(), harvest() and the mined intent. Unwrapping now follows
    the leading-only rule the system-reminder block already uses.
    
    Review on f5f1f9a: four literal cases failed there and pass here; pasted-only,
    pasted-then-follow-up and the Copilot privacy cases still pass.
    SU-AN-coder authored Oct 25, 2026
    Configuration menu
    Copy the full SHA
    ede409b View commit details
    Browse the repository at this point in the history
  3. fix(sleep): stop a re-run hook template from filling the session cap (m…

    …icrosoft#323)
    
    * fix(sleep): stop a re-run hook template from filling the session cap (microsoft#319)
    
    A local hook that summarises the previous session on a timer writes one
    single-turn transcript per run. Its only user prompt is the hook's own
    template: identical leading text, a fresh log appended. Neither the engine
    replay markers nor the sub-agent markers match it, and in the desktop app
    those runs carry `entrypoint: "claude-desktop"` like interactive sessions, so
    it was harvested as a user task. Because harvest walks newest first and stops
    at `--max-sessions`, an hourly hook filled the cap and pushed real sessions out
    of the night entirely.
    
    `harvest()` now drops single-turn sessions whose first 120 characters of
    prompt recur across at least three sessions of the same harvest window, before
    the cap is applied. The whole window is scanned first: a template can only be
    recognized by comparing it with the rest of the window, and the repeats must
    not be counted against the cap they used to fill. A session with a follow-up
    user turn is never matched, and neither is a prompt shorter than the prefix —
    a real one-shot request repeats entirely instead of sharing a long boilerplate
    header. `SKILLOPT_SLEEP_REPEAT_MIN_SESSIONS` tunes the threshold (default 3)
    and `0` disables the filter.
    
    `harvest()` takes an optional `stats` out-parameter that `harvest` and the
    `run` / `dry-run` progress line report, so a thin night says why:
    `harvest done: sessions=120 (skipped: 297 machine-generated, 2 past the session cap)`.
    
    The harvest filters that already existed were undocumented — no .md file named
    `SKILLOPT_SLEEP_AGENT_MARKERS`, `SKILLOPT_SLEEP_NEG_FEEDBACK` or
    `SKILLOPT_SLEEP_POS_FEEDBACK`. They are now documented in the sleep guide and
    the Claude Code plugin README, with the hourly-summary-hook example.
    
    * fix(sleep): make the repeat filter opt-in and keep the cap bounded
    
    Two corrections from review, both about the default path rather than the
    mechanism:
    
    * Recurrence is not evidence of who wrote a prompt. Three human code-review
      requests that share one long checklist prefix and differ in the task payload
      are structurally identical to a hook re-running itself: one single-turn
      session each, >120 shared leading characters, a different tail. Dropping
      those loses real user tasks, which is worse than harvesting a hook's summary
      sessions, so the filter is now off unless a project opts in with
      SKILLOPT_SLEEP_REPEAT_MIN_SESSIONS (3 documented as the usual value; 0
      disables an exported value again). The docstring and both READMEs say why,
      and name the counter-example.
    
    * With the filter off, harvest keeps its single bounded pass: it digests
      newest-first and stops at `limit`, so `--max-sessions 1` reads one transcript
      instead of the whole eligible window. The full-window scan now happens only
      when the filter is opted into, which is the mode that needs it.
    
    Counters no longer claim a producer: `repeated_template_sessions` /
    "repeated-template sessions" replaces `machine_generated_repeats` /
    "machine-generated", in the stats dict and in the progress note.
    
    Measured on the pre-change code with the new tests: 9 failed, 6 passed —
    including the shared-checklist case (3 requests dropped) and the bounded-cap
    case (40 transcripts read for `limit=1`). After: 15 passed. Full suite: 1,846
    passed, 11 skipped, 368 subtests.
    SU-AN-coder authored Oct 25, 2026
    Configuration menu
    Copy the full SHA
    b494019 View commit details
    Browse the repository at this point in the history
  4. test(sleep): document preference-scope evidence loss during deduplica…

    …tion (microsoft#331)
    
    * test(sleep): capture paired-session scope evidence contracts
    
    * docs(test): describe bounded preference-scope matrix
    GBX-Max1220 authored Oct 25, 2026
    Configuration menu
    Copy the full SHA
    5783cce View commit details
    Browse the repository at this point in the history
  5. Add keyless Parallel search for OfficeQA rollouts (microsoft#324)

    * Add anonymous Parallel MCP search to OfficeQA rollouts
    
    * Preserve existing custom search endpoint selection
    
    * Make Parallel search an explicit opt-in with an optional MCP dependency
    
    Addresses review feedback on the default selection and the core dependency.
    
    Restore the offline default: configs/officeqa/default.yaml and
    OfficeQAAdapter are byte-identical to main again, and rollout no longer
    resolves an `auto` mode. `auto` (like any unrecognized value) normalizes to
    `offline`, so a fresh configuration behaves exactly as it does on main.
    Parallel is reached only through an explicit `env.search_mode=parallel`.
    
    Move `mcp` out of core dependencies into a `parallel` extra and import the
    Parallel client lazily, so OfficeQA imports and offline installs no longer
    require it. Selecting the mode without the extra raises an error naming
    `pip install 'skillopt[parallel]'`. CI installs `.[dev,parallel]` so the
    provider stays covered.
    
    Document the data boundary in docs/reference/config.md and the changelog:
    selecting this mode sends the generated queries and the full task question
    (as the MCP `objective`) to Parallel's hosted endpoint. Describe the keyless
    tier as an anonymous, lower-rate free tier rather than unlimited search, and
    keep the note that this evidence is not directly comparable to the paper's
    Google-backed setting.
    
    Tests cover that a fresh `auto` config does not select Parallel, that
    explicit selection does, that the OfficeQA env imports with `mcp` absent,
    and that selecting Parallel without the extra reports the install command.
    MCP transport tests skip when the extra is not installed.
    
    Co-authored-by: George Pickett <[email protected]>
    
    * Raise the built-in TimeoutError when the MCP deadline fires
    
    The whole-request deadline fired on every supported version, but callers saw
    two different exception types. `asyncio.wait_for` raises
    `asyncio.exceptions.TimeoutError`, which only became an alias of the built-in
    `TimeoutError` in 3.11, so the strict deadline assertion failed the CI Python
    3.10 job while 3.11 and 3.12 passed.
    
    Normalize in the provider instead of loosening the assertion: inside the
    existing task-group unwrapping, re-raise an exceeded deadline as the built-in
    `TimeoutError` with the original timeout error as its cause. This covers both
    a direct `wait_for` timeout and one surfaced from an SDK task group, and
    leaves the deadline itself and the cancellation-cleanup path untouched.
    
    ---------
    
    Co-authored-by: George Pickett <[email protected]>
    Co-authored-by: Cursor Agent <[email protected]>
    Co-authored-by: George Pickett <[email protected]>
    4 people authored Oct 25, 2026
    Configuration menu
    Copy the full SHA
    69337c9 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    e2edfa9 View commit details
    Browse the repository at this point in the history
Loading