Exclude SkillOpt-generated Codex replay sessions from harvest - #287

Thank you for tackling the self-harvest feedback loop; preventing generated sessions from becoming user evidence is important.

Reviewed 4fabeeea0821.

skillopt_sleep/harvest_codex.py:35: The current Codex backend's ordinary attempt, judge, and reflect prompts are not recognized here. Capturing the actual calls and feeding their prompts through a synthetic archived session leaves all three in harvest: they start Complete the following task for the user., Score how well the response satisfies the rubric, and You are SkillOpt's optimizer.. Only attempt_with_tools is excluded. Please cover the actual emitted prompts (preferably with an explicit engine-origin marker) and add backend-capture regressions before closing #286.

skillopt_sleep/harvest_codex.py:45: These substring checks also discard ordinary user sessions. A first prompt of Please explain the ## CURRENT SKILL section, followed by an assistant response and perfect, thanks, survives main but produces no harvested session on this head; the same happens with an ordinary ## TASK heading. Please use unambiguous engine provenance or a complete generated prompt shape and add preservation coverage for quoted headings/multi-turn sessions.

Addressed the two findings in 59be37e.

The Codex subprocess boundary now prefixes every engine call with an explicit provenance sentinel, covering ordinary attempt/judge/reflect, tool replay, and benchmark/custom system prompts. Harvesting uses that prefix instead of generic heading substrings; legacy tool replay retains its complete-shape check. Ordinary quoted headings and multi-turn user sessions are preserved.

The regression captures the actual subprocess input from all five paths and feeds it into synthetic archived sessions. Against the previous head, these tests fail with 12 assertions (four engine paths retained and eight user-session cases discarded). With this change, the focused tests pass: 6 tests and 13 subtests.

Validation on Python 3.12: full suite 1,501 passed, 11 skipped; strict documentation build and focused Ruff checks passed. All backend responses were mocked; no live model calls were made. The full-suite run used a minimal PATH because this machine's installed Pi executable changes an unrelated path-resolution test. An initial run also hit an unrelated concurrent staging-pointer race; the final run passed. Upstream CI still needs its normal maintainer approval/execution.

Re-reviewed 59be37e50a8f4069c8ecdf55ff6594b135479043. Actual Codex attempt, judge, reflection, mining, tool and custom-template calls carry the producer marker and are excluded when imported through both supported message payload shapes. Ordinary user sessions quoting the old headings or grader text are retained. The production-boundary selection passes 9 tests plus 13 subtests; the standalone merge on the audited baseline passes 1,717 tests with 13 skips, and strict docs pass.

The first batch, including #304, is now merged. Please rebase onto current main (ae2f766ea02c), preserve both documentation sections in docs/sleep/README.md, and rerun the production provenance/ordinary-session controls plus official checks on the new head. #282's CLI-session discovery and genuine-repeat deduplication remain separate; this does not by itself close the broader #286 workflow.

Merged current main 1a44050 into this PR in b8d0bd7; both documentation sections are preserved.

Fresh checks on the merged tree: focused production provenance/ordinary-session controls 38 passed, 1 skipped, 13 subtests; full pytest 1,834 passed, 12 skipped, 381 subtests; strict docs, focused Ruff, and diff checks passed. An actual isolated skillopt-sleep harvest CLI run excluded the marked engine session while retaining the ordinary user session quoting ## CURRENT SKILL (1 session, 1 task). Tests mock backend subprocess responses; no live model calls were made. The full suite used a minimal PATH for the previously documented host Pi-path issue.

The PR description now explicitly limits the issue reference to the provenance subset and no longer auto-closes the broader #286 workflow. #282's discovery/deduplication work remains separate. Upstream CI and maintainer review are still required.