Trust the executor by default: acceptance criteria per step and a final test-and-repair phase - #88

Why this was done

When MandoCode runs a multi-step plan, every step used to be gated by a second model call that judged whether the step was really done. That gate was expensive and unreliable in practice:

• It doubled the model calls per step, which is the slowest and costliest part of a plan run.
• The judge frequently blocked correct work — it would reject a discovery step for "not finding a build system" in a genuinely empty folder, or demand new edits for functionality an earlier step had already delivered.
• When the judge returned malformed output, the run stalled with no usable explanation.

At the same time, the opposite failure was also real: the executor could report success without ever having run anything, so a plan could finish "green" on untested code.

This PR resolves both by moving trust to where the evidence actually is — the executor — and adding one explicit testing phase at the end of the plan that has to produce real, observable results.

What changed

1. Straightforward execution is now the default.
A step completes when the executor finishes without reporting a failure. Per-step checkpoints, targeted repair on failure, and resume-after-crash all still work exactly as before. The old behavior is preserved as an opt-in: /config set strictPlanVerification true.

2. Every proposed step now carries visible acceptance criteria.
The planner emits 2–5 concrete, observable checks per step. These are shown to the user at approval time, handed to the executor as its definition of done, and frozen for the life of the step — so a retry cannot silently move the goalposts. Criteria are persisted in checkpoints, so a resumed plan keeps the exact terms the user approved.

3. New final phase: "Test and repair the finished result."
Implementation plans automatically get one last step that runs the existing tests, adds a small reusable smoke test where coverage is missing, fixes the concrete defects it finds, and re-runs the affected checks. It has to end with an explicit SUCCESS or FAILED. Two guardrails back this up:

• Test commands are tracked by exit code, not by what the model says about them. A failing check stays failed across attempts until that same command actually returns zero — renaming or rewording it does not clear it.
• The executor must separate "I ran this and saw this" from "I did not check this," and must say console not inspected when it never captured browser output. Opening a preview page is explicitly not accepted as proof that a page rendered.

4. Strict mode, for those who opt in, got materially more robust.
The verifier now tries three response formats (tool call → JSON schema → plain JSON) instead of retrying one, rejects truncated or self-contradicting verdicts instead of treating them as a pass, and reports which format failed and why. It also gained one bounded, read-only follow-up: when a step fails only because an observation is missing, the agent may re-read files or re-run a check command it already ran — it cannot edit code, install anything, or expand scope.

5. Better recovery from ambiguous edits.
When a find-and-replace matches multiple places in a file, the error now shows the competing matches with line numbers and surrounding context, instead of the previous generic "provide a larger fragment." This was a common source of the agent retrying the identical failing edit in a loop.

Impact for users

• Plans run roughly twice as fast per step and cost proportionally less, because the per-step judge call is gone by default.
• Fewer false blocks on legitimate work, especially on new/empty projects and on steps that build on earlier steps.
• Plans end with an honest testing report instead of an unverified claim of success.
• Existing config files need no changes; they get the new default automatically.
• Plans already saved to a checkpoint keep their original shape — they do not retroactively gain a final testing phase.

Risk and rollback

Rollback is a config flag, not a revert: /config set strictPlanVerification true restores per-step independent verification. The planner / plannerEngine keys now point users at this flag rather than failing silently.

The main accepted trade-off is stated plainly in the docs: neither mode guarantees correctness. Straightforward mode trusts the executor's own checks, so the value of a plan run depends on the acceptance criteria being meaningful — which is exactly why they are now surfaced to the user before approval.

Testing

Full suite green: 701 passed, 0 failed, on both net8.0 and net10.0.

New test coverage: PlanAcceptanceTests, PlanFinalQualityTests, PlanQualityOutcomeTests, PlanQualityTests, StraightforwardPlanTests, plus extensions to PlanVerificationRecoveryTests.

Docs

• docs/PlannerExecution.md (new) — side-by-side comparison of the two modes and how to switch.
• docs/TaskPlanner.md — rewritten around the default flow; the strict-mode detail moved to the new file.

Related

Pairs with the MandoCode.Desktop PR, which renders the new acceptance checks on the plan review card.