fix(aidd-dev): a review appends a round and scores the whole plan by blafourcade · Pull Request #977 · ai-driven-dev/framework

blafourcade · 2026-10-07T21:56:15Z

🎯 What & why

One phase of a 7-phase plan, one diff, three models: 100% (6/6) on 7 blocks, 100% (6/6) on 1 block, 53% (16/34) where 16/34 is 47%, and boxes copied from the previous run. 05-review never said what a phase outside the diff becomes, how the percent is computed, or where a past round lives.

🛠️ How it works

A review is appended to, one section per round, and never rewritten. A round carries its date, its author from git config user.name, its diff, the axes that ran, a verdict and a score. Its Criteria list holds the plan's criteria, its Findings list the defects no criterion covers.

The score replaces the percent that had no definition: it counts over the plan's whole criteria total, in three numbers that sum to it — 1/6 met, 1 unmet, 4 out of the diff. One denominator, so a plan covered at a third can no longer read as finished. A phase the diff leaves untouched gets one Out of the diff: line instead of a block of empty boxes.

The skill now follows the authoring contract aidd-context:04-skill-generate applies to itself: the router holds the flow, the action table and one transversal rule, and nothing else. 01-prepare resolves the diff once for the three axes and opens the round; 05-finalize judges it and checks its shape. references/report-contract.md holds where the report lives, what diff is reviewed and how a round joins it. SKILL.md, the only always-on text, goes from 446 words to 189.

🧪 How to verify

node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'   # 580 pass, 0 fail
claude -p --plugin-dir "$PWD/plugins/aidd-dev" "Run the aidd-dev:05-review skill, functional axis only, on <plan.md>, for the diff HEAD~1...HEAD."

Guard: 26 assertions, 34 mutations, each reddening one assertion alone.
Volume: 55 runs over 11 fixtures, every fixture unanimous. A plan of 10 phases and 34 criteria whose diff touches 2 phases reads 3/34 met, 5 unmet, 26 out of the diff, 5 runs of 5.
Against next, same fixtures, same prompts, counting checked boxes against the truth: 6 wrong runs of 18 for next, 0 of 45 here. next both over-credits (6 boxes where 5 hold, crediting a stock: "untouched" label) and under-credits (a criterion evidenced by a context line, a not-applicable one).
Callers: aidd-orchestrator:01-sdlc, 00-async-dev and the ship-a-feature recipe name the skill and its findings, never a section or a percent. Nothing downstream reads the artifact's shape.

⚠️ Heads-up

The artifact changes shape. review.md becomes a log of rounds. That is not what fix(aidd-dev): review leaves out-of-diff phases and the verified percent undefined #858 asked for, and the issue names "the verified percent" that the Score line replaces — to settle before merge.
Four rules come from the volume campaign, each turning a split between runs into unanimity: a context line of the diff is evidence; a criterion whose subject is absent is a gap, not not-applicable; a phase projecting no file is traced, not out of the diff; a label echoing the criterion's words is not evidence. They cost 03-review-functional 97 words, and they are the only words in it whose effect is measured.
The skill is 1983 words against 1785 on next. The always-on router shrank by 58%; the total grew because 01-prepare and 05-finalize own work that was implicit, and because the contract requires a | Case | Pass | table per action.
Cross-model is unresolved. On haiku, both designs produce a report about half the time and neither is clearly better on the landed runs. haiku did expose a real defect, now fixed: nothing in the skill named review.md any more, and a reference named review-report.md sat one hyphen away, so haiku wrote its report there.
references/review-rubric.md keeps 🟡 warning in a scale whose neighbours are critical and minor. major would be consistent; it touches 04-audit, 03-assert and 07-refactor, so it belongs to its own change.

🔗 Linked issue

Refs #858

✅ I certify

I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z

One phase of a 7-phase plan, one diff, three models: `100% (6/6)` on 7 blocks, `100% (6/6)` on 1 block, `53% (16/34)` where 16/34 is 47%, and boxes copied from the previous run. The skill never said what a phase outside the diff becomes, how the percent is computed, or where a past round lives. A review now appends one section per round and is never rewritten. A round carries its date, its author from `git config user.name`, its diff, the axes that ran, a verdict and a score; its `Criteria` list holds the plan's criteria and its `Findings` list the defects no criterion covers. The score counts over the plan's whole criteria total, in three numbers that sum to it, so a plan covered at a third can no longer read as finished. The skill follows `skill-generate`'s authoring contract: the router holds the flow, the five actions and two transversal rules, and nothing else. `01-prepare` resolves the diff once for the three axes and opens the round; `05-finalize` judges it and checks its shape. `references/report-contract.md` holds where the report lives, what diff is reviewed and how a round joins it. An action says what to decide; the template owns the names and the form, the validator the field set. The skill reads 1583 words, against 1785 on `next`. Severity reads `critical`, `major`, `minor`: one scale, where `warning` put a log level between two gravities. Three files outside this skill still say `warning`. A critical is the harm a merge would do to production, its users, their data or what depends on it, never how strictly a rule is worded. Before, a broken declared rule read critical in 7 of 12 runs and a crash in production blocked in 3 of 5; now 0 of 10 and 5 of 5, over 50 runs on ten defect fixtures with every verdict unanimous. Measured, each with the mutation that proves it: 31 assertions, 39 mutations, each reddening one assertion alone. The assertions spell a path the way the documents spell it, so Windows and macOS read the same tree. 71 real runs on the final text: 50 of 50 functional runs over 10 fixtures scored right, including 10 phases and 34 criteria reading `3/34 met, 5 unmet, 26 out of the diff`; 3 of 3 runs without a plan read `not scored`; 33 of 33 planted defects found across the three axes; no run patched a file. Against `next` on the same fixtures, counting checked boxes against the truth: 6 wrong runs of 18. Four rules come from those campaigns, each turning a split between runs into unanimity: a context line of the diff is evidence, a criterion whose subject is absent is a gap rather than `not-applicable`, a phase projecting no file is traced rather than out of the diff, and a label echoing the criterion's words is not evidence. Refs #858 Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z AIDD-Session-Id: bf713ca4-3c1b-491d-ac0a-4d4533330b5c

#995) On `next`, a criterion only a run could show was checked from the code alone: "submitting the form displays a success banner" came out `[x]` citing `src/signup.js:3` in 9 runs of 9, including one where a recorded run had seen that banner fail. A review stays static. A criterion whose subject is in the diff but whose outcome only a run could show, what a screen displays, a timing, an outside system's answer, is now unverified at runtime: named in the round with what a run would show, never checked from the code, and never blocking the verdict. A run recorded beside the plan on the reviewed head decides it instead: passing checks it, citing the record; failing leaves it unchecked; a record from another head leaves it unverified. The score gains a fourth number, so met, unmet, unverified and out of the diff sum to the plan's whole criteria total. This replaces #977's "a not-applicable criterion counts as met": such a criterion is now unverified, and `not-applicable` is gone from the skill. Measured with `claude -p` against the plugin from this tree, 103 runs: - 85 functional runs: every fixture scored as expected, including six traps: a record naming a branch, a working-tree edit after the run, a stale record covering a criterion the code proves, a blocked run, a mixed criterion, and a backend criterion that must stay checked. - 18 all-axes runs: verdicts unchanged (approve, blocked, then changes-requested), no unverified line on criteria the code proves. A round whose every criterion is unverified approves with `0/N met`; its score says so. Refs #976 Claude-Session: https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z Co-authored-by: Claude Opus 5.5 <[email protected]>

…tor (#997) The review grades findings critical, major, minor since #977, but assert's architecture check, the audit template and refactor still named the middle level `warning`. An audit feeds refactor and the review grades the same code, so one severity carried two names depending on the skill that wrote it. `warning` becomes `major` in the three skills, with the audit's definition unchanged. A guard fails on any `warning` left in an aidd-dev skill, and one per skill pins its scale; each goes red under its own mutation. Measured with `claude -p` against the plugin from this tree: three code-quality audits counted `0 critical, 1 major, 3 minor`, and two architecture checks rated their violations `major`. No run wrote `warning`. Claude-Session: https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z Co-authored-by: Claude Opus 5.5 <[email protected]>

blafourcade force-pushed the fix/review-phase-scope branch from d639634 to 72ef2e1 Compare October 8, 2026 04:19

blafourcade changed the title ~~fix(aidd-dev): a review says what happens outside the diff and how the percent is computed~~ fix(aidd-dev): a review keeps its rounds, its phase scope and a computable percent Oct 8, 2026

blafourcade mentioned this pull request Oct 8, 2026

fix(aidd-dev): review leaves out-of-diff phases and the verified percent undefined #858

Closed

10 tasks

blafourcade force-pushed the fix/review-phase-scope branch 2 times, most recently from c9c182f to 5157028 Compare October 9, 2026 08:25

blafourcade changed the title ~~fix(aidd-dev): a review keeps its rounds, its phase scope and a computable percent~~ fix(aidd-dev): a review appends a round and scores the whole plan Oct 9, 2026

github-advanced-security Bot found potential problems Oct 9, 2026

View reviewed changes

Comment thread scripts/__tests__/a-review-appends-a-round-and-scores-the-whole-plan.test.js Fixed

blafourcade force-pushed the fix/review-phase-scope branch 4 times, most recently from 1f054fa to 8f3ebda Compare October 10, 2026 12:12

blafourcade force-pushed the fix/review-phase-scope branch from 8f3ebda to 390fa30 Compare October 10, 2026 18:45

blafourcade marked this pull request as ready for review October 10, 2026 21:36

blafourcade requested a review from a team as a code owner October 10, 2026 21:36

blafourcade merged commit b25ae36 into next Oct 10, 2026
38 of 40 checks passed

blafourcade mentioned this pull request Oct 11, 2026

fix(aidd-dev): a review never checks a runtime criterion from the code #995

Merged

1 task

blafourcade mentioned this pull request Oct 11, 2026

fix(aidd-dev): name one severity scale across assert, audit and refactor #997

Merged

1 task

aidd-bot Bot mentioned this pull request Oct 16, 2026

chore: release main #1039

Merged

Navigation Menu

Repository navigation

fix(aidd-dev): a review appends a round and scores the whole plan - #977

fix(aidd-dev): a review appends a round and scores the whole plan#977
blafourcade merged 1 commit into
nextfrom
fix/review-phase-scope

blafourcade commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

blafourcade commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🎯 What & why

🛠️ How it works

🧪 How to verify

⚠️ Heads-up

🔗 Linked issue

✅ I certify

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

blafourcade commented Oct 7, 2026 •

edited

Loading