Skip to content
This repository was archived by the owner on Jun 25, 2026. It is now read-only.

feat(review): add PR review handoff metadata - #178

Open
rtzll wants to merge 1 commit into
mainfrom
rascal/rtzll/rascal-163-aa349cb78d
Open

feat(review): add PR review handoff metadata#178
rtzll wants to merge 1 commit into
mainfrom
rascal/rtzll/rascal-163-aa349cb78d

Conversation

@rtzll

@rtzll rtzll commented Mar 15, 2026

Copy link
Copy Markdown
Owner

Persist deterministic review handoff artifacts for PR handoff, surface them in
run API output, and keep a managed PR-body section updated with reviewer hints
and risk signals.

Agent Details

    __( O)>  ● resuming · codex gpt-5.4
   \____)    20260314_1 · /work/repo
     L L     goose is ready
=== CODEX PROVIDER DEBUG ===
Command: "/usr/local/bin/codex"
Model: gpt-5.4
Reasoning effort: high
Skip git check: false
Prompt length: 12599 chars
Prompt: You are a general-purpose AI agent called goose, created by Block, the parent company of Square, CashApp, and Tidal.
goose is being developed as an open-source software project.
# Suggestion

The user has 7 extensions with 16 tools enabled, exceeding recommended limits (5 extensions or 50 tools).
Consider asking if they'd like to disable some extensions to improve tool selection accuracy.

# Response Guidelines

Use Markdown formatting for all responses.

Human: <info-msg>
It is currently 2026-03-15 17:47:00
Working directory: /work/repo

Current tasks and notes:
Once given a task, immediately update your todo with all explicit and implicit requirements

</info-msg>
# Rascal Run Instructions

Run ID: run_abc074584e7e4c55
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163

## Task

feat(review): add reviewer routing hints and diff risk signals for Rascal PR handoff

## Summary
Rascal can already create or update PRs and move runs into a `review` state, but the handoff to humans is still thin. It does not help prioritize risky diffs, suggest likely reviewers, or distinguish changes that deserve extra attention.

This issue adds a lightweight review-handoff layer that computes routing hints and risk signals when Rascal opens or updates a PR.

## Outcome
When a run reaches PR handoff, Rascal can attach reviewer suggestions, risk metadata, and review-priority signals so the human review step is faster and better targeted.

## Goals
- Improve the quality of the human handoff without turning Rascal into a noisy review bot.
- Surface high-risk changes explicitly.
- Suggest likely reviewers using repository-local signals.
- Keep everything explainable, inspectable, and GitHub-native.
- Preserve conservative defaults and avoid surprise reviewer spam.

## Non-goals
- Building a full inbox product with calendars, timezones, or org graphs.
- Auto-approving or auto-merging PRs.
- Replacing CODEOWNERS or branch protection.
- Emitting large volumes of AI-generated inline review comments.

## Scope
Implement V1 review-handoff support with the following behavior.

1. Review handoff artifact
- Persist explicit handoff metadata in the run directory, for example:
  - `review-handoff.json`
  - `review-handoff.md`
- Include at least:
  - suggested reviewers
  - risk score or risk level
  - reasons for the risk classification
  - changed path summary
  - notable signals such as tests changed or not changed

2. Reviewer suggestion heuristics
- Suggest reviewers from a small, deterministic set of signals such as:
  - CODEOWNERS matches when available
  - recent file-touch history within the repo
  - previous PR author/reviewer activity for the same area
- Keep the heuristics explainable and stable.
- If no high-confidence suggestion exists, record that explicitly instead of guessing.

3. Risk classification
- Compute a compact risk classification such as `low`, `medium`, `high`.
- V1 signals may include:
  - number of files changed
  - number of directories or packages touched
  - presence of config/deploy/runtime files
  - changes in security-sensitive or infra-sensitive paths
  - missing test deltas for production code changes
  - cross-cutting changes spanning multiple areas
- Keep the scoring heuristic simple and visible in the artifact.

4. GitHub presentation
- Add the handoff summary to the PR body or completion comment in a concise way.
- Optionally add conservative labels such as:
  - `rascal:risk-high`
  - `rascal:needs-review`
- Do not request reviewers automatically in V1 unless explicitly configured.
- Start with suggestions and metadata, not side effects.

5. CLI/API visibility
- Expose handoff metadata through API/JSON output and optionally a CLI view command.
- Operators should be able to inspect why Rascal considered a change risky.

6. Notification hygiene
- Ensure handoff metadata updates are idempotent when Rascal revises an existing PR.
- Avoid posting duplicate comments or churning labels on every follow-up run.

## Suggested implementation approach
1. Add a small review-handoff package.
- Separate reviewer suggestion, risk scoring, artifact rendering, and GitHub presentation.
- Keep the scorer independent from backend-specific execution details.

2. Start with repo-local deterministic signals only.
- Prefer CODEOWNERS and git history over external services.
- If CODEOWNERS parsing is absent today, add a narrow parser sufficient for reviewer discovery.

3. Keep side effects opt-in.
- Suggest reviewers first.
- Automatic reviewer requests can be a follow-up once confidence and ergonomics are proven.

4. Make idempotency explicit.
- Reuse the existing notification-marker pattern or its successor.
- PR body sections or comments should be updateable rather than appended repeatedly.

5. Keep the risk model intentionally small.
- The point is prioritization, not perfect prediction.
- A transparent heuristic is preferable to an opaque score.

## Acceptance criteria
- Rascal produces explicit review-handoff artifacts when a run reaches PR handoff.
- Reviewer suggestions use deterministic repo-local signals and include explanation.
- Rascal assigns a compact risk classification with visible reasons.
- GitHub presentation is concise and idempotent.
- The feature works for newly created PRs and existing PR updates.
- Tests cover CODEOWNERS/history suggestion paths, risk scoring, presentation idempotency, and no-suggestion fallback.

## Suggested test plan
- Unit tests for CODEOWNERS matching and reviewer suggestion ranking.
- Unit tests for risk classification heuristics.
- Integration tests for PR-body or comment updates without duplication.
- Regression tests for follow-up runs that revise the same PR and preserve stable handoff metadata.
- API/CLI tests for handoff artifact visibility.

## Implementation notes
- This should make review easier, not louder.
- Keep reviewer suggestions advisory until the heuristics earn operator trust.
- High-signal handoff metadata is much more valuable here than broad automation.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

Triggered by label 'rascal' on issue #163

# Rascal Run Instructions

Run ID: run_aa349cb78d8c9c4a
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163

## Task

feat(review): add reviewer routing hints and diff risk signals for Rascal PR handoff

## Summary
Rascal can already create or update PRs and move runs into a `review` state, but the handoff to humans is still thin. It does not help prioritize risky diffs, suggest likely reviewers, or distinguish changes that deserve extra attention.

This issue adds a lightweight review-handoff layer that computes routing hints and risk signals when Rascal opens or updates a PR.

## Outcome
When a run reaches PR handoff, Rascal can attach reviewer suggestions, risk metadata, and review-priority signals so the human review step is faster and better targeted.

## Goals
- Improve the quality of the human handoff without turning Rascal into a noisy review bot.
- Surface high-risk changes explicitly.
- Suggest likely reviewers using repository-local signals.
- Keep everything explainable, inspectable, and GitHub-native.
- Preserve conservative defaults and avoid surprise reviewer spam.

## Non-goals
- Building a full inbox product with calendars, timezones, or org graphs.
- Auto-approving or auto-merging PRs.
- Replacing CODEOWNERS or branch protection.
- Emitting large volumes of AI-generated inline review comments.

## Scope
Implement V1 review-handoff support with the following behavior.

1. Review handoff artifact
- Persist explicit handoff metadata in the run directory, for example:
  - `review-handoff.json`
  - `review-handoff.md`
- Include at least:
  - suggested reviewers
  - risk score or risk level
  - reasons for the risk classification
  - changed path summary
  - notable signals such as tests changed or not changed

2. Reviewer suggestion heuristics
- Suggest reviewers from a small, deterministic set of signals such as:
  - CODEOWNERS matches when available
  - recent file-touch history within the repo
  - previous PR author/reviewer activity for the same area
- Keep the heuristics explainable and stable.
- If no high-confidence suggestion exists, record that explicitly instead of guessing.

3. Risk classification
- Compute a compact risk classification such as `low`, `medium`, `high`.
- V1 signals may include:
  - number of files changed
  - number of directories or packages touched
  - presence of config/deploy/runtime files
  - changes in security-sensitive or infra-sensitive paths
  - missing test deltas for production code changes
  - cross-cutting changes spanning multiple areas
- Keep the scoring heuristic simple and visible in the artifact.

4. GitHub presentation
- Add the handoff summary to the PR body or completion comment in a concise way.
- Optionally add conservative labels such as:
  - `rascal:risk-high`
  - `rascal:needs-review`
- Do not request reviewers automatically in V1 unless explicitly configured.
- Start with suggestions and metadata, not side effects.

5. CLI/API visibility
- Expose handoff metadata through API/JSON output and optionally a CLI view command.
- Operators should be able to inspect why Rascal considered a change risky.

6. Notification hygiene
- Ensure handoff metadata updates are idempotent when Rascal revises an existing PR.
- Avoid posting duplicate comments or churning labels on every follow-up run.

## Suggested implementation approach
1. Add a small review-handoff package.
- Separate reviewer suggestion, risk scoring, artifact rendering, and GitHub presentation.
- Keep the scorer independent from backend-specific execution details.

2. Start with repo-local deterministic signals only.
- Prefer CODEOWNERS and git history over external services.
- If CODEOWNERS parsing is absent today, add a narrow parser sufficient for reviewer discovery.

3. Keep side effects opt-in.
- Suggest reviewers first.
- Automatic reviewer requests can be a follow-up once confidence and ergonomics are proven.

4. Make idempotency explicit.
- Reuse the existing notification-marker pattern or its successor.
- PR body sections or comments should be updateable rather than appended repeatedly.

5. Keep the risk model intentionally small.
- The point is prioritization, not perfect prediction.
- A transparent heuristic is preferable to an opaque score.

## Acceptance criteria
- Rascal produces explicit review-handoff artifacts when a run reaches PR handoff.
- Reviewer suggestions use deterministic repo-local signals and include explanation.
- Rascal assigns a compact risk classification with visible reasons.
- GitHub presentation is concise and idempotent.
- The feature works for newly created PRs and existing PR updates.
- Tests cover CODEOWNERS/history suggestion paths, risk scoring, presentation idempotency, and no-suggestion fallback.

## Suggested test plan
- Unit tests for CODEOWNERS matching and reviewer suggestion ranking.
- Unit tests for risk classification heuristics.
- Integration tests for PR-body or comment updates without duplication.
- Regression tests for follow-up runs that revise the same PR and preserve stable handoff metadata.
- API/CLI tests for handoff artifact visibility.

## Implementation notes
- This should make review easier, not louder.
- Keep reviewer suggestions advisory until the heuristics earn operator trust.
- High-signal handoff metadata is much more valuable here than broad automation.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

Triggered by label 'rascal' on issue #163


Assistant: 
Image files: 0
============================
{"type":"message","message":{"id":null,"role":"assistant","created":1773597977,"content":[{"type":"text","text":"I’m tracing the existing PR handoff path first so I can add review metadata where the run already persists artifacts and updates GitHub, then I’ll keep the change set tight around those integration points.\n\nI found the likely seams: PR body/completion rendering in `internal/runsummary`, notification idempotency in `internal/orchestrator/notifications.go`, and run finalization in `internal/worker/worker.go`. I’m reading those now to wire the new handoff artifact into the existing review transition instead of inventing a second path.\n\nThe runner side is simpler than expected: it already owns diff creation, PR body generation, and writes repo-local meta files into the run directory. I’m checking whether I can compute reviewer/risk signals entirely from local git state there, then expose the resulting artifact from the server without adding new persisted DB state.\n\nI have the implementation shape now. I’m going with a small pure `reviewhandoff` package for scoring/rendering, using repo-local git/CODEOWNERS data from the worker to write `review-handoff.json` and `review-handoff.md`, then exposing that JSON on `GET /v1/runs/{id}` and updating a single managed PR-body section in place for idempotency.\n\nI’m starting the code edits now. The main changes are a new `internal/reviewhandoff` package, a worker-side collector/writer for git-derived handoff data, and a small API extension so the stored handoff JSON is inspectable after the run completes.\n\nI’ve got the first pass in place, but there are a few details to tighten before verification: CODEOWNERS team handles need proper normalization, the PR body helper needs a cleaner insert path for the managed section, and I still need the focused tests for existing-PR updates and API visibility.\n\nThe remaining work is mechanical now: formatting, compile/test passes, then any cleanup from failures. After that I’ll run the required `make lint` and `make test`, and write the commit message file.\n\nI’ve finished the targeted tests. Next I’m running the repo’s required validation exactly as requested: `make lint` and `make test`.\n\nBoth `make` targets are still in dependency/download work. I’m waiting them through rather than substituting a narrower check, because the task explicitly asked for the real targets when present.\n\n`make lint` and `make test` are both still inside `sqlc`/tool bootstrap work. They aren’t hung; they’re just duplicating the `codegen` prerequisite in parallel, so I’m letting them finish and then I’ll report the actual target results.\n\nThe parallel `make` run turned out to be counterproductive: both targets are compiling the same large `sqlc` dependency tree at once. I’m collapsing that into a sequential run now so validation finishes instead of burning CPU on duplicated work.\n\n`make lint` found two real issues, both straightforward `wrapcheck` complaints. I’m patching those exact lines and then rerunning lint.\n\nThe implementation is complete and validation is green. I’m writing the required conventional commit message file now, then I’ll summarize the exact user-visible changes and the verification results.\n\nImplemented the review handoff layer end to end.\n\nA new [reviewhandoff package](/work/repo/internal/reviewhandoff/reviewhandoff.go#L13) now computes deterministic reviewer hints from CODEOWNERS and recent git history, assigns a visible `low`/`medium`/`high` risk score with reasons, renders both `review-handoff.json` and `review-handoff.md`, and manages a single PR-body handoff section idempotently. The worker now builds and persists that metadata during PR handoff, includes it on new PR bodies, and updates the same managed section on existing PRs instead of appending duplicates ([worker integration](/work/repo/internal/worker/worker.go#L274), [git/CODEOWNERS collection](/work/repo/internal/worker/review_handoff.go#L13), [PR body support](/work/repo/internal/runsummary/runsummary.go#L114)). I kept reviewer suggestions advisory only; there are no automatic reviewer requests or label side effects in this change.\n\nRun inspection now exposes the handoff metadata through `GET /v1/runs/{id}` via the `review_handoff` JSON field ([API response type](/work/repo/internal/api/types.go#L50), [handler](/work/repo/internal/orchestrator/api.go#L622)). I added focused tests for reviewer heuristics, risk scoring, PR-body idempotency, worker artifact generation/update behavior, and API visibility ([reviewhandoff tests](/work/repo/internal/reviewhandoff/reviewhandoff_test.go), [worker tests](/work/repo/cmd/rascal-runner/main_test.go), [API test](/work/repo/internal/orchestrator/api_reviewhandoff_test.go)). Validation passed with `make lint` and `make test`. I also wrote [/rascal-meta/commit_message.txt](/rascal-meta/commit_message.txt)."}],"metadata":{"userVisible":true,"agentVisible":true}}}
{"type":"complete","total_tokens":10469236}

Closes #163


Rascal run run_aa349cb78d8c9c4a completed in 19m 18s · 10.47M tokens

@rtzll rtzll added the rascal Trigger Rascal automation label Mar 15, 2026
@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

please rebase on origin/main and resolve rebase conflicts then amend and force push

@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

Rascal started run run_583b12d008cdfdb2 to address new PR feedback.

Run Settings
  • Requested by: rtzll

  • Trigger: pr_comment

  • Agent Runtime: goose-codex

  • Runner commit: 5f49c418e514264e0a0ea084807c08e6f890cd97

  • Branches: main -> rascal/rtzll/rascal-163-aa349cb78d

  • Session mode: all

  • Resume: true

  • Debug: true

  • Queue delay: 0s

  • Instruction: Address PR feat(review): add PR review handoff metadata #178 feedback

  • Context: please rebase on origin/main and resolve rebase conflicts then amend and force push

Persist deterministic review handoff artifacts for PR handoff, surface them in
run API output, and keep a managed PR-body section updated with reviewer hints
and risk signals.

Run: run_aa349cb78d8c9c4a
@rtzll
rtzll force-pushed the rascal/rtzll/rascal-163-aa349cb78d branch from 383d733 to 2d78027 Compare March 28, 2026 12:11
@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

@rtzll implemented in commit 2d7802739189.

Rebase onto origin/main and keep the updated runtime/test contracts aligned
with the rebased base branch.

Agent Details

    __( O)>  ● resuming · codex gpt-5.4
   \____)    20260314_1 · /work/repo
     L L     goose is ready
=== CODEX PROVIDER DEBUG ===
Command: "/usr/local/bin/codex"
Model: gpt-5.4
Reasoning effort: high
Skip git check: false
Prompt length: 19077 chars
Prompt: You are a general-purpose AI agent called goose, created by Block, the parent company of Square, CashApp, and Tidal.
goose is being developed as an open-source software project.
# Suggestion

The user has 7 extensions with 16 tools enabled, exceeding recommended limits (5 extensions or 50 tools).
Consider asking if they'd like to disable some extensions to improve tool selection accuracy.

# Response Guidelines

Use Markdown formatting for all responses.

Human: # Rascal Run Instructions

Run ID: run_abc074584e7e4c55
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163

## Task

feat(review): add reviewer routing hints and diff risk signals for Rascal PR handoff

## Summary
Rascal can already create or update PRs and move runs into a `review` state, but the handoff to humans is still thin. It does not help prioritize risky diffs, suggest likely reviewers, or distinguish changes that deserve extra attention.

This issue adds a lightweight review-handoff layer that computes routing hints and risk signals when Rascal opens or updates a PR.

## Outcome
When a run reaches PR handoff, Rascal can attach reviewer suggestions, risk metadata, and review-priority signals so the human review step is faster and better targeted.

## Goals
- Improve the quality of the human handoff without turning Rascal into a noisy review bot.
- Surface high-risk changes explicitly.
- Suggest likely reviewers using repository-local signals.
- Keep everything explainable, inspectable, and GitHub-native.
- Preserve conservative defaults and avoid surprise reviewer spam.

## Non-goals
- Building a full inbox product with calendars, timezones, or org graphs.
- Auto-approving or auto-merging PRs.
- Replacing CODEOWNERS or branch protection.
- Emitting large volumes of AI-generated inline review comments.

## Scope
Implement V1 review-handoff support with the following behavior.

1. Review handoff artifact
- Persist explicit handoff metadata in the run directory, for example:
  - `review-handoff.json`
  - `review-handoff.md`
- Include at least:
  - suggested reviewers
  - risk score or risk level
  - reasons for the risk classification
  - changed path summary
  - notable signals such as tests changed or not changed

2. Reviewer suggestion heuristics
- Suggest reviewers from a small, deterministic set of signals such as:
  - CODEOWNERS matches when available
  - recent file-touch history within the repo
  - previous PR author/reviewer activity for the same area
- Keep the heuristics explainable and stable.
- If no high-confidence suggestion exists, record that explicitly instead of guessing.

3. Risk classification
- Compute a compact risk classification such as `low`, `medium`, `high`.
- V1 signals may include:
  - number of files changed
  - number of directories or packages touched
  - presence of config/deploy/runtime files
  - changes in security-sensitive or infra-sensitive paths
  - missing test deltas for production code changes
  - cross-cutting changes spanning multiple areas
- Keep the scoring heuristic simple and visible in the artifact.

4. GitHub presentation
- Add the handoff summary to the PR body or completion comment in a concise way.
- Optionally add conservative labels such as:
  - `rascal:risk-high`
  - `rascal:needs-review`
- Do not request reviewers automatically in V1 unless explicitly configured.
- Start with suggestions and metadata, not side effects.

5. CLI/API visibility
- Expose handoff metadata through API/JSON output and optionally a CLI view command.
- Operators should be able to inspect why Rascal considered a change risky.

6. Notification hygiene
- Ensure handoff metadata updates are idempotent when Rascal revises an existing PR.
- Avoid posting duplicate comments or churning labels on every follow-up run.

## Suggested implementation approach
1. Add a small review-handoff package.
- Separate reviewer suggestion, risk scoring, artifact rendering, and GitHub presentation.
- Keep the scorer independent from backend-specific execution details.

2. Start with repo-local deterministic signals only.
- Prefer CODEOWNERS and git history over external services.
- If CODEOWNERS parsing is absent today, add a narrow parser sufficient for reviewer discovery.

3. Keep side effects opt-in.
- Suggest reviewers first.
- Automatic reviewer requests can be a follow-up once confidence and ergonomics are proven.

4. Make idempotency explicit.
- Reuse the existing notification-marker pattern or its successor.
- PR body sections or comments should be updateable rather than appended repeatedly.

5. Keep the risk model intentionally small.
- The point is prioritization, not perfect prediction.
- A transparent heuristic is preferable to an opaque score.

## Acceptance criteria
- Rascal produces explicit review-handoff artifacts when a run reaches PR handoff.
- Reviewer suggestions use deterministic repo-local signals and include explanation.
- Rascal assigns a compact risk classification with visible reasons.
- GitHub presentation is concise and idempotent.
- The feature works for newly created PRs and existing PR updates.
- Tests cover CODEOWNERS/history suggestion paths, risk scoring, presentation idempotency, and no-suggestion fallback.

## Suggested test plan
- Unit tests for CODEOWNERS matching and reviewer suggestion ranking.
- Unit tests for risk classification heuristics.
- Integration tests for PR-body or comment updates without duplication.
- Regression tests for follow-up runs that revise the same PR and preserve stable handoff metadata.
- API/CLI tests for handoff artifact visibility.

## Implementation notes
- This should make review easier, not louder.
- Keep reviewer suggestions advisory until the heuristics earn operator trust.
- High-signal handoff metadata is much more valuable here than broad automation.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

Triggered by label 'rascal' on issue #163

# Rascal Run Instructions

Run ID: run_aa349cb78d8c9c4a
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163

## Task

feat(review): add reviewer routing hints and diff risk signals for Rascal PR handoff

## Summary
Rascal can already create or update PRs and move runs into a `review` state, but the handoff to humans is still thin. It does not help prioritize risky diffs, suggest likely reviewers, or distinguish changes that deserve extra attention.

This issue adds a lightweight review-handoff layer that computes routing hints and risk signals when Rascal opens or updates a PR.

## Outcome
When a run reaches PR handoff, Rascal can attach reviewer suggestions, risk metadata, and review-priority signals so the human review step is faster and better targeted.

## Goals
- Improve the quality of the human handoff without turning Rascal into a noisy review bot.
- Surface high-risk changes explicitly.
- Suggest likely reviewers using repository-local signals.
- Keep everything explainable, inspectable, and GitHub-native.
- Preserve conservative defaults and avoid surprise reviewer spam.

## Non-goals
- Building a full inbox product with calendars, timezones, or org graphs.
- Auto-approving or auto-merging PRs.
- Replacing CODEOWNERS or branch protection.
- Emitting large volumes of AI-generated inline review comments.

## Scope
Implement V1 review-handoff support with the following behavior.

1. Review handoff artifact
- Persist explicit handoff metadata in the run directory, for example:
  - `review-handoff.json`
  - `review-handoff.md`
- Include at least:
  - suggested reviewers
  - risk score or risk level
  - reasons for the risk classification
  - changed path summary
  - notable signals such as tests changed or not changed

2. Reviewer suggestion heuristics
- Suggest reviewers from a small, deterministic set of signals such as:
  - CODEOWNERS matches when available
  - recent file-touch history within the repo
  - previous PR author/reviewer activity for the same area
- Keep the heuristics explainable and stable.
- If no high-confidence suggestion exists, record that explicitly instead of guessing.

3. Risk classification
- Compute a compact risk classification such as `low`, `medium`, `high`.
- V1 signals may include:
  - number of files changed
  - number of directories or packages touched
  - presence of config/deploy/runtime files
  - changes in security-sensitive or infra-sensitive paths
  - missing test deltas for production code changes
  - cross-cutting changes spanning multiple areas
- Keep the scoring heuristic simple and visible in the artifact.

4. GitHub presentation
- Add the handoff summary to the PR body or completion comment in a concise way.
- Optionally add conservative labels such as:
  - `rascal:risk-high`
  - `rascal:needs-review`
- Do not request reviewers automatically in V1 unless explicitly configured.
- Start with suggestions and metadata, not side effects.

5. CLI/API visibility
- Expose handoff metadata through API/JSON output and optionally a CLI view command.
- Operators should be able to inspect why Rascal considered a change risky.

6. Notification hygiene
- Ensure handoff metadata updates are idempotent when Rascal revises an existing PR.
- Avoid posting duplicate comments or churning labels on every follow-up run.

## Suggested implementation approach
1. Add a small review-handoff package.
- Separate reviewer suggestion, risk scoring, artifact rendering, and GitHub presentation.
- Keep the scorer independent from backend-specific execution details.

2. Start with repo-local deterministic signals only.
- Prefer CODEOWNERS and git history over external services.
- If CODEOWNERS parsing is absent today, add a narrow parser sufficient for reviewer discovery.

3. Keep side effects opt-in.
- Suggest reviewers first.
- Automatic reviewer requests can be a follow-up once confidence and ergonomics are proven.

4. Make idempotency explicit.
- Reuse the existing notification-marker pattern or its successor.
- PR body sections or comments should be updateable rather than appended repeatedly.

5. Keep the risk model intentionally small.
- The point is prioritization, not perfect prediction.
- A transparent heuristic is preferable to an opaque score.

## Acceptance criteria
- Rascal produces explicit review-handoff artifacts when a run reaches PR handoff.
- Reviewer suggestions use deterministic repo-local signals and include explanation.
- Rascal assigns a compact risk classification with visible reasons.
- GitHub presentation is concise and idempotent.
- The feature works for newly created PRs and existing PR updates.
- Tests cover CODEOWNERS/history suggestion paths, risk scoring, presentation idempotency, and no-suggestion fallback.

## Suggested test plan
- Unit tests for CODEOWNERS matching and reviewer suggestion ranking.
- Unit tests for risk classification heuristics.
- Integration tests for PR-body or comment updates without duplication.
- Regression tests for follow-up runs that revise the same PR and preserve stable handoff metadata.
- API/CLI tests for handoff artifact visibility.

## Implementation notes
- This should make review easier, not louder.
- Keep reviewer suggestions advisory until the heuristics earn operator trust.
- High-signal handoff metadata is much more valuable here than broad automation.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

Triggered by label 'rascal' on issue #163

<info-msg>
It is currently 2026-03-28 12:04:00
Working directory: /work/repo
Context: ~10469k/128k tokens used (8179%)

Current tasks and notes:
Once given a task, immediately update your todo with all explicit and implicit requirements

</info-msg>

Assistant: I’m tracing the existing PR handoff path first so I can add review metadata where the run already persists artifacts and updates GitHub, then I’ll keep the change set tight around those integration points.

I found the likely seams: PR body/completion rendering in `internal/runsummary`, notification idempotency in `internal/orchestrator/notifications.go`, and run finalization in `internal/worker/worker.go`. I’m reading those now to wire the new handoff artifact into the existing review transition instead of inventing a second path.

The runner side is simpler than expected: it already owns diff creation, PR body generation, and writes repo-local meta files into the run directory. I’m checking whether I can compute reviewer/risk signals entirely from local git state there, then expose the resulting artifact from the server without adding new persisted DB state.

I have the implementation shape now. I’m going with a small pure `reviewhandoff` package for scoring/rendering, using repo-local git/CODEOWNERS data from the worker to write `review-handoff.json` and `review-handoff.md`, then exposing that JSON on `GET /v1/runs/{id}` and updating a single managed PR-body section in place for idempotency.

I’m starting the code edits now. The main changes are a new `internal/reviewhandoff` package, a worker-side collector/writer for git-derived handoff data, and a small API extension so the stored handoff JSON is inspectable after the run completes.

I’ve got the first pass in place, but there are a few details to tighten before verification: CODEOWNERS team handles need proper normalization, the PR body helper needs a cleaner insert path for the managed section, and I still need the focused tests for existing-PR updates and API visibility.

The remaining work is mechanical now: formatting, compile/test passes, then any cleanup from failures. After that I’ll run the required `make lint` and `make test`, and write the commit message file.

I’ve finished the targeted tests. Next I’m running the repo’s required validation exactly as requested: `make lint` and `make test`.

Both `make` targets are still in dependency/download work. I’m waiting them through rather than substituting a narrower check, because the task explicitly asked for the real targets when present.

`make lint` and `make test` are both still inside `sqlc`/tool bootstrap work. They aren’t hung; they’re just duplicating the `codegen` prerequisite in parallel, so I’m letting them finish and then I’ll report the actual target results.

The parallel `make` run turned out to be counterproductive: both targets are compiling the same large `sqlc` dependency tree at once. I’m collapsing that into a sequential run now so validation finishes instead of burning CPU on duplicated work.

`make lint` found two real issues, both straightforward `wrapcheck` complaints. I’m patching those exact lines and then rerunning lint.

The implementation is complete and validation is green. I’m writing the required conventional commit message file now, then I’ll summarize the exact user-visible changes and the verification results.

Implemented the review handoff layer end to end.

A new [reviewhandoff package](/work/repo/internal/reviewhandoff/reviewhandoff.go#L13) now computes deterministic reviewer hints from CODEOWNERS and recent git history, assigns a visible `low`/`medium`/`high` risk score with reasons, renders both `review-handoff.json` and `review-handoff.md`, and manages a single PR-body handoff section idempotently. The worker now builds and persists that metadata during PR handoff, includes it on new PR bodies, and updates the same managed section on existing PRs instead of appending duplicates ([worker integration](/work/repo/internal/worker/worker.go#L274), [git/CODEOWNERS collection](/work/repo/internal/worker/review_handoff.go#L13), [PR body support](/work/repo/internal/runsummary/runsummary.go#L114)). I kept reviewer suggestions advisory only; there are no automatic reviewer requests or label side effects in this change.

Run inspection now exposes the handoff metadata through `GET /v1/runs/{id}` via the `review_handoff` JSON field ([API response type](/work/repo/internal/api/types.go#L50), [handler](/work/repo/internal/orchestrator/api.go#L622)). I added focused tests for reviewer heuristics, risk scoring, PR-body idempotency, worker artifact generation/update behavior, and API visibility ([reviewhandoff tests](/work/repo/internal/reviewhandoff/reviewhandoff_test.go), [worker tests](/work/repo/cmd/rascal-runner/main_test.go), [API test](/work/repo/internal/orchestrator/api_reviewhandoff_test.go)). Validation passed with `make lint` and `make test`. I also wrote [/rascal-meta/commit_message.txt](/rascal-meta/commit_message.txt).

Human: # Rascal Run Instructions

Run ID: run_583b12d008cdfdb2
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163
Pull Request: #178

## Task

Address PR #178 feedback

## Git Context

- Remote: `origin`
- Base branch: `main`
- Head branch: `rascal/rtzll/rascal-163-aa349cb78d`
- The repository is already cloned and checked out.
- You may use `git` and `gh` directly.
- Push only to `origin` branch `rascal/rtzll/rascal-163-aa349cb78d`.
- If you rewrite history, you must run `git push --force-with-lease origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Otherwise run `git push origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Do not push to any other branch.
- Before finishing, ensure the remote branch is updated and the working tree is clean.

- If the request involves rebasing, merge conflict resolution, or other history rewriting, do not rely on the harness to publish those changes for you. Perform the required `git push` yourself before finishing.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

please rebase on origin/main and resolve rebase conflicts then amend and force push


Assistant: 
Image files: 0
============================
{"type":"message","message":{"id":null,"role":"assistant","created":1774699932,"content":[{"type":"text","text":"I’m pulling the current branch state and PR #178 feedback first, then I’ll rebase onto `origin/main`, resolve any conflicts in place, run the required checks, and force-push the updated branch.\n\nThe PR feedback is minimal and concrete: rebase the feature branch onto the latest `origin/main`, resolve any conflicts, keep the branch clean, then re-run validation and force-push the amended result. After the rebase I’ll inspect the resulting diff to make sure nothing else shifted unexpectedly before I publish it.\n\n`origin/main` has advanced, so I’m rebasing the PR branch onto commit `99fde45` now. If Git surfaces conflicts, I’ll resolve them directly in the affected files and keep the branch history linear as requested.\n\nThe rebase only hit two import blocks, which is the good kind of conflict. I’m merging those with the newer `main` additions intact, then I’ll continue the rebase and inspect whether any generated behavior changed under the updated base.\n\nI’ve reduced the rebase to two import merges. I’m applying the minimal conflict resolution now, keeping both the new `main` runtime types and the review-handoff imports so the rebased branch still compiles against the updated base.\n\nThe rebase is blocked by missing local Git identity, not by code. I’m pulling the existing commit author metadata from the branch and setting that identity locally so the rebased commit preserves the expected authorship and can continue non-interactively.\n\nThe branch is rebased cleanly on top of `origin/main` now. I’ve checked the resulting commit shape; next is the required repo validation so I can confirm the conflict resolution didn’t subtly break anything before I amend and push.\n\nBoth validation targets are live. They’ve started from the shared `sqlc generate` prerequisite again, so I’m letting them advance briefly; if they just contend with each other like last time, I’ll collapse back to sequential execution to avoid wasting time.\n\n`make test` exposed a real rebase-following compile issue: the tests still import `internal/agent`, which no longer exists on top of current `main`. I’m locating the replacement type on the updated base, fixing that import/use site, then I’ll rerun validation.\n\nThe updated base renamed runtime wiring, and one of the new tests still reflected the pre-rebase package name. I’m correcting that now: dropping the stale unused import from the runner test and switching the API test onto `internal/runtime` with the current runtime constant.\n\nThe first `make test` run failed exactly where the stale runtime import was, and that’s fixed now. I’ve reformatted the touched tests and I’m resetting validation against the corrected tree rather than trusting the earlier partial run.\n\n`make lint` hit repo tooling hygiene rather than source failures: another `golangci-lint` process was already active from the earlier run, and the target refuses to run in parallel. I’m checking which validation processes are still alive, then I’ll rerun the targets sequentially once the stale lint worker is out of the way.\n\nThe tree diff is still just the expected rebase fixups. I’m clearing the stale `golangci-lint` and duplicate `go test` processes I started earlier so the final required validation can run once, sequentially, against the final tree.\n\nLint is in the normal long `golangci-lint` phase now, with no new errors surfaced yet. I’m keeping it sequential from here so the final result is unambiguous and not contaminated by duplicate tool processes.\n\n`golangci-lint` is still actively consuming CPU, so this is normal runtime rather than a deadlock. I’m just waiting it through now; once it exits I’ll run `make test` on the same final tree and then publish the branch.\n\n`make test` found one real regression after the rebase: a runner test stub no longer satisfies the updated branch-diff check on current `main`. I’m reading that test and the new `check_branch_diff` path now, then I’ll patch the fixture so it matches the current command contract and rerun the full test target.\n\nThe failure is just a stale test double. Current `main` always probes `git rev-list --left-right --count origin/<base>...HEAD` before updating an existing PR, and this one test never mocked that command, so it falls through to empty output. I’m fixing that fixture only, then I’ll rerun `make test`.\n\nThe remaining test regression is fixed; `make test` is green on the rebased tree now. I changed one more test fixture after the earlier lint pass, so I’m running `make lint` one last time against the final state before I amend and push.\n\nThe code is clean except for the two expected post-rebase test adjustments. I’m staging those, amending the existing feature commit so the branch stays as a single rebased change, then I’ll write the required commit message file and force-push.\n\nThe repo worktree is clean and the branch is now a single amended commit on top of `origin/main` (`2d78027`). I’m force-pushing with lease to the PR branch now, then I’ll do one last remote/status check before I close this out.\n\nRebased PR #178 onto `origin/main`, resolved the two rebase conflicts, amended the feature commit, and force-pushed the updated branch. The remote PR branch now points to `2d780273918953cda3370243e70fd263132d5476`, and the worktree is clean.\n\nThe only post-rebase fixes were test-alignment changes in [cmd/rascal-runner/main_test.go](/work/repo/cmd/rascal-runner/main_test.go) and [internal/orchestrator/api_reviewhandoff_test.go](/work/repo/internal/orchestrator/api_reviewhandoff_test.go): one stale `internal/agent` import had to move to `internal/runtime`, and one runner test fixture needed the current `git rev-list --left-right --count` mock that `main` now expects. I also wrote [/rascal-meta/commit_message.txt](/rascal-meta/commit_message.txt).\n\nValidation on the final tree:\n- `make lint`\n- `make test`\n\nPush performed:\n- `git push --force-with-lease origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`"}],"metadata":{"userVisible":true,"agentVisible":true}}}
{"type":"complete","total_tokens":2735427}

Closes #163


Rascal run run_583b12d008cdfdb2 completed in 7m 44s · 2.74M tokens

@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

rebase

@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

Rascal started run run_4eb70eac30bd4c2e to address new PR feedback.

Run Settings
  • Requested by: rtzll

  • Trigger: pr_comment

  • Agent Runtime: goose-codex

  • Runner commit: 99fde4590df928cc60014956b6c85c843c707b4e

  • Branches: main -> rascal/rtzll/rascal-163-aa349cb78d

  • Session mode: all

  • Resume: true

  • Debug: true

  • Queue delay: 0s

  • Instruction: Address PR feat(review): add PR review handoff metadata #178 feedback

  • Context: rebase again please

@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

Rascal started run run_ca3fb270d63864cb to address new PR feedback.

Run Settings
  • Requested by: rtzll

  • Trigger: pr_comment

  • Agent Runtime: goose-codex

  • Runner commit: 256842802ff7c6684a913ba0e9681d4b8903bf5a

  • Branches: main -> rascal/rtzll/rascal-163-aa349cb78d

  • Session mode: all

  • Resume: true

  • Debug: true

  • Queue delay: 39m 23s

  • Instruction: Address PR feat(review): add PR review handoff metadata #178 feedback

  • Context: rebase

@rtzll

rtzll commented Mar 28, 2026

Copy link
Copy Markdown
Owner Author

@rtzll posted the run details below.

Automated changes from Rascal run run_ca3fb270d63864cb.

Agent Details

    __( O)>  ● resuming · codex gpt-5.4
   \____)    20260328_1 · /work/repo
     L L     goose is ready
=== CODEX PROVIDER DEBUG ===
Command: "/usr/local/bin/codex"
Model: gpt-5.4
Reasoning effort: high
Skip git check: false
Prompt length: 4360 chars
Prompt: You are a general-purpose AI agent called goose, created by Block, the parent company of Square, CashApp, and Tidal.
goose is being developed as an open-source software project.
# Suggestion

The user has 7 extensions with 16 tools enabled, exceeding recommended limits (5 extensions or 50 tools).
Consider asking if they'd like to disable some extensions to improve tool selection accuracy.

# Response Guidelines

Use Markdown formatting for all responses.

Human: <info-msg>
It is currently 2026-03-28 16:41:00
Working directory: /work/repo

# Rascal Persistent Instructions

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Do not overwrite, revert, or discard user changes you did not make unless the task explicitly requires it.
- Use the repository's existing patterns and conventions.
- Prefer the repository's documented workflow over inventing a new one.
- If the repository provides verification commands, run the relevant ones before finishing.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.
- If working with GitHub branches or pull requests, only push to the designated Rascal branch for this run.
- If you must rewrite published history, prefer `git push --force-with-lease` over `git push --force`.
- Before finishing, ensure the working tree is clean unless the task explicitly requires uncommitted output.

Current tasks and notes:
Once given a task, immediately update your todo with all explicit and implicit requirements

</info-msg>
# Rascal Run Instructions

Run ID: run_4eb70eac30bd4c2e
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163
Pull Request: #178

## Task

Address PR #178 feedback

## Git Context

- Remote: `origin`
- Base branch: `main`
- Head branch: `rascal/rtzll/rascal-163-aa349cb78d`
- The repository is already cloned and checked out.
- You may use `git` and `gh` directly.
- Push only to `origin` branch `rascal/rtzll/rascal-163-aa349cb78d`.
- If you rewrite history, you must run `git push --force-with-lease origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Otherwise run `git push origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Do not push to any other branch.
- Before finishing, ensure the remote branch is updated and the working tree is clean.

- If the request involves rebasing, merge conflict resolution, or other history rewriting, do not rely on the harness to publish those changes for you. Perform the required `git push` yourself before finishing.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

rebase again please

# Rascal Run Instructions

Run ID: run_ca3fb270d63864cb
Task ID: rtzll/rascal#163
Repository: rtzll/rascal
Issue: #163
Pull Request: #178

## Task

Address PR #178 feedback

## Git Context

- Remote: `origin`
- Base branch: `main`
- Head branch: `rascal/rtzll/rascal-163-aa349cb78d`
- The repository is already cloned and checked out.
- You may use `git` and `gh` directly.
- Push only to `origin` branch `rascal/rtzll/rascal-163-aa349cb78d`.
- If you rewrite history, you must run `git push --force-with-lease origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Otherwise run `git push origin HEAD:rascal/rtzll/rascal-163-aa349cb78d`.
- Do not push to any other branch.

- If the request involves rebasing, merge conflict resolution, or other history rewriting, do not rely on the harness to publish those changes for you. Perform the required `git push` yourself before finishing.


## Additional Context

rebase


Assistant: 
Image files: 0
============================
{"type":"message","message":{"id":null,"role":"assistant","created":1774716070,"content":[{"type":"text","text":"Ran into this error: Request failed: Codex CLI error: You've hit your usage limit. To get more access now, send a request to your admin or try again at 5:17 PM..\n\nPlease retry if you think this is a transient or recoverable error."}],"metadata":{"userVisible":true,"agentVisible":true}}}
{"type":"complete","total_tokens":null}

Closes #163


Rascal run took 14s

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

rascal Trigger Rascal automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(review): add reviewer routing hints and diff risk signals for Rascal PR handoff

1 participant