feat(runner): add pi agent backend support - #174

    __( O)>  ● resuming · codex gpt-5.4
   \____)    20260315_1 · /work/repo
     L L     goose is ready
=== CODEX PROVIDER DEBUG ===
Command: "/usr/local/bin/codex"
Model: gpt-5.4
Reasoning effort: high
Skip git check: false
Prompt length: 12821 chars
Prompt: You are a general-purpose AI agent called goose, created by Block, the parent company of Square, CashApp, and Tidal.
goose is being developed as an open-source software project.
# Suggestion

The user has 7 extensions with 16 tools enabled, exceeding recommended limits (5 extensions or 50 tools).
Consider asking if they'd like to disable some extensions to improve tool selection accuracy.

# Response Guidelines

Use Markdown formatting for all responses.

Human: # Rascal Run Instructions

Run ID: run_924b92806c7a9d35
Task ID: rtzll/rascal#173
Repository: rtzll/rascal
Issue: #173

## Task

Add pi as an agent backend alongside Goose and Codex

## Summary

Add [pi](https://github.com/badlogic/pi-mono) as a third agent backend so Rascal can run tasks with `pi`, `goose`, or `codex`.

This is feasible with the current architecture, but it is not just an enum addition. The control plane is already mostly backend-agnostic for task/run/session state, while the runner, deploy surfaces, and stored-credential flow are still shaped around exactly two backends and one credential type.

## Current state

Rascal already stores `agent_backend` on tasks/runs and persists backend-specific task sessions, so the basic scheduling/state model can support a third backend.

The main coupling points today are:

- `internal/agent`: backend parsing/normalization only supports `goose` and `codex`
- `cmd/rascal-runner`: backend dispatch only supports `runGoose` and `runCodex`
- `internal/runner/docker`: session mounts and env wiring are only `GOOSE_PATH_ROOT` or `CODEX_HOME`
- `runner/Dockerfile`: only `goose-runner` and `codex-runner` images exist
- deploy / doctor / CLI flags: assume exactly two runner images
- stored credentials: explicitly modeled as `codex_credentials` and written to `runDir/codex/auth.json`

## What needs to change

### Backend plumbing

- Add `pi` to backend parsing and normalization
- Update config/image selection to support a Pi runner image
- Update CLI help/docs/tests that currently say `goose or codex`

### Runner implementation

- Add a `runPi(...)` path in `cmd/rascal-runner`
- Validate the `pi` binary in backend-specific command checks
- Map Rascal instructions/output/session handling onto Pi CLI behavior
- Preserve the canonical run artifacts (`agent.ndjson`, `agent_output.txt`, `meta.json`)

### Session support

- Add Pi-specific session root/home env wiring in Docker launcher
- Mount task-scoped Pi session state when session resume is enabled
- Persist/reuse backend session IDs for Pi if needed by its CLI model

### Runner image / deploy

- Add a `pi-runner` image target or install Pi in the shared runner base image
- Extend deploy/bootstrap to build/publish the Pi runner image
- Extend remote doctor to inspect Pi image config and presence
- Add config/env support for `RASCAL_RUNNER_IMAGE_PI`

### Credentials

This is the main design decision.

Short term:
- Support Pi with env/API-key auth only

Long term:
- Generalize stored credentials from `codex_credentials` to backend/provider-aware credentials
- Allow Rascal to broker both Codex and Pi credentials cleanly
- Stop hardcoding `runDir/codex/auth.json` as the only leased auth material path

## Proposed rollout

### Phase 1: MVP Pi backend

- Add `pi` backend enum/config/runner dispatch
- Add Pi runner image
- Add Pi session-dir wiring
- Support Pi with env/API-key auth only
- Keep existing Codex credential broker unchanged

### Phase 2: Credential model refactor

- Generalize stored credentials to support multiple backends/providers
- Add Pi credential acquisition/lease/write/cleanup flow
- Update auth management CLI and API accordingly

### Phase 3: Parity work

- Add Pi token-usage extraction if Pi JSON output exposes it
- Update docs/runbook/config/deploy help text
- Extend tests across runner, deploy, config, doctor, and state flows

## Acceptance criteria

- Rascal accepts `RASCAL_AGENT_BACKEND=pi`
- The orchestrator can schedule and execute Pi-backed runs end-to-end
- Pi runs can create/update PRs the same way Goose/Codex runs do today
- Session resume works for Pi when agent sessions are enabled
- Deploy/bootstrap/doctor support a Pi runner image
- Tests and docs are updated to describe all three backends

## Open questions

- Should Pi be introduced only as a direct/API-key backend first, or should stored/brokered Pi credentials be part of the first implementation?
- Does Pi expose stable enough session identifiers that we should persist `backend_session_id`, or is mounted session storage alone sufficient?
- Do we want three distinct runner images (`goose`, `codex`, `pi`) or a consolidated image with all agent CLIs installed?

## Suggested first milestone

Ship Phase 1 first: Pi backend + runner image + session support, without blocking on the credential-schema refactor.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

Triggered by label 'rascal' on issue #173

<info-msg>
It is currently 2026-03-28 15:19:00
Working directory: /work/repo
Context: ~13140k/128k tokens used (10266%)

# Rascal Persistent Instructions

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Do not overwrite, revert, or discard user changes you did not make unless the task explicitly requires it.
- Use the repository's existing patterns and conventions.
- Prefer the repository's documented workflow over inventing a new one.
- If the repository provides verification commands, run the relevant ones before finishing.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.
- If working with GitHub branches or pull requests, only push to the designated Rascal branch for this run.
- If you must rewrite published history, prefer `git push --force-with-lease` over `git push --force`.
- Before finishing, ensure the working tree is clean unless the task explicitly requires uncommitted output.

Current tasks and notes:
Once given a task, immediately update your todo with all explicit and implicit requirements

</info-msg>

Assistant: Using the repo directly to add `pi` as a third backend. I’m starting by locating the backend enum/config, runner dispatch, docker/session wiring, and any tests or docs that currently assume only `goose` and `codex`.

I’ve confirmed the main coupling points match the issue summary: backend parsing, runner dispatch, docker env/session mounts, runner image config, and deploy/doctor help text are all hard-coded for two backends. Next I’m reading those exact codepaths to patch the minimal set cleanly.

The backend model is simple enough that Phase 1 can stay scoped: add `pi` anywhere backend/image/session selection branches today, keep the existing Codex credential broker untouched, and let Pi rely on env/API-key auth only. I’m checking whether the runner already has enough generic plumbing to map Pi onto the existing `agent.ndjson` / `agent_output.txt` / `meta.json` artifacts without touching orchestration state.

I have the Pi CLI shape now. The cleanest MVP path is: `pi --mode json` for the canonical NDJSON log, extract the final assistant text into `agent_output.txt`, use task-scoped `--session-dir` for resume, and keep credentials env-only. I’m moving into edits across backend parsing, docker env/session wiring, deploy/doctor image config, and the runner implementation/tests.

I’m updating the shared backend/config layer first so every higher-level path can branch on `pi` cleanly instead of carrying special cases. After that I’ll wire the runner and deployment surfaces to the new backend-specific image/session paths.

The next edit is the runner itself: `runPi` plus Pi-specific config/session handling. I’m keeping the artifact contract unchanged by treating Pi’s JSON event stream as the canonical `agent.ndjson` and deriving `agent_output.txt` from the last assistant message.

The code changes are in place across the main plumbing. I’m formatting and then I’ll run focused tests to catch the expected misses: Pi-specific struct fields/defaults, deploy/doctor flag coverage, and any runner behavior I still need to adjust.

The first test batch is still running; the new backend constants and some runner logic already compile. I’m waiting out the rest so I can fix the actual breakpoints instead of guessing.

I’m down to user-facing fallout now: help snapshots and docs still describe a two-backend world. I’m updating those references and then I’ll rerun the full checks plus `make lint` / `make test` if the repo exposes them.

The targeted Go packages are passing. The remaining requirement is repository-level verification, so I’m running the actual `make lint` and `make test` targets now rather than substituting narrower checks.

`make lint` is still in progress because it has to regenerate sqlc output and bootstrap the pinned linter toolchain first. I’m letting that complete so we get the repo’s actual lint signal, not just the targeted package pass I already have.

`make lint` found only `wrapcheck` issues in the new Pi runner path and its tests. I’m fixing those now, then I’ll rerun lint and the full test target.

Verification is clean now. I’m checking the final diff for any generated/sqlc changes from the required `make` targets, then I’ll write the conventional commit message file before wrapping up.

Implemented Phase 1 Pi backend support alongside Goose and Codex.

The main changes are in [cmd/rascal-runner/main.go](/work/repo/cmd/rascal-runner/main.go), [internal/runner/docker.go](/work/repo/internal/runner/docker.go), [internal/deploy/deploy.go](/work/repo/internal/deploy/deploy.go), [cmd/rascal/doctor_remote.go](/work/repo/cmd/rascal/doctor_remote.go), and [runner/Dockerfile](/work/repo/runner/Dockerfile). Rascal now accepts `pi` in backend parsing/config, selects a Pi runner image, dispatches a `runPi(...)` path, mounts task-scoped Pi session storage via `PI_SESSION_DIR`, and preserves the canonical run artifacts by treating Pi JSON mode as `agent.ndjson` and extracting the final assistant text into `agent_output.txt`. Stored credentials remain unchanged; Pi is env/API-key only.

I also updated deploy/doctor/CLI help and docs for three backends, added Pi image env support (`RASCAL_RUNNER_IMAGE_PI`), extended tests, and wrote the commit message to `/rascal-meta/commit_message.txt`.

Verification: `make lint` passed, `make test` passed.

Human: # Rascal Run Instructions

Run ID: run_84f54c9058d3a2bb
Task ID: rtzll/rascal#173
Repository: rtzll/rascal
Issue: #173
Pull Request: #174

## Task

Address PR #174 feedback

## Git Context

- Remote: `origin`
- Base branch: `main`
- Head branch: `rascal/rtzll/rascal-173-924b92806c`
- The repository is already cloned and checked out.
- You may use `git` and `gh` directly.
- Push only to `origin` branch `rascal/rtzll/rascal-173-924b92806c`.
- If you rewrite history, you must run `git push --force-with-lease origin HEAD:rascal/rtzll/rascal-173-924b92806c`.
- Otherwise run `git push origin HEAD:rascal/rtzll/rascal-173-924b92806c`.
- Do not push to any other branch.
- Before finishing, ensure the remote branch is updated and the working tree is clean.

- If the request involves rebasing, merge conflict resolution, or other history rewriting, do not rely on the harness to publish those changes for you. Perform the required `git push` yourself before finishing.


## Constraints

- Do not ask for interactive input.
- Do not require MCP tools.
- Keep changes minimal and scoped to the requested task.
- Run `make lint` and `make test` before finishing if those targets exist.
- If one of those commands does not exist or cannot run, explain exactly why and run the closest equivalent checks instead.
- If you make changes, write /rascal-meta/commit_message.txt using a conventional commit title on the first line.
- Optionally add a commit body after a blank line in /rascal-meta/commit_message.txt.

## Additional Context

rebase


Assistant: 
Image files: 0
============================
{"type":"message","message":{"id":null,"role":"assistant","created":1774712222,"content":[{"type":"text","text":"Reviewing PR #174 feedback and the current branch state first. I’m going to pull the review comments with `gh`, inspect the rebase state, and then patch only the requested follow-ups before rerunning the repo checks and pushing the branch.\n\nThe PR feedback is only `rebase`, with no code-review findings attached. I’m syncing against `origin/main` now; if the branch replays cleanly, I’ll rerun `make lint` and `make test`, then force-push the rebased branch as required.\n\n`origin/main` has moved well past this branch, so this is a real history rebase rather than a no-op. I’m replaying the branch onto current `origin/main` next and will resolve only conflicts caused by mainline drift.\n\n`main` refactored several of the same surfaces touched by the Pi work, so the rebase stopped on real content conflicts rather than mechanical overlap. I’m reconciling the branch against current structure now, especially the `internal/agent` move and the CLI/config/help text churn that happened on `main`.\n\n`main` deleted the old `internal/agent` package, so I’m not going to mechanically keep those conflicted files. I’m checking the new backend package layout on `origin/main` and will reapply the Pi support into the new structure instead of preserving the obsolete paths.\n\n`main` already extracted the old runner logic into `internal/worker`, so several conflicts are just stale copies of code that no longer owns the behavior. I’m mapping the Pi changes onto the new `internal/runtime` and `internal/worker` packages, then I’ll drop obsolete conflict hunks instead of carrying duplicated logic forward.\n\nI’ve got the new base mapped out. The next edit step is to resolve the rebase by taking current `main` as the source of truth, then reapplying Pi support into the newer runtime/worker/deploy structure instead of trying to keep the pre-refactor files alive.\n\nI’m applying the Pi runtime into the new worker/orchestrator path now: runtime parsing, env/session loading, direct runner execution, broker bypass for Pi auth, Docker session wiring, and deploy image selection. After that I’ll update the CLI/help surfaces and the affected tests.\n\nThe core code path is wired. I’m on the test and CLI fallout now: adding Pi cases to runtime/worker/docker/deploy/doctor coverage, then I’ll clean the remaining docs/help references and run the full repo checks.\n\nThe Pi cases are in the tests now. I’m running targeted Go tests before touching the remaining docs so I can fix real compile or behavior regressions first, then I’ll finish the text/help cleanup and do the required full `make` verification.\n\nTwo targeted test failures came back and they’re both local to test assumptions, not the Pi implementation itself. I’m correcting those expectations now, then I’ll rerun the focused suite before finishing the remaining doc cleanup.\n\nThe code is clean but the rebase is still paused because Git needs the resolved files staged. I’m adding the resolved set now, then I’ll finish the rebase and run the repo-level `make lint` and `make test` checks before pushing.\n\nThe rebase stopped only because this checkout doesn’t have a local Git committer identity configured. I’m continuing it with an explicit temporary committer identity so the replayed commit keeps moving without changing anything else.\n\nThe rebase replay is complete now. I’m formatting the touched Go files and doing one last text sweep for stale two-backend wording before I run the full `make lint` and `make test` targets and push the rewritten branch.\n\nRepo-level verification is running now. Both `make` targets start by regenerating `sqlc`, so I’m waiting for the full lint and test signal rather than treating the earlier targeted package passes as sufficient.\n\n`make test` is clean. `make lint` is still in the linter pass, which is normal for this repo; I’m waiting it out so the final state includes the actual lint result after the rebase resolution, not just the targeted Go tests.\n\nThe branch is rebased and the worktree is clean; the only remaining step is publishing the rewritten history. I’m force-pushing with lease to the designated Rascal branch now, then I’ll do a final status check and wrap up.\n\nRebased PR #174 onto current `main` and pushed the rewritten branch to `origin/rascal/rtzll/rascal-173-924b92806c` at `5bcc328`.\n\nThe rebase was not mechanical because `main` had already moved the old backend code into `internal/runtime` and `internal/worker`. I reapplied the Pi work onto that structure: added `pi` as a runtime, wired direct Pi execution and session handling, skipped brokered stored-credential leasing for Pi, added `RASCAL_RUNNER_IMAGE_PI`, updated Docker/deploy/doctor/CLI surfaces, refreshed docs/help text, and extended the affected tests. I also wrote `/rascal-meta/commit_message.txt`.\n\nVerification passed: `make lint`, `make test`. The local branch and remote branch now match, and the worktree is clean."}],"metadata":{"userVisible":true,"agentVisible":true}}}
{"type":"complete","total_tokens":14032811}