feat(codex): env-mcp subcommand of codex-app-gateway - #78
Merged
Merged
Conversation
added 11 commits
May 10, 2026 20:25
…ouble-print, reject positional args)
Implements Translator.RunShell: process/start then polling process/read loop (up to 240 × 250ms = 60s), aggregates stdout/stderr, surfaces exit code and failures in ShellResult. BridgeCaller interface enables scripted unit tests without a real WebSocket server.
imryao
pushed a commit
that referenced
this pull request
May 10, 2026
…ead codex app-server subprocess Adds 2026-05-10-codex-app-gateway-subprocess.md, which replaces Subsystem 2 of the MCP-rewrite spec with a design that spawns one upstream `codex app-server --listen ws://...` subprocess per thread and proxies ws frames between the user's TUI and the subprocess. Cuts ~1100 LOC of Go (no 17-RPC handler reimplementation, no Postgres schema, no event mapper) at the cost of a per-thread subprocess supervisor + S3 CODEX_HOME round-trip. Validated against codex-cli 0.130.0: - PoC #3: full initialize → thread/start → turn/start → ServerNotification stream end-to-end on a loopback ws. - PoC #3b: two concurrent subprocesses with isolated CODEX_HOMEs run in independent threads with no sqlite contention. Subsystems 1 (no fork patches), 3 (codex-exec-gateway), and 4 (env-mcp, PR #78) are unchanged. Cross-references added to the MCP-rewrite spec header.
imryao
pushed a commit
that referenced
this pull request
May 10, 2026
…tion plan Marks the 2026-05-05 codex-app-gateway-and-exec-gateway design spec SUPERSEDED by the 2026-05-10 MCP-rewrite spec (and now further refined by the codex-app-gateway-subprocess spec). Marks the four 2026-05-05 plan files OBSOLETE with pointers to their successors. Adds 2026-05-10-codex-env-mcp.md, the implementation plan executed in PR #78 that delivered Subsystem 4 of the MCP-rewrite spec (env-mcp subcommand of codex-app-gateway).
5 of 7 tasks
imryao
added a commit
that referenced
this pull request
May 11, 2026
… app-server (Subsystem 2)
Implements Subsystem 2 of the codex-gateway design as a thin auth-proxy + per-thread \`codex app-server\` subprocess manager + transparent ws frame proxy. Replaces the originally-planned 17-RPC handwritten Go RPC layer.
- New: \`internal/codexappgateway/{auth,codexhome,supervisor,proxy}/\` packages + \`server.go\` + \`s3_store.go\`.
- Modified: \`cmd/codex-app-gateway/main.go\` (wires the \`serve\` subcommand previously stubbed in #78).
- 63 tests + 1 integration test against real codex 0.130.0 — all PASS, race-clean.
- Spec: \`docs/superpowers/specs/2026-05-10-codex-app-gateway-subprocess.md\`
- Plan: \`docs/superpowers/plans/2026-05-11-codex-app-gateway-subprocess.md\`
**Final round of post-review fixes (after holistic 5-reviewer audit):**
- Runtime correctness: Touch-on-frame, race-loser unlock, stale-handle health-check on \`EnsureSubprocess\`, \`ChildHandle.Cmd\` → unexported.
- Hardening: \`os.Environ()\` inheritance for subprocess, \`filepath.Rel\` tmpdir guard, \`ConfigInput\` cross-field validation, \`*slog.Logger\` fields on Supervisor/IdleReaper/BridgeClient/MCPServer, S3 endpoint scheme validation.
imryao
pushed a commit
that referenced
this pull request
May 11, 2026
Near-verbatim refresh of the 2026-05-05 plan with three contextual
updates aligned to the 2026-05-10 MCP-rewrite spec:
- Spec reference now points to 2026-05-10-codex-gateway-mcp-rewrite.md § Subsystem 3
- /bridge/{exe_id} consumer is the env-mcp child binary (PR #78), not
the spawned codex itself (wire protocol + auth model unchanged)
- Cap-token allow-list is single-exe-id per the 2026-05-10 refinement
(verification logic unchanged; minting is stricter, happens in
codex-app-gateway)
Old 2026-05-05 plan file's banner updated to point at this reissue.
Merged
5 of 7 tasks
imryao
pushed a commit
that referenced
this pull request
May 12, 2026
…uracy Six interlinked fixes from the holistic PR self-review: - C1: Config.Validate() rejects empty HMAC/internal secrets; NewServer now returns (*Server, error) and refuses zero-value config. Closes a path where empty CapTokenHMACSecret silently accepted forged tokens. - C2: inbound + bridge ws now SetReadLimit(-1). nhooyr's 32KB default would silently close large process/read responses — codex exec-server streams them. (Same fix env-mcp PR #78 already had.) - C3: RevokedSet.Add returns evictedLive bool; RevokeTurn handler logs at Warn when a still-live revocation is evicted at capacity. Closes a silent security regression where evicted-but-live tokens worked again. - C4: bridge pump context now derived from r.Context() so graceful shutdown drains active sessions instead of leaking pump goroutines. - I7: migrate() wraps each error with the migration name so operators can tell which migration failed. - I8: corrects 3 wrong comments (bridge.go step-3 rationale, server.go nil-fields claim, forwarder.go shutdown semantics). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
imryao
added a commit
that referenced
this pull request
May 12, 2026
…egistry (Subsystem 3) Implements **Subsystem 3** of the codex-gateway design — completes the gateway service trio (env-mcp #78, codex-app-gateway #79, codex-exec-gateway here). - New service \`cmd/codex-exec-gateway\` + \`internal/codexexecgateway/\`: - chi router + lifecycle, embedded Postgres migrations. - Inbound \`/codex-exec/{exe_id}\` ws acceptor (bcrypt tunnel-token verify, single-conn-per-exe eviction). - Bridge \`/bridge/{exe_id}\` ws acceptor (HMAC cap-token verify with single-exe-id allow-list, revocation check, per-exe-id bridge mutex via \`AcquireBridge\`/\`ReleaseBridge\` to prevent concurrent-pump data races). - In-memory \`ConnRegistry\` (concurrent-safe, identity-guarded \`Unregister\`). - Bounded \`RevokedSet\` (cap 10000, FIFO eviction, periodic exp pruning, \`Add\` returns \`evictedLive bool\` for security alerting). - HTTP endpoints: \`POST /api/codex-exec/register\`, \`POST/GET/DELETE /api/codex-exec/workspaces/{wid}/executors\`, internal \`GET /api/exec-gateway/connected\`, \`POST /api/exec-gateway/revoke-turn\` (shared-secret-protected). - Cross-package factor-out: \`internal/codexappgateway/proxy/\` (from PR #79) moved to shared \`internal/wsbridge/\`, consumed by both gateways. - Shared DTOs in \`internal/codexexecgateway/execmodel\` (eliminates parent↔handlers import-cycle adapters). **Self-review** (5 specialized reviewers in parallel) flagged 4 Critical + 11 Important + 7 Minor; 3 fix batches landed (\`10fd4a1\`, \`f1c7f3e\`, \`1a52358\`): - C1: \`Config.Validate()\` rejects empty HMAC; \`NewServer\` returns error on zero-value config. - C2: inbound + bridge \`SetReadLimit(-1)\` (closes silent 32KB-frame truncation). - C3: revoked-set eviction logs Warn when dropping a still-live revocation. - C4: bridge pump derives ctx from \`r.Context()\` (graceful shutdown drains active sessions). - I3: per-exe-id bridge mutex prevents concurrent-pump data race. - Plus observability (auth failure logs, pump-error log levels, close-error logs), comment accuracy, dead-config removal, \`Store\` encapsulation tightening. Final: 35 PASS / 15 SKIP (DB-gated) / 0 FAIL, \`-race\` clean, vet clean, build clean. **Spec / Plan** - Spec: \`docs/superpowers/specs/2026-05-10-codex-gateway-mcp-rewrite.md\` § Subsystem 3 - Plan: \`docs/superpowers/plans/2026-05-12-codex-exec-gateway.md\` (14 TDD tasks, all complete)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First subsystem of the new
codex-gateway-mcp-rewritetrack. Addscodex-app-gateway env-mcp ..., a stdio MCP server that bridges MCPtools/calltraffic from a spawned codex subprocess to a single executor's exec-server endpoint via WebSocket. This is Subsystem 4 of the new spec — the runtime/foundations/exec-gateway subsystems are tracked separately.cmd/codex-app-gateway/with a multi-subcommand entry (env-mcpimplemented;serveplaceholder).internal/codexappgateway/envmcp/:bridge.go— JSON-RPC client over one WebSocket to/bridge/{exe_id}(race-tested concurrency).translator.go— MCPshelltool call → exec-serverprocess/start+ pollingprocess/readloop.mcp_server.go— newline-delimited JSON-RPC stdio MCP server exposing oneshelltool.envmcp.go— top-levelRun()wiring.codex exec-server).Dockerfile.codex-app-gateway— multi-stage build (golang:1.26-trixie → debian:trixie-slim).Spec / Plan
docs/superpowers/specs/2026-05-10-codex-gateway-mcp-rewrite.md§ Subsystem 4 (PoC-validated 2026-05-10).docs/superpowers/plans/2026-05-10-codex-env-mcp.md— 7 tasks, all complete, executed via subagent-driven-development with per-task spec + code-quality reviews.This branch supersedes the env-mcp slice that was previously planned inside the 2026-05-05 codex-app-gateway runtime plan (now marked OBSOLETE).
Test Plan
go test ./internal/codexappgateway/envmcp/ ./cmd/codex-app-gateway/— 21/21 PASSgo test -race ./internal/codexappgateway/envmcp/— cleango test -tags integration ./internal/codexappgateway/envmcp/ -run TestRun_AgainstRealCodexExecServer— PASS against realcodex0.128.0go vet ./cmd/codex-app-gateway/ ./internal/codexappgateway/envmcp/— cleanNotes for the reviewer