Rebuild koshell as a hybrid Rust + Node monorepo#1
Merged
Conversation
Rebuild koshell around the hybrid architecture: a Cargo workspace for the Rust terminal-core (crates/koshell-rs, crates/koshell-proto) alongside a pnpm workspace for the Node AI daemon (packages/ai-daemon). The pre-rewrite TypeScript prototype and its docs are frozen under reference/. Adds rust-toolchain pin, CI workflow, and refreshed README/AGENTS/docs. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add the koshell-rs interactive session: shell resolution, PTY env construction, and nested-start guard ported from the frozen prototype, plus a portable-pty session that forwards stdin/stdout, propagates SIGWINCH resizes, and exits with the child's code. Reaches transparent shell-wrapper parity. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add the terminal-core data layer: an alacritty_terminal-backed headless mirror with plain-text snapshots and alt-screen detection, the append-only timeline store with its queries, line-level screen diffing, and terminal context selection. Semantics match the frozen prototype, covered by ported unit tests. Restructure koshell-rs as a lib plus thin binary. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Generate temporary bash/zsh rc files that emit OSC 777 command-boundary markers, and parse them out of the PTY stream with a chunk-boundary-safe scanner so they never leak to the terminal. Rework the session around a single processor thread that records PTY output, human input, and screen snapshots into the timeline, tracks command spans, and on a #? command assembles the terminal context package (placeholder inline ack until the daemon is wired). Clamp zero-sized PTYs to avoid an emulator panic. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Send #? requests to the AI daemon over newline-delimited JSON on a Unix domain socket, connecting lazily and degrading gracefully when the daemon is absent. On a #? the session sends a hello handshake then the ai_request carrying the assembled context package, and acknowledges inline whether it reached the daemon. Socket path follows XDG conventions. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Implement the JSONL Unix-socket receiver: an incremental NDJSON decoder, a protocol parser for hello/ai_request/bye, and a server that logs #? requests with their context package and acks them. Socket path resolution mirrors the Rust side. Covered by framing and protocol unit tests. pi integration is the next stage. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
License the project under Apache-2.0 (LICENSE plus license metadata in the workspace manifests) and add contributor-facing files for public contribution: CONTRIBUTING, a Code of Conduct, a pull request template, and bug/feature issue templates. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
pnpm/action-setup needs an explicit version because the repo omits the packageManager field. Pin to major 11 to match the engines floor. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
A #? question asked twice in a row only triggered on the first attempt. The shell-integration hooks detected #? from shell history: bash deduped by command text, and zsh's history read was defeated entirely by hist_ignore_dups (identical repeats collapse to one entry). bash now dedups by history entry number; zsh stops consulting history and instead captures the submitted line via an accept-line widget (delegating to any pre-existing widget) plus a preexec-captured command line, immune to hist_ignore_dups/hist_ignore_space. Add real-PTY regression tests driving interactive bash and zsh (the latter reproducing the reported hist_ignore_dups config), since this class of bug only manifests in an interactive shell's line editor and cannot be reached by unit tests. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Document how #? will work inside foreground CLI programs (REPLs) where shell integration is dormant and no OSC markers exist. Records the research: PTY probes show no single universal in-band completion signal (PyREPL emits bracketed-paste and termios toggles per command; node emits neither and stays raw). Specifies a layered, confidence-ranked completion detector gated by shell command spans and non-alt-screen: bracketed-paste edges (S1) with an output-quiescence plus learned-prompt floor (S3), driving deferred #? launches. Captures prompt learning, nested-prompt pairing, timeout/degradation, known limits, and the S1+S3 MVP scope with S0/S2/OSC 133/per-program integration as future work. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Implement the S1+S3 prototype from design-0001 so #? works inside foreground REPLs where shell integration is dormant. A ReplDetector owned by SessionState is gated to run only inside a shell command span and off the alternate screen, so it never overlaps the shell OSC path. It captures a submitted #? from keystrokes, then defers the trigger until the command completes: S1 reads bracketed-paste ESC[?2004h/l edges from the output stream (readline/libedit/PyREPL), and S3 falls back to output quiescence for programs that emit no such edges (node). The deferred trigger reuses build_context_package, so the AI sees the post-completion screen. Add real-PTY regression tests exercising python (S1) and node (S3), asserting #? fires after the command's output. #? capture uses keystrokes rather than the mirror-read for now; see the design's Prototype status for that and other deferred hardening. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Design-0001 is revised after a first-principles review: #? is pinned as a marker token with one firing rule (every #? fires at the next completion/stabilization point of its line; no-op lines complete immediately), armed by echo plus non-alt-screen instead of any whitelist, with quote-parity suppression. Bracketed-paste (S1) is dropped as a load-bearing signal in favor of output stabilization with escalating debounce, a prompt-shape modulator, and a bounded max-wait; shell OSC command_end stays authoritative with no short timeout. Pending-trigger interaction is defined (delayed receipt, Ctrl+C and bare-Esc cancel, FIFO serialization) along with accepted line-semantics corners (background commands, REPL multi-line, the "#? /" command namespace). The prototype still implements the earlier S1+S3 iteration; the deltas are listed in the doc. New design-0002 records the AI-stage boundaries: the mirror consumes the full byte stream written to the user's terminal (PTY plus AI presentation output), presentation buffers whichever side is bounded, the context package pushes the question anchor plus a pullable-material inventory while deeper history stays behind tools, and the daemon holds one conversation per terminal session. architecture.md and rewrite-0001 are annotated to match (in-program #? detection supersedes the OSC-markers-only decision). Co-Authored-By: Claude Fable 5 <[email protected]>
Replace the S1+S3 prototype with the stabilization-based design of design-0001: mirror-read capture of the cursor's logical line at submit (doubling as the echo-verification arming check), quote-parity suppression, escalating debounce tiers with a prompt-shape modulator and a bounded max-wait, still-running stabilization firing for non-terminating commands, and completion metadata (form, completion authority, stillRunning, exit code) in the context package. The pending-trigger interaction lands too: a ~1s delayed receipt notice, Ctrl+C cancel with command_end re-fire suppression, and bare-Esc cancel with a 40ms continuation-timeout disambiguation in the stdin thread. Presentation output now feeds the terminal mirror (the design-0002 mirror-feed invariant, for the lines that exist today). Dogfooding immediately caught a false positive through fzf's Ctrl+R widget (rendered `#?` UI text captured at the prompt, double-firing at command_end), so the prompt layer is marker-owned: mirror capture is armed only inside command spans and in shells without integration, and the command_start marker now carries the full typed line (zsh preexec already does; the bash DEBUG trap reads it back from history), emitted once per accepted line. Real-PTY regressions cover echo arming, quote parity, still-running single-fire, and both cancel paths. Co-Authored-By: Claude Fable 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
The previous single-package TypeScript koshell was a half-finished shell wrapper: the timeline, terminal-context, ai-context, and screen-diff modules were only test-driven and never wired into the running process. This PR rebuilds the repository around the hybrid architecture recorded in the internal workspace — a native Rust foreground terminal process (
koshell-rs) plus a Node.js AI daemon (koshell-ai-daemon) that communicate over newline-delimited JSON on a Unix domain socket.Scope for this stage (agreed): complete the Rust terminal-core and a minimal Node receiver that logs and acks
#?requests. pi integration, provider configuration, the tool loop, and streaming AI responses are the next stage.Confirmed decisions: single monorepo (Cargo + pnpm workspaces);
alacritty_terminalfor terminal emulation;#?via OSC shell-integration markers only.What's included
crates/koshell-rs,crates/koshell-proto,packages/ai-daemon); the old TypeScript prototype and its docs are frozen underreference/; refreshed README/AGENTS/docs; CI workflow.portable-ptysession, stdin/stdout forwarding, resize, signals, nested-start guard (transparent shell-wrapper parity).alacritty_terminal-backed mirror with plain-text snapshots and alt-screen detection, the timeline store, line-level screen diffing, and terminal context selection (semantics ported from the prototype, unit-tested).#?detection, command spans, and live recording into the timeline.hello+ai_request(with the assembled context package).#?requests with their context package and repliesack.Verification
cargo test(35),cargo clippy -D warnings,cargo fmt --check— all green.pnpm check(format, lint, typecheck, 8 tests) — green.echo hi #? explain this outputprintshi, the OSC marker never leaks.#?, connects, and sendshello+ai_request; the daemon logs the question and full context package, then disconnects cleanly.Known limits (next stage)
visible_outputis not produced yet, soprimaryText/recentPtyOutputstill contain raw ANSI (currentScreenis clean text).hello/ai_request/ackonly; the daemon is started manually (no auto-launch).koshell_*tool round-trips, and AI streaming (ai_delta/ai_tool_call/ai_response_end) are deferred.🤖 Generated with Claude Code