feat(ws): pipeline each connection's requests behind ordering barriers - #210

Draft for visibility. Stacked on the release checkpoint 1 branch (fm/il-rc1); it will be retargeted to main once checkpoint 1 lands. Plan item W02 (partial #132).

What it does

Each /ws connection becomes one loop that owns its read half, the single writer, its subscriptions and a bounded request pipeline. Requests run as pipeline tasks, so a long query no longer holds up the connection's notifications and subscription deltas.

• Ordering barriers. Programs made only of queries overlap. Writes, rules, session facts/rules, .kg …, .session …, .subscribe/.unsubscribe and other meta commands run alone and in order. A pipelined insert sent after .kg use other always lands in other.
• In-order replies. Replies are released in request order, so clients without request IDs keep working unchanged.
• Bounded. At most http.rate_limit.ws_max_in_flight_requests (default 16) requests can be unanswered at once; at that bound the loop stops reading the socket.
• Subscribe off the loop. .subscribe evaluates its snapshot in a task. Commits that land meanwhile are recorded and re-evaluated after registration, so no delta is lost.
• Disconnect. Running queries are abandoned; a started write finishes before the session closes.
• No new locks. All state is owned by the connection task. Query classification is lexical (no extra parse).

Docs: WebSocket guide, AsyncAPI spec, configuration reference, TESTING.md.

Tests

• Pipeline unit tests (ordering, barriers, capacity, panic release, shutdown abort/drain).
• tests/ws_pipeline_tests.rs, in-process:
  • a query running ≥500 ms while an external write produces a delta before the query's reply;
  • KG switch + insert ordering behind a running query (fails if every program is treated as a concurrent query);
  • session and subscription command ordering;
  • bounded pipeline;
  • disconnect cleanup;
  • saturated mixed query/write/delta traffic across 4 subscribers.
• New required e2e_reactive scenario: an agent receives deltas while its own long query runs (release; writer ack to delta ≤250 ms).

Performance

Depends on TCP_NODELAY from #200. Because pushes now go out while a request runs, a writer's own persistent_update often precedes its reply. Under Nagle that delays the write ack by ~40 ms (perf gate: insert ack p50 10.6 → 43 ms without it).

Requests are futures polled by the connection task (no task per request); results over 256 rows are serialized on the blocking pool; the idle timeout is one pinned timer.

Perf gate vs fm/il-rc1, final code with #200's change applied, 10 rounds: no FAIL, 15 required metrics PASS and 5 INCONCLUSIVE. The INCONCLUSIVE point estimates are ≤ 1.014; their intervals straddle the budget from noise.

• cheap_query (20 rounds): p50 1.034 (1.014..1.045), p99, qps and concurrent all PASS.
• Writes are far faster than baseline: insert_single ack p99 42.1 → 4.9 ms, facts/s 47 → 627.
• Deltas unchanged.

Also run: make test-fast, make e2e-reactive (11/11), full .iql.out snapshot suite (1149/1149, also as a 96-worker stress run), make oracle-test.

Tracker: #199