Repository navigation
feat(protocol): share one evaluation among standing queries that differ only in bound constants, and cache their plans - #254
Merged
Merged
Conversation
…ations Each standing-query re-evaluation recompiled its query with every persistent rule it depends on (parse, specialization, SIP, Magic Sets, IR, optimize): about 6.9 ms per evaluation on the voice-agent pack, repeated for every view on every commit. The engine now splits compilation from execution (IQLEngine::compile_program and execute_compiled_profiled). Snapshots keep their parsed persistent rules in a PersistentRules value that the next snapshot shares when the rules are unchanged. That value also holds the plans compiled against those rules, keyed by query program and optimizer passes. Standing queries (Handler::query_snapshot) look their plan up there; a rule change publishes new PersistentRules with no plans. Lookups are lock-free (ArcSwap, copy-on-write insert, bounded at 256 plans per rule set). Ad-hoc queries are unchanged. RuleCatalog::all_rules now lists rules in a deterministic order, so snapshots of the same rules have the same rule text (they differed by HashSet order). Sharing the parsed rules also saves re-parsing them on every commit.
…riables with The planner orders a rule's joins along a spanning tree of its join graph, then builds them left-deep in that order. It used a post-order traversal, which joins all of a node's children before the node itself. For a hub with two spokes that share nothing, such as shipment(O, S), promised(O, P), eta(S, D), it joined promised and eta first: a cross product of |promised| x |eta| rows. SIP generates exactly this shape for the voice-agent pack's late/on_time rules, so their semijoin rule grew quadratically with the number of sessions: 11 ms of a 37 ms evaluation at 100 sessions. The full speech query took 83 ms at 200 sessions and now takes 35 ms. The joins are now built in pre-order: each relation comes after its parent in the tree, so it always shares a variable with what is already joined.
…in bound constants
With one session-bound subscription per session on one knowledge graph
(?speech("s-1", ...), speech_owner("s-1", "sp-1", 1), and the same for
s-2 ...), every commit re-evaluated every session's view. Each evaluation
cost about as much as the unbound query, so CPU grew with sessions x commits
x tenant size. The voice-agent pack saturated a 32-core host between 25 and
50 sessions.
Such views now form a family. A query's string constants in positive atoms,
and its integer constants that no other positive atom repeats, are lifted to
parameters (?speech(_L0, ...), speech_owner(_L0, _L1, _L2)). The family
evaluates the lifted query once per revision (a round, lock-free: ArcSwap
plus a OnceCell per round). Each view takes the rows of its own constants,
projected to its own columns, and diffs them against its previous result
as before. Rows are matched as the engine matches the constant: strings
exactly, integers numerically (int, timestamp, float within tolerance).
Subscribers, deltas, revisions, authorization and the shared-view state
machine are unchanged.
A view evaluates its own query when it starts and after every rule change,
so its errors stay its own. It also does when a round fails, is truncated,
or holds more rows for it than max_result_rows. A family with one binding
does not share. A family whose round costs more than twice a view's own
evaluation and saves less than half of all views' own evaluations stops
sharing, and probes again after 256 own evaluations.
http.rate_limit.subscription_share_parameterized (default true) turns it
off.
On the voice-agent pack at 1 receipt/s per session (red-team harness):
25 sessions use 0.51 instead of 17.4 cores; 100 sessions deliver at
p50/p95/p99 40/64/81 ms on 1.6 cores (baseline: 896/1151/1151 ms,
saturated).
perf-gate sessions (make bench-sessions, scripts/bench-sessions.sh) rebuilds the hostile review's load harness as a repeatable benchmark. It loads the article's Appendix A pack verbatim, seeds N sessions, and gives each session a connection with its bound speech subscription. One executor connection subscribes to work. Every session commits a result-neutral receipt about once per second, while an adapter replaces random sessions' ETAs and times the commit and that session's delta. Each size runs on a fresh server. The run fails on a missing or stray delta or an error, and optionally on a write-to-delta p99 budget. --baseline-rev measures a baseline commit (cached with the gate's baselines) on the same sizes, and --mode shared-view measures the review's client-side routing comparison. The client gains API-key authentication: concurrent password logins from one address are throttled by design. Server specs gain extra environment for benchmarks, and running servers report their CPU time.
…onversion in sharing cost A family's latest round kept its snapshot until the next round replaced it, so an idle family pinned old data. The round now hands its snapshot to the evaluation that takes it. The cost that decides whether sharing pays also counts converting the rows and partitioning them, not only engine time: a lifted query with a wide result is no longer judged cheaper than it is.
With more distinct standing queries than plan slots (for example more than 256 sessions evaluating their own queries), clearing the whole map on every overflow left most lookups missing. Evicting one arbitrary plan keeps the rest.
…milies by parameter kinds
…ll hosts by shape
…or the sharing switch
This was referenced Oct 4, 2026
This was referenced Oct 4, 2026
jsam
force-pushed
the
fm/il-session-scale-2
branch
from
October 4, 2026 19:33
05d720f to
71ebbd2
Compare
jsam
changed the title
…a test fixture. It was a timing problem in the test, not a bug in the engine. `long_query_does_not_delay_deltas_on_its_connection` (`tests/ws_pipeline_tests.rs`) needs the cyclic triangle query on the complete bipartite graph to run for at least 500 ms (`LONG_QUERY`), so the test can show that deltas aren't held up behind it. On CI it finished in 390 ms. Locally, at the old size, it took about 620 ms, so the margin was always thin. This PR's join-planner change (pre-order joins in `src/join_planning/mod.rs`, which avoids cross products) likely made the query faster still. I didn't time it on the base commit to confirm. **What must hold:** the triangle query has to run well past every timing budget the tests measure against. Two places rely on it: `tests/ws_pipeline_tests.rs` (the 500 ms floor, the 300 ms idle-timeout test, and the disconnect test), and `tests/ws_cancel_tests.rs`, which uses the same graph and needs the query still running when its cancels arrive at 100–300 ms. **Fix:** in both files I raised the debug-build graph size `SIDE` from 10 to 14. The release-build value (20) is unchanged and I didn't time it; CI runs debug builds. Locally the query now takes 2.3 s instead of 0.62 s, so CI should see roughly 1.4 s, which is a wide margin. No engine code changed. **Verification:** I ran `cargo test --all-features --test ws_pipeline_tests --test ws_cancel_tests` under systemd-run with MemoryMax=16G, MemorySwapMax=0, CARGO_BUILD_JOBS=4, CARGO_PROFILE_DEV_DEBUG=0 and CARGO_INCREMENTAL=0. All 9 pipeline tests and all 6 cancel tests passed. I didn't re-run the rest of the workspace suite or any benchmarks
This was referenced Oct 4, 2026
jsam
added a commit
that referenced
this pull request
Oct 5, 2026
… suite, and publish engine baselines (#273) * feat(perf-gate): run the gate on the dedicated benchmark host (scripts/perf-gate-remote.sh) * feat(perf-gate): engine suite fixtures (rules, closure, deletes and updates, claims, .why, sessions, memory, recovery, async WAL) and a summary command * feat(perf-gate): make perf-gate-remote, make bench-engine-remote, summary.md and --no-verdict * docs(perf-gate): heavy perf runs go to the benchmark host; document the engine suite * fix(perf-gate): record the host's CPU count, not the pinned gate's affinity; make perf-gate-remote.sh safe to edit while it runs * fix(perf-gate): summary counts runs, not rounds * fix(perf-gate): measure facts and graphs memory on separate servers, so freed memory cannot hide growth * feat(perf-gate): perf-gate-remote.sh --wait-lock queues behind another benchmark on the host * docs(perf-gate): engine baseline 2026-10-04 on sam-dev-benchmarks, A/A calibration records; summary skips empty tables * feat(perf-gate): perf-gate-remote.sh --bench sessions runs #254's session-scale benchmark on the host * feat(perf-gate): make bench-sessions-remote * fix(perf-gate): perf-gate-remote.sh also waits out servers of a benchmark that skipped the lock * fix(perf-gate): bench-sessions tells late deltas from missing ones (schema sessions/v2) A probe whose delta took longer than the 10 s probe timeout counted as missing, and its delta, when it came, as stray, so a saturated server looked like one that loses deltas. Overdue probes now wait for their delta (and 60 s more after the load stops); one that arrives counts as late with its latency. * docs(perf-gate): session-scale results on the benchmark host (100-1,000 sessions at 1 and 0.1 writes/s); late, not lost, deltas at saturation * no-mistakes(review): Fail budget on late deltas; harden remote copy-back and runner * no-mistakes(review): Untrack perf-gate build artifacts and ignore perf-gate/target * no-mistakes(document): Document late deltas failing the sessions latency budget
jsam
added a commit
that referenced
this pull request
Oct 5, 2026
The three plan specs that #254's join-planner change broke (33_meta/17_debug_join, 18_debug_recursive, 26_debug_complex) were regenerated by #282 and pass on main. The new plans are correct: joins follow body order, the identity Maps are gone, and 26's last rule no longer builds a cross product (Join[L:[], R:[]]) before joining on PId. They still ran only in the full suite, so a plan change could land unnoticed. The fast gate's test job now runs them (make plan-snapshots) with the dev binaries cargo test already built, so no second build. - run_snapshot_tests.sh: --profile picks the binaries' cargo profile (default release); a filter that matches no spec fails instead of passing vacuously. - ci.yml path filter includes the runner script. Closes #290
jsam
added a commit
that referenced
this pull request
Oct 5, 2026
The three plan specs that #254's join-planner change broke (33_meta/17_debug_join, 18_debug_recursive, 26_debug_complex) were regenerated by #282 and pass on main. The new plans are correct: joins follow body order, the identity Maps are gone, and 26's last rule no longer builds a cross product (Join[L:[], R:[]]) before joining on PId. They still ran only in the full suite, so a plan change could land unnoticed. The fast gate's test job now runs them (make plan-snapshots) with the dev binaries cargo test already built, so no second build. - run_snapshot_tests.sh: --profile picks the binaries' cargo profile (default release); a filter that matches no spec fails instead of passing vacuously. - ci.yml path filter includes the runner script. Closes #290
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Keep fixing bugs and QCing the engine. Millisecond query and update latency must not regress; no new locks on hot paths. (Context: a hostile red-team of the voice-agent reference architecture measured the engine under that design, full method and data in ~/firstmate/data/voice-arch-novelty/report.md §6.2 and its appendices: with one session-bound standing subscription per session in one tenant graph, every commit re-evaluates every session's view, each evaluation costs about as much as the whole tenant (session bindings do not restrict work for non-recursive packs; Magic Sets only rewrites recursive relations), and ~6.9 ms of each re-evaluation is recompilation with no plan cache. Result: about 10 sessions fit a voice budget (write->delta p99 <= 100 ms); 50 sessions give p50 214 ms; 200 give 3.6 s. One shared unbound view with client-side routing cut CPU 16-27x and latency 2-24x.)
What Changed
.subscribe) that differ only in string or integer constants of positive atoms (for example one per session,?speech("s-1", G, T)and?speech("s-2", G, T)) now form a family (src/protocol/subscription/parameterized.rs). The family evaluates the lifted query once per commit and gives each view exactly the rows for its own constants, so deltas, revisions, errors and access checks stay per subscription. Sharing adjusts to latency. A family shares only when it has more views than query threads, and only after a background probe shows the shared evaluation is no slower than the views' own evaluations run in parallel. The probe uses its own permit, so no delta waits for it. A family stops sharing when shared rounds are slower on average and tries again later with backoff. Rule changes reset this decision, and a family never shares when a constant binds a relation that reads a recursive relation. The new[subscriptions] share_parameterizedsetting (defaulttrue, overridable withINPUTLAYER_SUBSCRIPTIONS__SHARE_PARAMETERIZED) turns the feature off.src/storage_engine/snapshot.rsevicts one plan at the cap instead of clearing all of them. An evaluated round can drop its snapshot. Join planning (src/join_planning) no longer joins a relation before the one it shares variables with.perf-gatesession-scale benchmark on the voice-agent pack, run withscripts/bench-sessions.sh/make bench-sessions. Also adds unit, plan-cache and standing-query integration tests, and documents the feature in the configuration and WebSocket API guides.Risk Assessment
Testing
I built the server at the target commit under the 16G cap and ran it as a real process on 4 CPUs in disposable data and home directories. A Node WebSocket driver ran 40 session-bound subscriptions through targeted writes, cross-session moves, retractions, in-flight rule changes and a recursive bound pack, with sharing on and off. Every session's accumulated deltas always equaled its one-off query, and no session received another session's rows. With sharing on, write-to-delta was about 22 ms p50 and server CPU about half of the sharing-off run. The first rule-change run failed because of my driver: it stopped reading once the rows happened to match, while valid interim deltas were still queued. After fixing the driver to drain until quiet, three runs with sharing on and one with it off passed. The stale-rules guard's internal decision isn't visible on the running server, so I didn't validate it live; only unit tests cover it. The targeted stale-rules unit tests and the parameterized integration tests pass. No benchmarks or full suite were run, per the recorded instructions. All servers are stopped, the temporary directories are removed, and the worktree is clean.
Evidence: 40 sessions, sharing on: exact deltas reach only the written session, ~22 ms p50
Evidence: 40 sessions, sharing off by env override: same correctness, ~192 ms p50, about 2x CPU
Evidence: Rule changes interleaved with writes during in-flight rounds (run 1)
Evidence: Rule changes interleaved with writes during in-flight rounds (run 2)
Evidence: Rule changes interleaved with writes during in-flight rounds (run 3)
Evidence: Rule changes in flight with sharing off (control)
Evidence: First rule-change attempt: failed from a driver bug (stopped reading before the queued interim deltas arrived)
Evidence: Bound recursive views stay exact
Evidence: WebSocket driver used for all scenarios
Evidence: Server launcher (memory cap, taskset, env overrides)
Evidence: Parameterized unit tests (stale-rules guard)
Evidence: Parameterized end-to-end integration tests
Pipeline
Updates from git push no-mistakes
... (4 earlier update rounds omitted to keep the PR body within GitHub's 65536-char limit; full history is in the run log.)
🔧 Fix applied.
2 warnings still open:
src/config.rs:514- Simplification: the new config optionhttp.rate_limit.subscription_share_parameterized(default true, read at src/protocol/subscription/connection.rs:98, documented in config.example.toml, docs/guides/configuration.md and websocket-api.mdx) is an added mode. The intent (fix session-scale cost without regressing latency) doesn't require it, because the family's own latency guard already turns sharing off per shape. Smallest remedy: remove the option, unless the author wants it as an operator kill switch for a default-on heuristic.src/config.rs:800- The second fix round (8dedd65) moved the sharing kill switch into a new top-level[subscriptions]section, butConfig::env_sourceonly passesINPUTLAYER_*keys from the sections listed inSECTIONS = ["storage", "optimization", "logging", "http"](src/config.rs:800). As a result,INPUTLAYER_SUBSCRIPTIONS__SHARE_PARAMETERIZED=falseis silently dropped and sharing stays on. Before the move, the same switch could be set throughINPUTLAYER_HTTP__RATE_LIMIT__SUBSCRIPTION_SHARE_PARAMETERIZED. docs/guides/configuration.md also says "All config options can be overridden with environment variables". This matters most in container deployments, where an operator would most likely flip the default-on heuristic through an environment variable, and it fails with no error. Fix: add "subscriptions" toSECTIONSinenv_source. Same class, also at src/config.rs:786: theseed()minimal config has no"subscriptions": {}entry, even though its comment says every nested section must appear there. This is harmless today becauseSubscriptionsConfig::default()matches the field default, but add the entry for consistency.🔧 Fix applied.
✅ Re-checked - no issues remain.
src/protocol/subscription/parameterized.rs:617- On any host with 3 or more compute permits (the default on a typical server),Family::probenever consultsbinds_recursion. That check only runs on thecompute_permits() < MIN_PERMITS_FOR_PROBESbranch (lines 618-624). So a family of recursion-bound views such as?reach("n1", Y),?reach("n2", Y), … with more bindings than permits makes the server evaluate the lifted, unbound?reach(_L0, Y)in a background probe. That is the full transitive closure, which is quadratic or worse. Base never computes it: each bound view stays restricted by Magic Sets. The testa_probe_too_slow_to_share_never_makes_a_view_wait(tests.rs:379) shows this happening on a 400-node chain. The probe's only bounds arequery_timeout_ms(default 30 s) andmax_result_rows. Neither caps the fixpoint's intermediate memory, and there is no per-query memory limit yet (issue Engine: per-graph memory budget and per-query memory limit, refuse with resource_exhausted (EN-9) #262). The attached verification plan names exactly this unbound?reachclosure as what OOM-crashed the devbox (~115 GiB), and its adversarial evidence covers only 2 permits, where the shape check prevents the probe. After a failed probe the family re-probes after 256×bindings own evaluations, with backoff, so the exposure repeats for the life of the subscriptions. Smallest remedy: apply the existingbinds_recursion(&shape, &snapshot.rules)check before any probe, whatever the permit count, so recursion-bound shapes are decided by shape and never probed. Doing that reverses the earlier fix round 8ac7d51's choice to decide by shape only on small hosts, so the author needs to confirm it.src/protocol/subscription/parameterized.rs:547- The latency guard compares against cost estimates measured under the old rules after a rule change. On a rule change, every view sees avalidatedmismatch and evaluates its own query. That evaluation compiles a plan, sorecord_own(None)leavesown_cost_usunchanged (line 600).sharingandshared_cost_usare not reset either. On the next commit the views read a round under the new rules, andkeeps_sharing(line 551) judges it against an own cost measured under the old rules. Own cost is refreshed only by one sample everySAMPLE_EVERY=64 rounds, at 1/8 EWMA weight. Concrete case: 200 bindings on 4 permits, old own cost 5 ms, so the unshared bound is 250 ms. A rule change makes bound evaluations cheap (for example, the read relation becomes recursive so Magic Sets restricts it to ~0.1 ms) while the lifted round takes 200 ms. The family keeps sharing at 200 ms per delta against ~5 ms unshared, about 40× slower, until two own samples (~128 commits) pull the average under the threshold. That violates "latency must not regress" with no error. The defect is in the guard that the earlier fix round 322e26b introduced (the EWMA replacing STOP_MARGIN_PERCENT); that round did not handle rule changes. Fix: tie the costs to the rules they were measured under. When a round's or own evaluation'sPersistentRulesdiffers from the last-measured one, resetown_cost_us/shared_cost_usand stop sharing without counting a failed probe (do not bumpstops), so the family re-probes once a plan-cached own cost under the new rules is known.🔧 Fix applied.
3 issues (1 error, 2 warnings) still open:
src/protocol/subscription/parameterized.rs:617- On any host with 3 or more compute permits (the default on a typical server),Family::probenever consultsbinds_recursion. That check only runs on thecompute_permits() < MIN_PERMITS_FOR_PROBESbranch (lines 618-624). So a family of recursion-bound views such as?reach("n1", Y),?reach("n2", Y), … with more bindings than permits makes the server evaluate the lifted, unbound?reach(_L0, Y)in a background probe. That is the full transitive closure, which is quadratic or worse. Base never computes it: each bound view stays restricted by Magic Sets. The testa_probe_too_slow_to_share_never_makes_a_view_wait(tests.rs:379) shows this happening on a 400-node chain. The probe's only bounds arequery_timeout_ms(default 30 s) andmax_result_rows. Neither caps the fixpoint's intermediate memory, and there is no per-query memory limit yet (issue Engine: per-graph memory budget and per-query memory limit, refuse with resource_exhausted (EN-9) #262). The attached verification plan names exactly this unbound?reachclosure as what OOM-crashed the devbox (~115 GiB), and its adversarial evidence covers only 2 permits, where the shape check prevents the probe. After a failed probe the family re-probes after 256×bindings own evaluations, with backoff, so the exposure repeats for the life of the subscriptions. Smallest remedy: apply the existingbinds_recursion(&shape, &snapshot.rules)check before any probe, whatever the permit count, so recursion-bound shapes are decided by shape and never probed. Doing that reverses the earlier fix round 8ac7d51's choice to decide by shape only on small hosts, so the author needs to confirm it.src/protocol/subscription/parameterized.rs:547- The latency guard compares against cost estimates measured under the old rules after a rule change. On a rule change, every view sees avalidatedmismatch and evaluates its own query. That evaluation compiles a plan, sorecord_own(None)leavesown_cost_usunchanged (line 600).sharingandshared_cost_usare not reset either. On the next commit the views read a round under the new rules, andkeeps_sharing(line 551) judges it against an own cost measured under the old rules. Own cost is refreshed only by one sample everySAMPLE_EVERY=64 rounds, at 1/8 EWMA weight. Concrete case: 200 bindings on 4 permits, old own cost 5 ms, so the unshared bound is 250 ms. A rule change makes bound evaluations cheap (for example, the read relation becomes recursive so Magic Sets restricts it to ~0.1 ms) while the lifted round takes 200 ms. The family keeps sharing at 200 ms per delta against ~5 ms unshared, about 40× slower, until two own samples (~128 commits) pull the average under the threshold. That violates "latency must not regress" with no error. The defect is in the guard that the earlier fix round 322e26b introduced (the EWMA replacing STOP_MARGIN_PERCENT); that round did not handle rule changes. Fix: tie the costs to the rules they were measured under. When a round's or own evaluation'sPersistentRulesdiffers from the last-measured one, resetown_cost_us/shared_cost_usand stop sharing without counting a failed probe (do not bumpstops), so the family re-probes once a plan-cached own cost under the new rules is known.src/protocol/subscription/parameterized.rs:589- Fix round 64f1a3c added the rule-change reset (Family::binds_recursion, lines 484-502). That fix only handles new evaluations; a round evaluation already running when the rules change can undo the reset.evaluatechecks the shape once, beforerun_query. It then judges the finished round against whateverown_cost_usholds at that moment (lines 589-610).Concrete sequence, family sharing under rules R1 with a ~200 ms round:
record_own. That resetssharing=falseandown_cost_us=0.own_cost_us=0,keeps_sharingreturns true ("an unknown own cost keeps sharing"), so it setssharing=trueand puts an R1 cost intoshared_cost_us.The views then read R2 rounds with no measured own cost, and the guard can't stop them. Every round keeps sharing until a sample arrives after SAMPLE_EVERY=64 rounds. A probe spawned with a pre-change snapshot causes the same thing: its
evaluateflipsjudgedback to R1, resets the costs, and then shares on own=0.This is the same failure the stale-costs-across-rule-change fix was meant to close (sharing under new rules without a comparison measured under them). It is still reachable, and it is now worse, because 0 means "always keep sharing" where the old stale cost at least gave a comparison. Recursion-bound shapes stay safe because the pre-check refuses them; the exposure is latency for non-recursive shapes. A round that fails under the old rules has a similar side effect: it calls
stop_sharing, which counts a failure the reset had just cleared.Fix at the judging step in
evaluate: afterrun_query, skipsmooth,keeps_sharing,stop_sharingandsharing.store(true)unlessself.judgedstill holds the round'srules(Arc::ptr_eq). Also, don't let a round start sharing whileown_cost_us == 0.🔧 Fix applied.
1 info still open:
src/protocol/subscription/parameterized/tests.rs:486- The new testa_round_from_before_a_rule_change_does_not_share_againonly exercises theown > 0half of the fix in 6cf7687. The stale probe callsbinds_recursion(before)at the start ofevaluate, which setsjudgedback to the old rules, socurrentis true when the round finishes. Onlyown_cost_us == 0keeps the family from sharing. TheArc::ptr_eq(judged.rules, rules)check afterrun_query(parameterized.rs:576-580) is never reached as the deciding guard. The test would still pass if that check were removed. Removing it would let this sequence happen undetected: a round at R1 is in flight, R2 is judged, an R2 own cost is recorded (>0), then the R1 round finishes, is compared against the R2 own cost, and re-arms sharing. To cover it, holdprobe_gateso an R1 round evaluates pastbinds_recursion, then switch to R2 and record an R2 own cost before releasing the gate. Assert!family.shares()andshared_cost_us == 0.🔧 Fix applied.
2 issues (1 warning, 1 info) still open:
src/protocol/subscription/parameterized.rs:672- The judging guard only moves forward on the round path. The own-cost path still lets a cost measured under the old rules into the state for the new rules. This is a sibling left behind by fix rounds 64f1a3c (reset costs on rule change), 6cf7687 (discard stale round verdicts) and 57b4aba (forward-only judging). After 57b4aba,binds_recursionreturns early for a snapshot older than the judged revision and does not reset anything.record_own(line 672) then callssmooth(&self.own_cost_us, cost)with that snapshot's cost anyway. Concrete sequence: a family with 200 bindings on 4 permits, under rules R1. View A takes an R1 snapshot (reevaluate, line 811) and evaluates its own query with a cached plan, queued behind permits. A rule change to R2 commits. View B's own evaluation under R2 callsrecord_own, which judges R2 and resetsown_cost_usto 0. A finishes and callsrecord_own(Some(R1 cost), R1 snapshot). The stale-revision branch skips the reset, andsmoothsetsown_cost_usto the R1 cost, because the previous value of 0 means it is taken verbatim. With stops==0, the next plan-cached R2 own evaluation returns true and probes at R2. The round is judgedcurrent, so it is compared against an average that is still mostly the R1 cost ((7·R1+R2)/8). If R1 own evaluations were expensive and R2 ones are cheap, the family starts sharing and keeps sharing, because once views read rounds the own cost is refreshed only every SAMPLE_EVERY=64 rounds at 1/8 weight. That is the exact latency regression the stale-costs fix was meant to prevent. Fix at the same boundary: inrecord_own, afterbinds_recursion, return false without callingsmoothunlessself.judgedholdssnapshot.persistent_rules()(Arc::ptr_eq), mirroring thecurrentcheck inevaluate. Add a test: judge R2 (own reset), callrecord_own(Some(cost), before), and assert thatown_cost_usstays 0 and the call returns false.src/protocol/subscription/parameterized.rs:605- Inevaluate,currentis read once afterrun_query(lines 595-599). If a rule change is judged (and the costs reset) between that read andsharing.store(true)/smooth(&self.shared_cost_us, …), a round from the old rules can still re-arm sharing. The window is a few instructions wide, so this is very unlikely in practice. Re-checking the judged rules (Arc::ptr_eq) just beforesharing.store(true)would narrow the window further, without adding a lock.🔧 Fix applied.
3 issues (1 warning, 2 infos) still open:
src/protocol/subscription/parameterized.rs:672- The judging guard only moves forward on the round path. The own-cost path still lets a cost measured under the old rules into the state for the new rules. This is a sibling left behind by fix rounds 64f1a3c (reset costs on rule change), 6cf7687 (discard stale round verdicts) and 57b4aba (forward-only judging). After 57b4aba,binds_recursionreturns early for a snapshot older than the judged revision and does not reset anything.record_own(line 672) then callssmooth(&self.own_cost_us, cost)with that snapshot's cost anyway. Concrete sequence: a family with 200 bindings on 4 permits, under rules R1. View A takes an R1 snapshot (reevaluate, line 811) and evaluates its own query with a cached plan, queued behind permits. A rule change to R2 commits. View B's own evaluation under R2 callsrecord_own, which judges R2 and resetsown_cost_usto 0. A finishes and callsrecord_own(Some(R1 cost), R1 snapshot). The stale-revision branch skips the reset, andsmoothsetsown_cost_usto the R1 cost, because the previous value of 0 means it is taken verbatim. With stops==0, the next plan-cached R2 own evaluation returns true and probes at R2. The round is judgedcurrent, so it is compared against an average that is still mostly the R1 cost ((7·R1+R2)/8). If R1 own evaluations were expensive and R2 ones are cheap, the family starts sharing and keeps sharing, because once views read rounds the own cost is refreshed only every SAMPLE_EVERY=64 rounds at 1/8 weight. That is the exact latency regression the stale-costs fix was meant to prevent. Fix at the same boundary: inrecord_own, afterbinds_recursion, return false without callingsmoothunlessself.judgedholdssnapshot.persistent_rules()(Arc::ptr_eq), mirroring thecurrentcheck inevaluate. Add a test: judge R2 (own reset), callrecord_own(Some(cost), before), and assert thatown_cost_usstays 0 and the call returns false.src/protocol/subscription/parameterized.rs:605- Inevaluate,currentis read once afterrun_query(lines 595-599). If a rule change is judged (and the costs reset) between that read andsharing.store(true)/smooth(&self.shared_cost_us, …), a round from the old rules can still re-arm sharing. The window is a few instructions wide, so this is very unlikely in practice. Re-checking the judged rules (Arc::ptr_eq) just beforesharing.store(true)would narrow the window further, without adding a lock.src/protocol/subscription/parameterized.rs:583- Fix round 71ebbd2 missed one sibling write. That round gated every cost/sharing write on the judged rules, butevaluatestill runsself.sharing.store(false)unconditionally wheneverbinds_recursion(&snapshot)is true. For a snapshot older than the judged revision,binds_recursionreturns the verdict under that old snapshot's rules (thejudged.revision > snapshot.revisionbranch), so a stale round changes sharing state under the current rules. Concrete sequence: rules R1 make a parameter bind a recursive relation. A rule change to R2 removes the recursion, and the family is judged and starts sharing under R2. A view that last validated under R1 then reads with an R1 snapshot, and no newer round exists yet.round_forcreates an R1 round,evaluatesees binds_recursion=true under R1, and sharing is cleared under R2 without resetting costs or counting a stop. The next own evaluation then probes again. The result is wasted probe CPU and own evaluations, not a latency regression, but it breaks the stated invariant that snapshots older than the judged revision never update anything. Fix: only store false whenself.judges(snapshot.persistent_rules()). For example, return the Err without writing when the snapshot's rules are not the judged ones.✅ **Test** - passed
✅ No issues found.
falsestarts the HTTP listenercargo test --test standing_query_tests parameterized(5 passed), which runs an in-process axum server from…perf-gate sessions --server <head release> --compare <base 0926f0f0 release> --sessions 1,3,10,50,100,200 --load-secs 10undersystemd-run --user --scope -p MemoryMax=16G -p MemorySwapMax=0perf-gate sessions --server <head> --server-env INPUTLAYER_SUBSCRIPTIONS__SHARE_PARAMETERIZED=false --sessions 10,50(env kill switch)perf-gate sessions --server <wrapper writing config.local.toml [subscriptions] share_parameterized=false> --sessions 50(TOML kill switch)Manual server start with[subscriptions]containing a misspelled key, a non-boolean value, and a validfalsecargo test --test standing_query_tests parameterized(5 WebSocket end-to-end tests against an in-process server on loopback)cargo test --lib -- subscription::parameterized test_env_override_switches_parameterized_sharing_off(includes a_family_whose_views_fit_on_the_compute_permits_never_shares)✅ No issues found.
systemd-run --user --scope -p MemoryMax=16G -p MemorySwapMax=0 env CARGO_BUILD_JOBS=4 CARGO_PROFILE_DEV_DEBUG=0 CARGO_INCREMENTAL=0 cargo build --bin inputlayer-serverrun.sh shared shared 40: real server (memory-capped, taskset 0-3), 40 WebSocket sessions each subscribed to?view("sN", X, "a"); warm-up writes, 30 single-session writes with a delta check and quiet check on all other sessions, a move between sessions and a retraction, each compared against the one-off queryrun.sh off shared 40 INPUTLAYER_SUBSCRIPTIONS__SHARE_PARAMETERIZED=false: the same scenario with sharing switched off by envrun.sh rules rules 40x3: rule add, clause remove and full rule delete interleaved with writes and not awaited (rounds in flight); then drain every delta strictly, compare with the one-off query, then a targeted write and a quiet checkrun.sh rulesoff rules 40 INPUTLAYER_SUBSCRIPTIONS__SHARE_PARAMETERIZED=falserun.sh rec recursive 40: 20 bound?reach("nK", Y)subscriptions over a 40-node chain; edges added, removed and cut, each converged against the one-off querycargo test --lib protocol::subscription::parameterized(23 tests, including an_own_cost_from_before_a_rule_change_is_dropped and a_round_from_before_a_rule_change_does_not_share_again), under the memory capcargo test --test standing_query_tests parameterized(5 end-to-end server tests), under the memory cap✅ **Document** - passed
✅ No issues found.
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
✅ No issues found.