Repository navigation
Conversation
An `execute` frame may carry `params`: values bound to the program's
`$name` references on its parsed statements. A value is never lexed,
parsed or written back into IQL text, so no value can change what a
program does, whatever characters it holds; the literal encoder in a
client is no longer the injection boundary.
- ws-protocol: `Params` / `ParamValue` with a strict wire typing (bare
JSON or one-key `{"int"|"float"|"string"|"bool"|"vector": ...}`).
Nothing converts silently: null (JSON.stringify's NaN/Infinity),
integers outside int64, ambiguous bare integral numbers >= 2^63,
inexact `{"float": int}`, duplicate or non-identifier names fail
the frame with `invalid_request`. Protocol version 4.
- Parsers read `$name` as `Term::Param` / `ArithExpr::Param` wherever a
literal stands; `params::bind_statement` replaces each with the
constant term its value denotes, once, before authorization and
execution. A missing, unused or misplaced parameter fails the whole
program with `validation`; meta commands take none.
- Fail closed: the IR builder, the rule catalog and every constant
consumer refuse an unbound parameter.
- Bound values are never re-parsed: queries, session rules, the
dirty-session path, conditional deletes, updates and persistent rule
registration now evaluate the parsed syntax tree instead of
formatting it to text and parsing it again (also one parse fewer
per query).
- Parameter values count toward max_query_size_bytes and are never
logged.
serde_json's default float parser can be one ULP off for some inputs (the Python SDK's round-trip fuzz found 6.076327178933334e-236), so a float parameter could bind a different value than its literal, which Rust parses exactly. Enable `float_roundtrip`, and test 20,000 random bit patterns in both shortest forms.
Inserts, deletes, conditional deletes, programs (guards, tokens and
constant rows included), claims, rule definitions and session writes
compile inside `withParams`: every value becomes a `$pN` reference and
travels in the frame's `params`, typed exactly as its literal would be
(`{float}` for integral floats JSON would turn into ints, `{int}` for
BigInts). Equal values share one name, so compiled text compares as
before; every refusal (NaN, infinities, unsafe integers, null) still
holds. Aggregate settings stay literal, and so do queries, which also
feed `.subscribe` and `.why`.
`execute(program, { params })` sends them, refusing up front on an
engine older than protocol 4, which would ignore them. Protocol 4;
`ProgramResult.params` names what was sent.
Inside `collect_params()` (a ContextVar scope, so concurrent tasks and threads never share one), `encode()` returns a `$pN` reference and collects the value, typed exactly as its literal would be. Inserts, deletes, conditional deletes, programs, claims, rule definitions and session writes send their values this way; `literal()` keeps aggregate settings and `limit` literal, and queries keep literals. `Connection.execute(..., params=...)` and `KnowledgeGraph.execute` send them, refusing up front on an engine older than protocol 4; frames refuse NaN. Program rule clauses compile at commit, inside the scope. Protocol 4; `ProgramResult.params` names what was sent.
The mapper fills a pack template's slots with parameters instead of literals: the statement text is the template, the extracted values travel beside it (`crate::iql::Stmt`). Ledger writes and reads, the conversation record, retractions and the one-shot cleanup do the same, and one evaluation is one parameterised `Program`. A parameter is named by a SHA-256 of its typed value, so equal values share one name and statements compose, and replay from the ledger, without renaming; a name bound to two values fails the request. `il_fact` keeps a JSON record of each statement's text and parameters, so a retraction replays it exactly; records from before parameters (the statement text) still replay. Values with IQL syntax (parens, brackets, quotes, `<-`, `:=`) are no longer dropped: no character can reach IQL syntax. Control characters still drop their row: they would reshape the line-per-row digest in the extraction prompt. Ledger text keeps its percent-encoded storage format, so existing ledgers stay readable. Traces and events show statements with values written as literals, for people only. The `.why` proof goal stays literal: meta commands take no parameters.
…ON floats exactly The client read 1e-7 * 1000000 = 0.09999999999999999 as 0.1 (and 6.0000000000000005e100 as 6.000000000000001e100): serde_json's default float parser can be one ULP off. With float_roundtrip on, the client shows the value the engine computed.
origin/main's engine already prints these plans (14 IR nodes, the join operands swapped); the expected files predate it. Not a change of this branch: reproduced with a build of origin/main.
# Conflicts: # docs/spec/asyncapi.yaml # src/protocol/rest/handlers/ws/request.rs # ws-protocol/src/client.rs
jsam
marked this pull request as ready for review
October 5, 2026 13:18
This was referenced Oct 5, 2026
jsam
added a commit
that referenced
this pull request
Oct 5, 2026
The three plan specs that #254's join-planner change broke (33_meta/17_debug_join, 18_debug_recursive, 26_debug_complex) were regenerated by #282 and pass on main. The new plans are correct: joins follow body order, the identity Maps are gone, and 26's last rule no longer builds a cross product (Join[L:[], R:[]]) before joining on PId. They still ran only in the full suite, so a plan change could land unnoticed. The fast gate's test job now runs them (make plan-snapshots) with the dev binaries cargo test already built, so no second build. - run_snapshot_tests.sh: --profile picks the binaries' cargo profile (default release); a filter that matches no spec fails instead of passing vacuously. - ci.yml path filter includes the runner script. Closes #290
jsam
added a commit
that referenced
this pull request
Oct 5, 2026
The three plan specs that #254's join-planner change broke (33_meta/17_debug_join, 18_debug_recursive, 26_debug_complex) were regenerated by #282 and pass on main. The new plans are correct: joins follow body order, the identity Maps are gone, and 26's last rule no longer builds a cross product (Join[L:[], R:[]]) before joining on PId. They still ran only in the full suite, so a plan change could land unnoticed. The fast gate's test job now runs them (make plan-snapshots) with the dev binaries cargo test already built, so no second build. - run_snapshot_tests.sh: --profile picks the binaries' cargo profile (default release); a filter that matches no spec fails instead of passing vacuously. - ci.yml path filter includes the runner script. Closes #290
This was referenced Oct 5, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
I want you to really hyper focus on making the symbolic engine, you know, crisp and really truly production ready. Like I really need you to, you know, put insane amount of effort into doing that. And then once you have that, I will give you the endpoints, but only then.
(Context: approved plan ~/firstmate/data/nsdl-plan/report.md §6.1, GitHub milestone 7 'Engine production readiness'. This task is item EN-5: parameterised programs (E-12) — why:
/v1/factsand decisions carry customer values; values travel out of band, so the encoder is no longer the injection boundary)The referenced item, EN-5 (GitHub issue #258 "Engine: parameterised programs, values out of band"): Goal: parameterised programs, customer values travel out of band from the IQL text, bound by the engine. Why: facts and decisions carry customer-supplied values; with parameters, the literal encoder in the gateway and SDKs is no longer the injection boundary. Done when:
executeaccepts a program with typed parameters; values never pass through the parser; string, number, NaN and infinity edge cases are refused or encoded exactly (the class of bug fixed in #251, where a NaN literal became a variable and mass-deleted rows, cannot recur); both SDKs (JS and Python) and the gateway use it for writes. Security, needed before any hosted use. Rule for every engine PR in the milestone: millisecond query and update latency must not regress.What Changed
$nameparameters wherever a literal can appear (fact arguments, rule heads and bodies, query filters, arithmetic, updates, vector search). Theexecuteframe takes aparamsmap, and the engine binds those values to the parsed statements, so a value is never lexed or parsed. Protocol version goes to 4. Values are typed exactly from their JSON form or from an explicit{"int"|"float"|"string"|"bool"|"vector": ...}tag. These are rejected withinvalid_requestand nothing runs:null(which is how JSON encodes NaN and the infinities), out-of-range ints, bare negative zero, lossy float/int conversions, bad names and duplicate names. A parameter that is missing or never referenced gets avalidationerror. Meta commands refuse$name, and the request log records how many parameters a request had, not their values. The work is mainly insrc/params.rs,ws-protocol/src/params.rsandsrc/protocol/handler.rs, with tests intests/param_program_tests.rs.gateway/src/iql.rsis used by the ledger, mapper and pipeline. Rawexecutein both SDKs acceptsparams, and both SDKs have new parameter test suites..iql.outsnapshots for floats and trig functions show exact values. The.debugsnapshots were refreshed to match the planner on main. The WebSocket API guide, AsyncAPI spec, syntax reference and SDK guides document parameters and their wire format.Risk Assessment
✅ Low: The fix round implements the recorded decision as instructed: a bare negative zero is refused,
{"int": -0}binds Int(0), Float(-0.0) serializes in the explicit form, and the Python SDK sends{"float": -0.0}. The docs now match the decided behaviour for floats beyond the u64 range, and tracing the sibling paths found no new defect. Those paths are the gateway hash names, JS -0→0, Python dedup keyed by literal text and vector elements.Testing
I started a real inputlayer-server from this branch in a throwaway data dir and drove parameterised execute frames through four clients: a raw /ws client sending literal frame text, the Python SDK, the JS SDK, and the gateway's live pipeline tests. Every scenario passed. - Injection: values are bound without being parsed, so a hostile string is stored as written. - NaN regression (#251): null (what JSON.stringify writes for NaN), {float:"NaN"} and 1e999 are refused, and nothing is deleted. - Negative zero: bare -0 and -0.0 are refused with guidance. {int:-0} deletes the int-0 row and {float:-0.0} round-trips as -0.0. - Big numbers: large ints stay exact and every documented refusal returns invalid_request with the parameter named. - Missing or extra params: these fail validation and nothing runs. - SDKs: both send$pNreferences with typed params and refuse NaN and infinity before sending anything. - Gateway: all 3 live tests pass, and the engine log shows its writes carrying parameters. The millisecond-latency rule was not checked. The perf gate was stopped during its baseline build to meet the deadline, so it took no measurements. I removed the temp server, data dir, JS node_modules/dist and the partial target/perf-gate. This change has no UI surface, so no screenshots.S-77"), +admin(1) is stored as written and creates no admin relationmake perf-gate(or scripts/perf-gate…Evidence: Raw /ws transcript of param edge cases
Evidence: Raw reply text: exact ints and -0.0
Evidence: Python SDK live run with wire frames
Evidence: JS SDK live run with wire frames
Evidence: Gateway live pipeline results and engine param log
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 1 issue found → auto-fixed ✅
ws-protocol/src/params.rs:296- Two numeric wire edge cases get through without an error, contrary to this module's documented contract and to EN-5's requirement that "number ... edge cases are refused or encoded exactly". (1) A bare JSON-0is documented as an int ("a number without fraction or exponent"), but serde_json's parse_integer returnsF64(-0.0)for-0(serde_json de.rs:519-521, "Convert into a float ... on-0"). SoBare::visit_f64bindsParamValue::Float(-0.0)instead ofInt(0). With{"p": -0}, a delete like-r($p)binds float -0.0 and does not match a stored int 0 row; it reports success but deletes nothing. The same path makes{"int": -0}fail with "not an integer". (2){"float": n}with an integer literal above u64::MAX (e.g.18446744073709551617) goes through serde_json's parse_long_integer as an f64, soTyped::Float::visit_f64accepts the rounded value 18446744073709551616.0. The docs say such an n is refused ("{\"float\": n}for an integernthat no f64 equals"); only i64/u64-range integers reachexact_f64. Fix: route integral visit_f64 values in the Typed::Float and Bare paths through an exactness check (or refuse integer-form literals beyond u64 in{"float"}), and map an integral-0.0in Bare and Typed::Int toInt(0)(or refuse it explicitly). Add both cases tovalues_that_would_change_are_refused/bare_values_are_typed_by_their_json_form. Neither SDK triggers this (JS and Python write -0 as0), so only hand-written /ws clients are affected.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
S-77"), +admin(1) is stored as written and creates no admin relationmake perf-gate(or scripts/perf-gate…Builtcargo build --bin inputlayer-serverand started it on port 18765 with an isolated temp data dir and a bootstrap API keyRaw /ws driver (Node WebSocket, literal frame text)node raw.mjs: injection string, null/NaN/1e999 deletes, bare -0/-0.0, {int:-0}, {float:-0.0}, int64 bounds, {float:2^53+1}, {float:>u64}, unknown/two-key tags, duplicate/invalid names, strings with escapes/NUL/unicode, bools, vectors, missing/extra params, meta-command refusal, rule/filter/update placementsnode rawtext.mjs: raw reply text, so exact ints (2^53+1, i64::MAX) and -0.0 are visiblePython SDK live script (uv run --offline python py_sdk.py) with the websocket send logged: insert/delete/conditional delete/retract/raw execute(params), NaN/inf refusalJS SDK built withnpx tsup src/index.ts --format esm, live scriptnode js_sdk.mjswith ws send logged: insert/delete/retract/raw execute(params), bigint ints, NaN/Infinity refusalGateway:INPUTLAYER_TEST_SERVER=http://127.0.0.1:18765 ... cargo test -p inputlayer-gateway --test live_pipeline(3 live tests against the running engine and the public ontology registry), plus the engine request log showingparam_count>0on gateway writesStartedscripts/perf-gate.sh --baseline-rev 6e4c9cdc; stopped it during its baseline release build, before it took any measurementsdocs/content/docs/guides/websocket-api.mdx:129- The full list of refused parameter values exists in three full copies: the ws-protocol/src/params.rs module doc, the asyncapi.yaml Params description, and the websocket-api.mdx Parameters section. This change had to edit all three for the negative-zero and large {"float": n} rules. All three are accurate now. As a follow-up, keep the full list in one owner (the websocket-api guide or the AsyncAPI spec) and cut the other two down to a pointer, so they cannot drift apart.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.