Repository navigation
feat(inputlayer-js): add guarded programs, claim() and any()/NOT(any()) existence checks - #240
Merged
Merged
Conversation
…n relation()/from() Guarded programs compile to the transaction-token form (strict: false) or the abort form (strict, the default): the guard is evaluated once, every fact write is conditioned on the token, and a false guard rolls the whole program back, rules and schema included, raising PreconditionFailed from the assertion's error index. claim() is the guarded insert with unless as a negated atom plus a query on the key in the same program. any() and NOT(any()) work in rules, queries, conditional deletes and guards; a negated atom bound only to constants shares a variable with an equal positive column, else binds through il_const_<t> (queries) or a staged il_txn_const_<t> row (writes). kg.define() declares il_txn, il_txn_pending and il_assert in one program. Unit tests pin the compiled text; live tests (guard_cursor_advance_applies_whole, guard_first_terminal_outcome, guard_note_replacement, claim_twenty_way_race, strict_guard_raises_precondition_failed_from_error_index, stale_deploy_is_refused_including_rules, deploy_applies_whole_when_guard_holds, guarded_program_is_all_or_nothing, count-reply grammar, the hero flow) join make js-test-live.
This was referenced Oct 4, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
So fix the SDKs by improving them. (Context: the captain asked to rework the SDKs so agents and agentic loops use InputLayer without writing Datalog, and to fix the SDK release blockers by improving the SDKs rather than patching. The agreed design, signed off by an independent challenger after three rounds, is ~/firstmate/data/sdk-design/report.md; this item is its phase-1 PR P1-J P1-J (JS).)
What that phase-1 item P1-J of the agreed design asks for, in the JS SDK (packages/inputlayer-js), on today's relation()/from() API:
What Changed
any(Relation, {...})andNOT(any(...))existence checks to the JS SDK. They work infrom().whererule bodies, queries and write guards. The compiler puts body atoms in a fixed order: positive atoms, then negated atoms, then comparisons. When a negation is bound only to constants, the compiler either rewrites it, binds it through an SDK-owned constant relation (il_const_<type>, a session fact on reads and a staged persistent row on writes), or refuses it withCompileError. That refusal also applies indebug()/why().kg.program()(newPrograminprogram.ts). It sends all its statements as one request and one transaction..when(guard)makes the whole program conditional through anil_txntoken, so it applies whole or not at all.strict: trueuses anil_assertabort form and raisesPreconditionFailed, read fromerrors[0].index.strict: falsereturnsapplied=false. Guarded inserts use the update form anchored on a ghost delete row. Write counts come from per-statement reply messages using a shared, pinned grammar.kg.claim(Relation, row, {when, unless, key}), which returns{won, holder}. Added and exported theCompileError,PreconditionFailedandConflictErrorerror classes, and documented them in the JS SDK guide. Added unit tests on compiled text (guards.test.ts) and live engine tests (guards.integration.test.ts);make js-test-livenow runs the live tests.Risk Assessment
✅ Low: The round-3 fix routes debug() and why() through explainable(). These are the only consumers of plan.debug and plan.why. A query whose NOT(any()) binds only constants is now refused with a CompileError before anything is sent to the engine. The new unit test checks that refusal through the public KnowledgeGraph API and checks that nothing reaches the connection. Re-checking the earlier fixes (the il_ghost(0) anchor and the shared parseWriteMessage grammar) and the rest of the diff turned up no new defects.
Testing
I built the engine from this commit and ran it against a disposable data dir. Against it I ran the targeted guard/claim/compiler vitest files (all passed, including guard_cursor_advance_applies_whole, guard_first_terminal_outcome, guard_note_replacement, claim_twenty_way_race, strict_guard_raises_precondition_failed_from_error_index, stale_deploy_is_refused_including_rules, deploy_applies_whole_when_guard_holds and the guarded_program_is_all_or_nothing property test). I also ran a separate end-user script through the public SDK. It confirmed: guarded writes anchored on il_ghost(0) leave existing paused(false)/counter(-1) rows untouched; strict aborts roll back rules, schema and facts and raise PreconditionFailed with no il_txn/il_assert leftovers; strict:false returns applied=false; NOT(any()) guards bound only to constants work and leave no il_txn_const rows behind; debug()/why() and rules refuse constant-only negations with CompileError; a 20-way claim race across 20 connections has exactly one winner and every caller sees the same holder. Insert/query latency matched the base SDK (about 1.6 ms and 0.8 ms medians). Only the ConflictError mapping could not be provoked live. All scratch files, the build and the server were torn down; the worktree is clean.
Evidence: Live vitest run: guards integration + unit compiled-text tests
Evidence: End-user SDK drive transcript (S1-S9) with compiled IQL and engine state
Evidence: Latency base vs change
base-a4be4a2e insert 1.57/1.61/1.61 query 0.79/0.80/0.82 change-3bfce994 insert 1.82/1.56/1.62 query 0.81/0.79/0.75 (median ms over 200)Evidence: Engine server log
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 5 issues found → auto-fixed (2) ✅
packages/inputlayer-js/src/program.ts:64- The ghost delete row of a guarded insert isn't guaranteed to be absent:ghost()builds it from ordinary values in the user's own relation ("",-1,-1.0,false,[]). The intent asks for a "never-present ghost delete row", but these are valid values that can be stored. Two concrete cases: (1) A relationPaused(on: bool)holdspaused(false), orCounter(id: int)holdscounter(-1). Any guarded insert into that relation compiles to-paused(false), +paused(true) <- il_txn("t-…"), which silently deletes the user's existing row and counts it indeleted. (2) Inserting a row equal to the ghost, for example a guardedinsert(Paused, {on: false}), gives-paused(false), +paused(false), whose result depends on engine delete/insert ordering. The same ghost is emitted bycompileClaim(program.ts:375), sokg.claim()has the same problem. Fix: anchor the update form on a row of an SDK-owned relation that is never written, such as-il_txn(""). The token statement already uses that form, and the live grammar test already runs-il_ghost(0), +flag_grammar("b") <- …, showing the engine accepts a delete on a different relation. A test with a pre-existing row equal to the old ghost would pin this.packages/inputlayer-js/src/knowledge-graph.ts:327-KnowledgeGraph.retract()is a new top-level public write method. The P1-J intent coversany()/NOT(any()),kg.program().when(),kg.claim()and the error types, not a standalone retract on the knowledge graph.Program.retractalready does this, and the tests could usekg.program().retract(...).commit(). Recommend removing it unless the design calls for it.packages/inputlayer-js/src/relation.ts:79-RelationDef.any()is a second spelling of the free functionany(Relation, {...})that the intent specifies. No requirement needs the method form. Recommend removing it so the public API has one way to write an existence check.packages/inputlayer-js/src/proxy.ts:172-AND/ORnow accept any number of arguments instead of exactly two. This is a public API change the P1-J intent doesn't ask for; the hero rule could nest binaryANDs. It's backward-compatible and small, but it widens scope. Confirm it's wanted, otherwise remove it.packages/inputlayer-js/src/knowledge-graph.ts:866- The insert-reply regex is now defined twice:INSERTEDin knowledge-graph.ts:866 (used byinsertedCount) andINSERTEDin program.ts as part of the reply grammar the live test pins. If the engine's message changes, one copy can be updated and the other missed. MakeinsertedCountuseparseWriteMessageso there is a single owner of the grammar.🔧 Fix applied.
3 issues (2 warnings, 1 info) still open:
packages/inputlayer-js/src/knowledge-graph.ts:327-KnowledgeGraph.retract()is a new top-level public write method. The P1-J intent coversany()/NOT(any()),kg.program().when(),kg.claim()and the error types, not a standalone retract on the knowledge graph.Program.retractalready does this, and the tests could usekg.program().retract(...).commit(). Recommend removing it unless the design calls for it.packages/inputlayer-js/src/relation.ts:79-RelationDef.any()is a second spelling of the free functionany(Relation, {...})that the intent specifies. No requirement needs the method form. Recommend removing it so the public API has one way to write an existence check.ℹ️
packages/inputlayer-js/src/proxy.ts:172-AND/ORnow accept any number of arguments instead of exactly two. This is a public API change the P1-J intent doesn't ask for; the hero rule could nest binaryANDs. It's backward-compatible and small, but it widens scope. Confirm it's wanted, otherwise remove it.packages/inputlayer-js/src/knowledge-graph.ts:816-explainable()is defined but never called, sokg.debug()andkg.why()give wrong results without failing for a query whose NOT(any()) binds only constants. In that casecompileQueryPlanbinds the constant through anil_const_<type>session fact, but it adds that fact only toplan.programs(compiler.ts:895), not toplan.debugorplan.why.debug()(knowledge-graph.ts:694) andwhy()(knowledge-graph.ts:707) send the bare statement. It readsil_const_s(K), which has no rows, so for examplekg.why({select:[ToolPolicy], where: NOT(any(KillSwitch,{tool:'x'}))})explains an empty result or hits an unknown-relation error, whilekg.query()with the same options returns rows. Fix: wrap the plan inexplainable(plan, 'debug')/explainable(plan, 'why')in both methods. That is the CompileError refusal the helper was written for, and it was introduced in 0ad8571.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ No issues found.
CARGO_TARGET_DIR=/tmp/nm-target-01M43CYT cargo build --release --bin inputlayer-server, then started the server on 127.0.0.1:18791 with a throwaway --data-dir (removed afterwards)npx vitest run tests/guards.test.ts tests/compiler.test.ts(unit tests on compiled text, 65 passed)INPUTLAYER_TEST_SERVER=ws://127.0.0.1:18791/ws npx vitest run --reporter=verbose tests/guards.integration.test.ts(12 live guard/claim tests passed)npx vitest run tests/docs-queries.integration.test.ts tests/integration.test.tslive (81 passed; these are the other suitesmake js-test-liveruns)Temporary live driver tests/zz-nm-adhoc.integration.test.ts (copied to evidence, then deleted from the worktree), covering: the ghost anchor with stored paused(false)/counter(-1) rows; strict:false vs strict on a false guard; CompileError for a constant-only NOT(any()) in a rule and for OR in a guard; the rule rewrite when a positive column is pinned to the same constant; claim winner/loser/when-false; median latency over 30 runs🚨
packages/inputlayer-js/tests/guards.integration.test.ts:326- claim_twenty_way_race failed every time with 'Unexpected server response: 429'. Since the connection core (feat(inputlayer-js): routed connection core with request ids, deadlines and reconnect #237), each KnowledgeGraph handle opens its own lazily-bound socket. The test bound all 20 handles with Promise.all, which opened 19 sockets at once and went over the engine's per-IP limit on unauthenticated sockets (ws_max_preauth_per_ip = 16; the server log shows ws_preauth_limit_exceeded). Fixed in the test by binding the handles one at a time. The 20 claims still go out together, and the test now passes reliably.packages/inputlayer-js/tests/integration.test.ts:979- 'Integration: Engine failures > dropping the current knowledge graph rejects without switching it' fails in the js-test-live file set (dropKnowledgeGraph resolves instead of rejecting with QueryError). It fails the same way on the base commit a4be4a2 against the same server, so this change didn't cause it, and it's outside P1-J. But it will make the js-sdk-live CI job fail, and the intent relies on that job.Live validation: ✅ go - 13 of 13 scenarios driven live against the product
cargo build --release --bin inputlayer-serverand an isolated server on 127.0.0.1:8197 with a throwaway --data-dir (removed afterwards)INPUTLAYER_TEST_SERVER=ws://127.0.0.1:8197/ws npx vitest run --reporter=verbose tests/guards.integration.test.ts(before fix: 11/12, race test 429; after fix: 12/12)npx vitest run tests/guards.integration.test.ts -t twenty, 5 more runs, all passnpx vitest run tests/guards.test.ts tests/compiler.test.ts(unit tests on compiled text, including 3+-argument AND/OR and the debug/why CompileError)js-test-live file set:npx vitest run tests/docs-queries.integration.test.ts tests/integration.test.ts tests/connection.integration.test.ts tests/guards.integration.test.ts(99/100; the one failure also happens on base)Base-commit check: ran a4be4a2e's tests/integration.test.ts against the same server (same failure)End-user scriptnpx vite-node demo.tsagainst the live server: rule NOT(any()), constant-only refusals, read-path il_const, when() strict:false apply/replay, strict PreconditionFailed, stale deploy with rule, ghost anchor keeping Paused(false), 10-way claim race, OR with 3 arguments, latency comparison🔧 Fix applied.
✅ Re-checked - no issues remain.
CARGO_TARGET_DIR=/tmp/nm-il-target cargo build --release --bin inputlayer-server, then the server started with a throwaway --data-dir on 127.0.0.1:8197INPUTLAYER_TEST_SERVER=ws://127.0.0.1:8197/ws npx vitest run tests/guards.integration.test.ts tests/guards.test.ts tests/compiler.test.ts --reporter=verbose(78/78 passed, including all 8 named live tests from the intent)End-user script run withnpx vite-nodethat imports the SDK and checks: ghost-collision rows, strict:false applied=false, strict abort rolling back rule+schema+facts, strict apply-whole with a rule, NOT(any()) guard bound only to constants, read-path NOT(any()) plus debug()/why() refusal, rule CompileError, 20-connection claim race, and claim with when/unlessLatency comparison: the base-commit SDK (git archive a4be4a2e) against this change on the same server, median of 200 insert and query calls, 3 rounds🔧 **Document** - 1 issue found ✅
docs/content/docs/guides/js-sdk.mdx- The JS SDK guide does not yet cover how to use any()/NOT(any()), kg.program().when()/strict, kg.claim() or kg.retract(). The intent leaves docs out of scope for P1-J (they belong to the separate phase-1 docs/README hero item), so this pass only added CompileError, PreconditionFailed and ConflictError to the guide's error-class list. Follow-up: write the usage section for guards and claims in that docs item.✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
✅ No issues found.