Repository navigation
feat(inputlayer-py): add guarded programs, claim() and R.any()/~R.any() existence checks - #247
Merged
Merged
Conversation
…() existence checks Guarded programs compile to the transaction-token form (strict=False) or the abort form (strict=True, the default): the guard is evaluated once, every fact write is conditioned on the token, and a false guard rolls the whole program back, rules and schema included, raising PreconditionFailed from the assertion's error index. claim() is the guarded insert with unless as a negated atom plus a query on the key in the same program. R.any() and ~R.any() work in rules, queries, conditional deletes and guards on Relation and Derived classes; a negated atom bound only to constants shares a variable with an equal positive column, else binds through il_const_<t> (queries, views) or a staged il_txn_const_<t> row (writes). kg.define() declares il_txn, il_txn_pending and il_assert in one program; kg.retract() and the sync mirror join. Unit tests pin the compiled text; live tests (guard_cursor_advance_applies_whole, guard_first_terminal_outcome, guard_note_replacement, claim_twenty_way_race, strict_guard_raises_precondition_failed_from_error_index, stale_deploy_is_refused_including_rules, deploy_applies_whole_when_guard_holds, guarded_program_is_all_or_nothing, count-reply grammar, the hero flow) join make python-sdk-live.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
So fix the SDKs by improving them. (Context: agreed SDK design ~/firstmate/data/sdk-design/report.md; this is phase-1 PR P1-I P1-I (Py). Its JS twin merged as #240 (claim(), program().when(), any(), retract, variadic AND/OR); keep semantics identical. Python compiler v2 (#235, #239) and connection core (#236) are merged.)
Substance of the referenced design item P1-I (Python SDK): kg.claim(), kg.program().when() in both guard forms (token and abort) and the il_txn_const write-path binding, R.any() on today's Relation/Derived/From; il_txn, il_txn_pending, il_assert declared by define(); count-grammar and abort-form conformance tests; live tests guard_cursor_advance_applies_whole, guard_first_terminal_outcome (negation-only shape), guard_note_replacement, claim_twenty_way_race, strict_guard_raises_precondition_failed_from_error_index, stale_deploy_is_refused_including_rules, deploy_applies_whole_when_guard_holds, property test guarded_program_is_all_or_nothing.
Design semantics referenced (sections 5, 9, 11 and the JS twin #240): kg.program() collects statements committed as one request and one transaction. .when() makes the whole program conditional and the guard is evaluated once: the token form (strict=False, fact-only programs) is statement 1
-il_txn(""), +il_txn("t-<id>") <- guard, every insert becomes-il_ghost(0), +r(...) <- il_txn("t-<id>")(a single-head +r(...) <- body is never emitted for a base relation because the engine registers it as a rule), every conditional delete gets, il_txn("t-<id>")appended, and the last statement removes the token; applied is read from the token statement's count and strict=False returns applied=False instead of raising. The abort form (strict=True, the default, mandatory for programs holding rules or schema, which a token cannot condition) adds +il_txn_pending("t-") before the token statement and the assertion-il_assert(0), +il_assert("precondition_failed:t-<id>") <- il_txn_pending(K), K = "t-<id>", !il_txn(K)after it, with il_assert(v: int), so a false guard fails the program at the assertion index and the engine rolls back everything, rules and schema included; PreconditionFailed is raised structurally from errors[0].index == assertion index, never from message text. strict=False with rules or schema is refused with a CompileError. claim(row, when=, unless=, key=) is the guarded insert with unless as a negated atom in the guard plus a query on the key in the same program; it returns Claim(won, holder) with holder None when the when-guard failed; of 20 concurrent claims exactly one wins. R.any(**cols) / ~R.any(...) are existence checks usable in rules, queries, conditional deletes and guards; a negated atom must share a variable with a positive atom: when a positive atom already carries the constant (an equality or the same constant in a positive any()) the compiler shares a variable there, otherwise it binds the constant through an SDK-owned typed constant relation: a program-local session fact il_const_ on the read path, persistent il_const_ rows for views, and on the write path a staged persistent il_txn_const_ row inserted before the guard and deleted last (session facts are invisible to update bodies). Write counts are read from the pinned per-statement reply grammar (Inserted N fact(s) into ''., Update: N deleted, M inserted., Conditional delete: N fact(s) deleted from ''., Deleted N facts from ''.) and a live conformance test pins that grammar and the abort form against every engine build in CI. A conflict on a conditional write raises Conflict (retry with backoff). Live tests run against a real engine in CI (make python-sdk-live).What Changed
kg.program(), which commits its statements as one request and one transaction..when()makes the whole program conditional. Both guard forms exist: the token form (strict=False, fact-only, returnsapplied=False) and the abort form (strict=True, the default, which also works for rules and schema). Also addedkg.claim(row, when=, unless=, key=), which returnsClaim(won, holder), pluskg.retract(). The sync client has matchingProgramSync,program(),claim()andretract(). New exports:Program,ProgramResult,Claim, and thePreconditionFailedandConflictexceptions.PreconditionFailedis raised from the error index of the assertion statement, never from message text.R.any(**cols)/~R.any(...)existence checks for rules, queries, conditional deletes and guards. A negated atom shares a variable with a positive atom when one already carries the constant. Otherwise the compiler binds the constant through SDK-ownedil_const_<t>relations (read path and views) or stagedil_txn_const_<t>rows (write path).define()now declaresil_txn,il_txn_pendingandil_assert. Write counts are parsed from the pinned per-statement reply grammar.test_guards.py) and live tests (guards_live.py) covering the count-reply grammar and abort-form conformance, the 20-way claim race, stale deploys, and the all-or-nothing property. The live tests are wired intomake python-sdk-live. Added guarded-write and claim sections to the Python and JS SDK guides, and updated the READMEs to saysubscribe()/claim()are merged on main but not yet in a release.Risk Assessment
Testing
Built the release server all the way through and started it on an isolated port with a throwaway data dir. Ran the make python-sdk-live suites against it: docs queries, compile rules, subscriptions and guards, 70 tests, all passing. That covers every guard and claim live test, including the count-reply grammar conformance test, the 20-way claim race and the all-or-nothing property test. Also ran an extra driver for the constant-row index fix, rule handling under guards, the strict=False refusal and guard leftovers; every check passed. The same driver fails with InternalError on the pre-fix SDK, so the regression is reproduced and shown fixed. A first attempt on port 8197 hit a port collision, so I discarded it and re-ran on a port I had confirmed was free. Afterwards I removed all transient state and ran cargo clean.
Evidence: Live SDK suites (make python-sdk-live equivalent) transcript
Evidence: Extra live scenarios driver
Evidence: Extra live scenarios transcript (branch SDK, PASS)
Evidence: Same driver on pre-fix SDK e3c62df (reproduces InternalError)
Evidence: Release server build log
~/.no-mistakes/evidence/01M43JW4WPWEQA59H7XZKEQ0QK/server.log)Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
🔧 **Rebase** - 5 issues found → auto-fixed ✅
Makefile- merge conflict rebasing onto origin/mainpackages/inputlayer-py/src/inputlayer/__init__.py- merge conflict rebasing onto origin/mainpackages/inputlayer-py/src/inputlayer/client_sync.py- merge conflict rebasing onto origin/mainpackages/inputlayer-py/src/inputlayer/exceptions.py- merge conflict rebasing onto origin/mainpackages/inputlayer-py/src/inputlayer/knowledge_graph.py- merge conflict rebasing onto origin/main🔧 Fix applied.
✅ Re-checked - no issues remain.
🔧 **Review** - 1 issue found → auto-fixed ✅
packages/inputlayer-py/src/inputlayer/program.py:1279- Program.compile counts one statement per_Plainentry, butcompile_rule()(compiler.py:1542) returns"\n".join([*compiled.constants, compiled.clause]). That is several statements whenever a persistent rule's negated~R.any()binds a constant through+il_const_<t>(...)rows. Each_Plaintext is appended as a single element oflines, so everywrite_indexesentry after it is too small by the number of constant rows. Example:kg.program().define_rules(V).insert(X(...)).commit(), where V's clause is~KillSwitch.any(tool="t")and no positive atom carries "t". The engine applies the whole program, but_commit_programreads reply rows at the wrong indexes.parse_write_messagethen gets a rule-registration message, or "" past the end, and raises InternalError after the writes have already committed. Even when no error is raised, the inserted/deleted counts are wrong. The same off-by-N affects the guarded path at program.py:1317-1323 (strict deploys holding such rules plus fact writes), anddefine_rules(program.py:1218-1234) is the producer. Fix: in both the unguarded and guarded loops, add each_Plainaslines.extend(s.text.split("\n")), or store one_Plainper statement indefine_rules, so line index equals statement index. Then add a unit test assertingwrite_indexesfor a program that holds a rule with a constant-bound negation.🔧 Fix applied.
✅ Re-checked - no issues remain.
🔧 **Test** - 2 issues found → no changes applied ✅
make python-sdk-live.make python-sdk-live.make python-sdk-live.make python-sdk-live.make python-sdk-live.make python-sdk-live.make python-sdk-live.make python-sdk-live.uv run --extra dev pytest tests/test_guards.py -q(42 passed; offline unit tests, not live)cargo build --release --bin inputlayer-server(started for make python-sdk-live; stopped before it finished)Wrote the live driver ~/.no-mistakes/evidence/01M43JW4WPWEQA59H7XZKEQ0QK/extra_live_scenarios.py (not run: no server binary)🔧 No changes applied.
✅ Re-checked - no issues remain.
CARGO_INCREMENTAL=0 CARGO_PROFILE_RELEASE_DEBUG=0 cargo build --release --bin inputlayer-server(ran to completion, 2m45s)Startedtarget/release/inputlayer-serverwith INPUTLAYER_ADMIN_PASSWORD=admin on free port 36703 and a throwaway --data-diruv run --extra dev pytest tests/docs_queries_live.py tests/compile_rules_live.py tests/subscriptions_live.py tests/guards_live.py -vagainst that server (make python-sdk-live equivalent): 70 passedpython extra_live_scenarios.py(S1-S5: define_rules with constant-bound negation plus writes, guarded strict deploy, false guard, strict=False refusal, leftovers) against the live server with the branch SDK: OVERALL PASSSame driver with PYTHONPATH pointed at the pre-fix SDK source (e3c62df9): InternalError 'Unexpected write reply ... Rule allowed registered.', which reproduces the buguv run --extra dev pytest tests/test_guards.py -q(write_indexes unit regression): 42 passedTeardown: killed server, removed data dir and pre-fix copy,cargo clean🔧 **Document** - 2 issues found → auto-fixed ✅
docs/content/docs/guides/python-sdk.mdx- The Python SDK guide owns user-facing usage but does not document the new kg.program().when() (token and abort forms, strict=), kg.claim()/Claim(won, holder), or R.any()/~R.any(). It only lists the new exceptions. Its JS twin feat(inputlayer-js): add guarded programs, claim() and any()/NOT(any()) existence checks #240 shipped no guide section either. I left this out of scope rather than writing a new section in a docs pass. Suggested follow-up: add one guarded-writes section to each SDK guide.README.md:17- The main README hero still says subscribe() and claim() are phase 1 'in progress, not merged yet'. subscribe() was merged in feat(inputlayer-py): subscribe(), watch() and on() with typed Change events #245/feat(inputlayer-js): subscribe(), watch() and on() with typed Change events #241, and claim() is merged in JS (feat(inputlayer-js): add guarded programs, claim() and any()/NOT(any()) existence checks #240) and in Python with this change. The JS package README line 7 has the same wording. These are curated positioning texts tied to phase-1 completion and release status, so the operator should decide whether to reword them to 'not in a release yet' (as what-it-replaces.mdx already says). I did not edit them.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.