Repository navigation
Conversation
A program whose writes commit now replies with `revision`: the revision of the snapshot its commit published, the one standing-query pushes carry. A write that changes nothing names the current revision. Queries, rolled back programs and session-scoped writes carry none.
…e for session schemas
…s nothing durable
jsam
marked this pull request as draft
October 4, 2026 21:04
Contributor
Author
|
Superseded by #293 (same work, synced with main). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
please continue developing the input layer engine so we make sure that the symbolic engine is ready to go live and completely production ready while we develop the GoTM uh, market.
(Context: the captain approved the neurosymbolic decision engine plan; its milestone "Engine readiness" makes the symbolic engine production-ready for the hosted decision API. This task is item EN-3: commit revision on write replies (E-3): a program whose writes commit returns the revision it committed at (
result.revisionfor committed programs) — why: every API write returnsrevision; decision records carryrevision_commitwithout a second query)What Changed
revisionfield (QueryResult, plusResultFrameandResultStartFrameinws-protocol, on both the single-frame and streamed WebSocket paths). It names the knowledge-graph revision that standing-query pushes carry once the program's writes are visible. It is set only when the program changed persistent state: committed fact, schema or rule writes,.rel drop,.clear prefix,.index create|drop|rebuild,.kg createand.ontology install|remove|upgrade. When a program does several of these, the reply carries the last one's revision. Session-only schemas, queries and rolled-back writes leave it out.ProgramCommitgained arevisionfield, andcreate_knowledge_graph_at,drop_relation_in,clear_relations_by_prefix_inand thecreate_index_in/drop_index_in/rebuild_index_infunctions return it. The protocol handler and write run pass it through, and keep the meta-command's revision when a later write run in the same program commits nothing durable.revisionfield in the AsyncAPI spec and the WebSocket API guide. Added tests covering atomic programs, HNSW index commands, standing queries (a push at the returned revision reflects the write), pinned proofs, andws-protocolframe serialization.Risk Assessment
✅ Low: The change adds an optional
revisionfield to the reply. Each storage path reads it under the KG write lock it already holds, right after publishing. I traced every write path (committed write runs, durable no-ops, session-schema-only runs, .rel drop, .clear prefix, .index create/drop/rebuild, .kg create, .ontology install/remove/upgrade, pinned proofs): each returns the revision the docs describe. The program-boundary check stops meta commands from sharing a program with writes, so the revisions can't be mixed up within one program.Testing
I built and started a real inputlayer-server on a throwaway data dir and port, then drove it over the WebSocket API with Node scripts, one subscriber connection and one writer. I covered 15 scenarios, including the edge cases: a write that changes nothing, a rolled-back write, a session schema alone, a meta command mixed with a session schema, and a write followed by a proof. All of them passed. Each persistent write (facts, rules,
.rel drop,.clear prefix,.index create/rebuild/drop,.kg create) named a revision. Where a subscription could see the change, the next push carried exactly that revision. Queries, rolled-back writes, session schemas,.kg use,.kg dropand inspection commands named none. I also ran the four new targeted integration and unit tests, and all passed. This is a protocol change with no visual UI, so the evidence is the transcripts. I stopped the server and deleted its data dir. The worktree is clean..rel dropand.clear prefixname the revision their retraction push carries.rel dropmixed with a session schema in one program is rejected, applies nothing and names nothing.kg use,.rel,.index listand.kg dropname no revision.kg createnames the new graph's revision.index create,.index rebuildand.index dropeach name a revision higher than the one before.whynames the commit revision; the proof is pinned to the snapshot before itEvidence: Live WebSocket transcript: facts, rules, relation commands, kg create
Evidence: Live WebSocket transcript: index commands, write followed by proof, kg drop
Evidence: Driver scripts
Evidence: Targeted test runs
Evidence: Key excerpt
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 2 issues found → auto-fixed (2) ✅
src/storage_engine/program_commit.rs:84- Programs that only change session state still get arevisionback, which the docs say they shouldn't. A non-persistent schema declaration is queued into the write run asCatalogStatement::Schema→CatalogChange::DefineSessionSchema(catalog_staging.rs:45).commit_programthen takes the early return for session-schema-only deltas, and that return now setsrevision: db.snapshot.load().revision.commit_write_runreturns this asOk(revision), so handler.rs:3268 attaches it to the reply. Example: a program made of one session (non-persistent) schema declaration replies withrevision: N. Three places promise otherwise:QueryResult.revisionsays "Present only when the program's writes to persistent state committed" (src/protocol/wire.rs:335-339),ResultFrame.revisionsays "Set only when the program wrote persistent state" (ws-protocol/src/server.rs:189-193), and the guide says session-scoped writes carry norevision(docs/content/docs/guides/websocket-api.mdx:159). Fix one side so they agree: either return no revision when the delta has no durable facts or catalog (e.g. makeProgramCommit.revisionan Option, or havecommit_write_runreturn None when no relation or durable catalog changed), or reword the three doc sites to say session-only writes name the current revision.src/protocol/handler.rs:3264- Some writes to persistent state never get arevision. Only statements that go throughWriteRunproduce one..rel drop <name>(handler.rs ~2773,storage.drop_relation_in) and.clear prefix <p>(handler.rs ~2860,storage.clear_relations_by_prefix_in) write and publish facts and schemas directly, so their replies have norevision. Example: a program that is only.rel drop etadeletes persistent facts, publishes a new snapshot that standing-query pushes will carry, and replies withoutrevision. The intent says "every API write returnsrevision". Whether these admin meta commands count as API writes, or the decision API will only ever send fact/rule programs, is a product decision. If they count, the fix is to return the published revision from those storage calls as well.🔧 Fix applied.
2 warnings still open:
src/protocol/handler.rs:3264- Some writes to persistent state never get arevision. Only statements that go throughWriteRunproduce one..rel drop <name>(handler.rs ~2773,storage.drop_relation_in) and.clear prefix <p>(handler.rs ~2860,storage.clear_relations_by_prefix_in) write and publish facts and schemas directly, so their replies have norevision. Example: a program that is only.rel drop etadeletes persistent facts, publishes a new snapshot that standing-query pushes will carry, and replies withoutrevision. The intent says "every API write returnsrevision". Whether these admin meta commands count as API writes, or the decision API will only ever send fact/rule programs, is a product decision. If they count, the fix is to return the published revision from those storage calls as well.src/protocol/handler.rs:3283- A session-only write run wipes out a revision that an earlier meta command in the same program already set. Round 1 introduced this when it combined two of its own changes: meta commands now setrevisionas they run (.rel dropat 2785,.clear prefixat 2873,.index create|drop|rebuildat 3065/3094/3175,.kg createat 2633), andcommit_write_runnow returnsNonewhen every queued statement is a session schema (write_run.rs:425). The write run commits after the statement loop, andOk(committed) => revision = committedoverwrites unconditionally. Concrete case: the program.rel drop gonefollowed bys(a: int)(a session schema).drop_relation_inpublishes a new snapshot and setsrevision = Some(N). The write run then holds only the session schema, socommit_write_runreturnsOk(None)andrevisionbecomesNone. The reply has norevisioneven though the program changed persistent state. That breaks the operator's decision that every statement changing persistent state returns a revision, and it contradicts the docs (wire.rs:489-495, server.rs:778-783, websocket-api.mdx:159), which say a program with several such statements names the last one's. Fix: keep the earlier value, i.e.Ok(committed) => revision = committed.or(revision). A durable commit always runs after the loop, so its revision is the latest and still wins. Add the mixed program above totest_relation_commands_name_the_revision_their_push_carries. No other site has this overwrite: the ontology paths already use.or(revision)(handler.rs:4993, 5052).🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
.ontology install|removeneeds an ontology pack registry on disk, which the live setup…cargo build --bin inputlayer-server --bin inputlayer-client, then ran inputlayer-server on 127.0.0.1:18731 with a throwaway INPUTLAYER_STORAGE__DATA_DIR in /tmp and INPUTLAYER_BOOTSTRAP_API_KEYnode drive.mjs(WebSocket subscriber and writer): 13 live checks covering write and push revisions, no-op writes, queries, rollback, session schema, write+query, .clear prefix, .rel drop, a mixed meta+session program, .rel, .kg create and .kg usenode drive_index.mjs: .index create, rebuild and drop each name a new revision and a following no-op write names the same one; .index list names nonecargo test --test standing_query_tests -- revision(3 tests)cargo test --test atomic_program_tests failed_pack_installation_applies_nothing(ontology install/remove revisions)cargo test --test hnsw_index_tests test_index_commands_name_the_revisioncargo test -p inputlayer-ws-protocol revision(serialization of the ResultFrame revision field)✅ No issues found.
.rel dropand.clear prefixname the revision their retraction push carries.rel dropmixed with a session schema in one program is rejected, applies nothing and names nothing.kg use,.rel,.index listand.kg dropname no revision.kg createnames the new graph's revision.index create,.index rebuildand.index dropeach name a revision higher than the one before.whynames the commit revision; the proof is pinned to the snapshot before itStartedtarget/debug/inputlayer-server --config <evidence>/config.toml --port 18431with a throwaway data dir andINPUTLAYER_ADMIN_PASSWORD, then drove it with Node WebSocket scripts (node drive.mjs,node drive2.mjs), one subscriber connection and one writer connection+n(1)with a.subscribe nsub ?n(X)open: reply revision compared to the push revision+n(1)sent again (changes nothing): reply revision compared to the previous one?n(X),.rel,.index list,.kg use,.kg drop: checked that the reply has no revision+n(2)\n?n(X)(write plus query): reply revision compared to the push revision+n(3)\n+n("three", 4)(arity error rolls the program back): no revision and n(3) is absents(a: int)(session schema only): no revision+m(X) <- n(X)(persistent rule): revision present.rel drop goneand.clear prefix p_: reply revision compared to the subscription push revision.rel drop n\ns(a: int)(meta command mixed with a session schema): rejected whole, nothing named.kg create,.index create/rebuild/drop: each names a revision higher than the one before+need("s2","say_eta")\n.why ?need(S,T): reply revision (23) is after the pinned proof's revision (22)cargo test --test standing_query_tests test_write_reply_names_the_revisioncargo test --test standing_query_tests name_the_revisioncargo test --test hnsw_index_tests test_index_commands_name_the_revisioncargo test --lib the_reply_names_the_revision✅ **Document** - passed
✅ No issues found.
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
✅ No issues found.