Repository navigation
Verified Completions M0: POST /v1/verify - extract facts, return consistency findings with proofs #83
Description
Why
POST /v1/verify is the smallest useful piece of Verified Completions: send a conversation, get back a list of contradictions with quoted evidence and a proof tree. No completion is generated. This alone is enough to lint a system prompt in CI or batch-audit stored conversations, and it exercises the whole pipeline we need later for the full proxy.
How it works
- Client POSTs an OpenAI-style
messagesarray - The gateway calls Claude (Haiku, temperature 0, structured outputs so the response always matches our JSON schema) to turn the messages into small factual claims. Every claim must carry a verbatim quote from the message it came from
- A validator checks each claim before it touches the engine (see below)
- Valid claims become plain fact inserts into a per-conversation knowledge graph over the WebSocket API. The rule pack is loaded into that KG once at creation
- The gateway queries the findings, fetches
.whyproof trees, and returns the response block defined in the RFC (section 9)
What to do
- Gateway crate skeleton: axum server, config sections
[llm]and[verify](see RFC section 4.1), route only mounted when an Anthropic API key is configured - Extraction call: Anthropic Messages API with structured outputs pinned to
docs/internals/verified-completions/extraction/claim-schema.json, prompt caching on the static instruction prefix, usingdocs/internals/verified-completions/extraction/fact-lifecycle-prompt.md(NOT the supersededextraction-prompt.md) - Ingestion validator. Drop a claim (and count it in metrics) when:
- its quote is not a verbatim substring of the message it points to
- its message index is outside the batch being extracted
- its id collides with an existing claim id
- a value that claims to be a date does not parse as ISO-8601
- it tries to override a seeded ontology entry (extending is allowed, overriding is not)
- it duplicates an existing claim from the same message
- Numeric mirroring: dates become YYYYMMDD integers, datetimes become epoch seconds, quantities become plain integers, all inserted into
claim_num/constraint_num. The engine cannot order-compare strings, so every "X before Y" or "X exceeds limit" check rides these integer columns - Fact compilation: validated claims to parameterized IQL inserts over the WS API. Fixed relation set only. Conversation text must never be able to inject rules or queries
- Findings assembly: query
finding_src, walk the.whyproof trees (already available as JSON over the wire), collapse mirrored pairs like (a,b)/(b,a) into one finding, build theconsistencyresponse block from RFC section 4.2 - Fail open: any extractor or engine error returns
status: "unverified"with a reason instead of failing the request
Done when
Demo works end to end: paste a conversation that contradicts itself, get back findings where each one shows the two quoted sentences and the rule that connects them.
Depends on
- Rule pack validated on the real engine
- Workspace split
Activity
feat: convert repo to a Cargo workspace with a gateway placeholder (#82)
feat: gateway service - separate binary, container, and configuration
feat: Cargo workspace with gateway placeholder (#82) (#99)
feat: gateway service - separate binary, container, and configuration
feat: gateway service - separate binary, container, and configuration
jsam commentedon Aug 20, 2026
Design decision (2026-08-20): ontology selection is PER REQUEST, not per deployment - one gateway serves many chats, each bound to its own ontology.
Contract for M0/M1:
jsam commentedon Aug 20, 2026
Refinement to the per-request contract: X-IL-Ontology accepts name or name@version. Name alone resolves to the allowlist's pinned version; if the header states a version it MUST equal the allowlist pin (else 400) - clients can assert expectations, drift is caught, callers still cannot select unpinned code.
Clarified data flow for extraction: the resolved registry entry supplies the full LLM-facing contract per request - the extraction prompt (which embeds the ontology itself: seeded property classes, modality rules, normalization, worked examples, verbatim-shared with the rule pack) AND the structured-outputs JSON schema that hard-constrains the model. Downstream, the validator (verbatim quotes, protected seeds) and the mapper allowlist complete the four-layer binding. The prompt+ontology is a static prefix per ontology, so Anthropic prompt caching keys on it across all chats sharing that ontology.
feat: Verified Completions M0 - POST /v1/verify bound to registry ont…
jsam commentedon Aug 21, 2026
Implementation note for the record: the shipped contract in #105 differs from the pinned sketch in two ways decided during review. (1) No GATEWAY_ONTOLOGIES allowlist and no GATEWAY_DEFAULT_ONTOLOGY - the gateway loads every ontology published in the registry index at startup, pinned by version and digest for the process lifetime. (2) Selection is mandatory on every request (x-il-ontology header or il_ontology body field, optional name@version assertion checked against the pin); a request without a selection is rejected 400 before any model call. Fail-open semantics, quoted-span findings, and the response envelope match the sketch. Closes via #105 merge.
feat!: course correction - chat-only ingestion, engine-owned ontologi…
feat!: course correction - chat-only ingestion, engine-owned ontologi…