Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -429,10 +429,11 @@ js-test:
cd packages/inputlayer-js && npm ci --ignore-scripts && npm test

# js-test-live runs every query example in the JS SDK guide
# (docs/content/docs/guides/js-sdk.mdx) and the SDK's integration tests
# (including the shared meta-command fixture, packages/conformance) against a
# live server, so a form the engine rejects fails here rather than in a
# user's hands.
# (docs/content/docs/guides/js-sdk.mdx), the SDK's integration tests
# (including the shared meta-command fixture, packages/conformance) and the
# connection core's live tests (routing, deadlines, reconnect) against a live
# server, so a form the engine rejects fails here rather than in a user's
# hands.
JS_LIVE_PORT ?= 8091
js-test-live:
@cargo build --release --bin inputlayer-server
Expand All @@ -450,7 +451,7 @@ js-test-live:
INPUTLAYER_TEST_SERVER=ws://127.0.0.1:$(JS_LIVE_PORT)/ws \
INPUTLAYER_TEST_USER=admin \
INPUTLAYER_TEST_PASSWORD=admin \
npx vitest run tests/docs-queries.integration.test.ts tests/integration.test.ts; \
npx vitest run tests/docs-queries.integration.test.ts tests/integration.test.ts tests/connection.integration.test.ts; \
TEST_EXIT=$$?; \
cd - > /dev/null; \
kill $$SERVER_PID 2>/dev/null || true; \
Expand Down
29 changes: 25 additions & 4 deletions docs/content/docs/guides/js-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -60,11 +60,26 @@ const il = new InputLayer({
username: "admin",
password: "admin",
autoReconnect: true, // reconnect on disconnect (default: true)
reconnectDelay: 1.0, // seconds between attempts (default: 1.0)
reconnectDelay: 1.0, // first backoff in seconds, doubled with jitter up to 30 (default: 1.0)
maxReconnectAttempts: 10, // give up after N failures (default: 10)
defaultTimeoutMs: 30_000, // deadline of a call without its own (default: 30 000; 0 for none)
keepaliveMs: 20_000, // ping a silent connection so it is never idle-closed; reconnect one with no pong or frame
// within timeoutGraceMs, unless the request window is full (call deadlines bound it then) (default: 20 000)
});
```

Calls on one client run concurrently: every request carries an id and each reply is routed back to its call, so `Promise.all` over queries pipelines them on the connection. A call takes a deadline and an `AbortSignal`:

```typescript
const controller = new AbortController();
const rows = await kg.execute("?path(X, Y)", { timeoutMs: 2000, signal: controller.signal });
// DeadlineExceededError or CancelledError when stopped before committing: nothing was applied.
```

The deadline runs from the call, including time spent waiting to be sent. If the engine has not replied `timeoutGraceMs` (default 2 000) after it, the call fails locally: a query with `DeadlineExceededError`, a program that may write with `OutcomeUnknownError` (read the state back before retrying it). Every error a call rejects with carries the program it sent as `iql`.

When the connection drops, calls in flight fail with `ConnectionLostError` (its `mayHaveCommitted` is set for a write that was sent: read the state back before resending it), calls not yet sent wait for the reconnect, and the client re-opens on the same knowledge graph, replaying the notifications you missed. `il.events` announces `disconnected`, `reconnected`, `session_reset` (session facts, rules and subscriptions are gone: re-create them) and `notification_gap` (also after a reconnect when no notification had arrived yet, since there was no cursor to replay from).

Unlike the Python SDK which has separate sync and async clients, the TypeScript SDK is `async/await` throughout - which is the natural pattern for JavaScript.

## Knowledge Graphs
Expand All @@ -79,12 +94,14 @@ const kg = il.knowledgeGraph("myapp");
const graphs = await il.listKnowledgeGraphs();

// Drop a knowledge graph and all its data
await il.knowledgeGraph("default").execute(".kg use default");
await il.dropKnowledgeGraph("myapp");
```

The server refuses to drop the session's current knowledge graph. Switch to
another graph first, as above; `dropKnowledgeGraph()` does not switch for you.
Each handle has its own connection, bound to its graph when it first runs a
call (the graph is created then if it is missing), so handles never switch
graphs under one another. `dropKnowledgeGraph()` runs on the client's own
connection, which stays on `initialKg` (the server's `default` unless set);
the server refuses to drop that one.

## Defining Schemas

Expand Down Expand Up @@ -754,6 +771,10 @@ import {
OutcomeUnknownError, // commit outcome unknown; store may be read-only
StoreReadOnlyError, // write refused until restart recovery
ConnectionError, // network/connection issues
ConnectionLostError, // closed with the call pending; see mayHaveCommitted
DeadlineExceededError, // timeoutMs passed before committing; nothing applied
CancelledError, // signal aborted before committing; nothing applied
RateLimitedError, // still rate limited after one automatic resend
AuthenticationError, // bad credentials
SchemaConflictError, // schema mismatch on define()
ValidationError, // invalid data
Expand Down
4 changes: 2 additions & 2 deletions gui/lib/docs-bundle.ts

Large diffs are not rendered by default.

144 changes: 126 additions & 18 deletions packages/inputlayer-js/src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,14 @@
* InputLayer - top-level async client.
*/

import { Connection, type ConnectionOptions } from './connection.js';
import { Connection, type ConnectionEventType } from './connection.js';
import { KnowledgeGraph } from './knowledge-graph.js';
import type { NotificationEvent, NotificationCallback } from './notifications.js';
import { QueryError } from './errors.js';
import {
NotificationDispatcher,
type NotificationEvent,
type NotificationCallback,
} from './notifications.js';
import {
type UserInfo,
type ApiKeyInfo,
Expand All @@ -31,14 +36,26 @@ export interface InputLayerOptions {
apiKey?: string;
/** Enable auto-reconnect on connection loss (default: true) */
autoReconnect?: boolean;
/** Delay between reconnect attempts in seconds (default: 1.0) */
/** First reconnect backoff in seconds, doubled per attempt with jitter (default: 1.0) */
reconnectDelay?: number;
/** Longest reconnect backoff in seconds (default: 30) */
maxReconnectDelay?: number;
/** Max reconnect attempts before giving up (default: 10) */
maxReconnectAttempts?: number;
/** Initial knowledge graph to use (default: "default") */
/** Knowledge graph of the client's own connection (default: the server's "default") */
initialKg?: string;
/** Last notification sequence for replay on reconnect */
/** Last notification sequence seen, to replay what followed it... */
lastSeq?: number;
/** ...and the `stream_epoch` it belongs to (`InputLayer.epoch`). */
epoch?: string;
/** Deadline of a call without its own `timeoutMs` (default: 30 000; 0 for none) */
defaultTimeoutMs?: number;
/** Wait past a deadline, after cancelling, before failing the call locally (default: 2 000) */
timeoutGraceMs?: number;
/** Ping an otherwise silent connection this often so it is never idle-closed, and probe the transport: no pong and no frame within `timeoutGraceMs` drops and reconnects it, except while the request window is full, when each call's deadline bounds it (default: 20 000; 0 disables) */
keepaliveMs?: number;
/** Requests in flight per connection (default: 15, one below the server's bound so a cancel can always be read) */
maxInFlight?: number;
}

/**
Expand Down Expand Up @@ -68,33 +85,109 @@ export interface InputLayerOptions {
* ```
*/
export class InputLayer {
private readonly opts: InputLayerOptions;
private readonly conn: Connection;
private readonly dispatcher = new NotificationDispatcher();
private creating: Promise<unknown> = Promise.resolve();
private readonly kgs = new Map<string, KnowledgeGraph>();

/**
* Events of every connection the client holds (see `ConnectionEventType`),
* each `CustomEvent`'s `detail` naming its `knowledgeGraph` handle (absent
* for the client's own connection).
*/
readonly events = new EventTarget();

constructor(opts: InputLayerOptions) {
this.conn = new Connection({
this.opts = opts;
this.conn = this.newConnection(opts.initialKg, {
lastSeq: opts.lastSeq,
epoch: opts.epoch,
});
}

private newConnection(
kg: string | undefined,
extra: { lastSeq?: number; epoch?: string; handle?: string } = {},
): Connection {
const opts = this.opts;
const conn = new Connection({
url: opts.url,
username: opts.username,
password: opts.password,
apiKey: opts.apiKey,
autoReconnect: opts.autoReconnect,
reconnectDelay: opts.reconnectDelay,
maxReconnectDelay: opts.maxReconnectDelay,
maxReconnectAttempts: opts.maxReconnectAttempts,
initialKg: opts.initialKg,
lastSeq: opts.lastSeq,
defaultTimeoutMs: opts.defaultTimeoutMs,
timeoutGraceMs: opts.timeoutGraceMs,
keepaliveMs: opts.keepaliveMs,
maxInFlight: opts.maxInFlight,
initialKg: kg,
lastSeq: extra.lastSeq,
epoch: extra.epoch,
dispatcher: this.dispatcher,
lazy: extra.handle !== undefined,
createKg: extra.handle !== undefined ? (name) => this.createIfMissing(name) : undefined,
});
const types: ConnectionEventType[] = [
'disconnected',
'reconnected',
'session_reset',
'notification_gap',
'closed',
];
for (const type of types) {
conn.events.addEventListener(type, (event) => {
const detail = (event as CustomEvent).detail ?? {};
this.events.dispatchEvent(
new CustomEvent(type, { detail: { ...detail, knowledgeGraph: extra.handle } }),
);
});
}
return conn;
}

/**
* A handle's graph refused the open: create it if it does not exist. The
* client's own connection creates it (`conflict` means it exists, so the
* refusal stands) and switches back, because `.kg create` also moves the
* creating session onto the new graph, which could then not be dropped.
* Creations run one at a time so none reads another's switch as home.
*/
private createIfMissing(name: string): Promise<boolean> {
const run = this.creating.then(() => this.createKg(name));
this.creating = run.catch(() => undefined);
return run;
}

private async createKg(name: string): Promise<boolean> {
await this.conn.connect();
const home = this.conn.boundKg;
try {
await this.conn.execute(`.kg create ${name}`);
} catch (e) {
if (e instanceof QueryError && e.code === 'conflict') return false;
throw e;
}
if (home && this.conn.currentKg !== home) await this.conn.execute(`.kg use ${home}`);
return true;
}

// ── Connection lifecycle ────────────────────────────────────────

/** Connect and authenticate. */
/** Connect and authenticate the client's own connection. */
async connect(): Promise<void> {
await this.conn.connect();
}

/** Close the connection. */
/** Close every connection: the client's own and each knowledge graph handle's. */
async close(): Promise<void> {
await this.conn.close();
await Promise.all([
this.conn.close(),
...[...this.kgs.values()].map((kg) => kg.connection.close()),
]);
}

// ── Properties ──────────────────────────────────────────────────
Expand All @@ -115,17 +208,27 @@ export class InputLayer {
return this.conn.role;
}

/** The highest notification `seq` dispatched; pass it back with `epoch` as `lastSeq`. */
get lastSeq(): number {
return this.conn.lastSeq;
return this.dispatcher.lastSeq;
}

/** The engine run (`stream_epoch`) notification `seq` numbers belong to. */
get epoch(): string | undefined {
return this.conn.epoch;
}

// ── KG management ───────────────────────────────────────────────

/** Get a KnowledgeGraph handle. Switches the session's active KG. */
/**
* Get a KnowledgeGraph handle. Each handle has its own connection, bound to
* its graph when it first opens (creating the graph if it is missing), so
* handles never switch graphs under one another.
*/
knowledgeGraph(name: string): KnowledgeGraph {
let kg = this.kgs.get(name);
if (!kg) {
kg = new KnowledgeGraph(name, this.conn);
kg = new KnowledgeGraph(name, this.newConnection(name, { handle: name }));
this.kgs.set(name, kg);
}
return kg;
Expand All @@ -139,7 +242,9 @@ export class InputLayer {

async dropKnowledgeGraph(name: string): Promise<void> {
await this.conn.execute(`.kg drop ${name}`);
const kg = this.kgs.get(name);
this.kgs.delete(name);
await kg?.connection.close();
}

// ── User management ─────────────────────────────────────────────
Expand Down Expand Up @@ -201,7 +306,10 @@ export class InputLayer {
// ── Notifications ───────────────────────────────────────────────

/**
* Register a notification callback.
* Register a notification callback. Notifications arrive on every open
* connection of the client: its own (on `initialKg`) and each knowledge
* graph handle's once that handle has made a call. One seen on several
* connections is delivered once.
*
* @param eventType - Filter by event type (e.g. "persistent_update")
* @param callback - Function to call when event arrives
Expand All @@ -212,16 +320,16 @@ export class InputLayer {
callback: NotificationCallback,
opts?: { relation?: string; knowledgeGraph?: string },
): void {
this.conn.dispatcher.on(eventType, opts ?? {}, callback);
this.dispatcher.on(eventType, opts ?? {}, callback);
}

/** Remove a notification callback. */
off(callback: NotificationCallback): void {
this.conn.dispatcher.off(callback);
this.dispatcher.off(callback);
}

/** Async iterator yielding notification events. */
async *notifications(): AsyncIterableIterator<NotificationEvent> {
yield* this.conn.dispatcher.events();
yield* this.dispatcher.events();
}
}
Loading
Loading