Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
pull-requests: read
outputs:
rust: ${{ steps.filter.outputs.rust }}
python: ${{ steps.filter.outputs.python }}
steps:
- uses: actions/checkout@v4

Expand All @@ -53,6 +54,10 @@ jobs:
- 'clippy.toml'
- 'Makefile'
- '.github/workflows/ci.yml'
python:
- 'packages/inputlayer-py/**'
- 'packages/conformance/**'
- '.github/workflows/ci.yml'

fmt:
name: Format
Expand Down Expand Up @@ -153,3 +158,28 @@ jobs:

- name: Run unit + integration tests
run: make unit-test

# The Python SDK's unit tests, including the connection core against a
# mock server replaying packages/conformance. The live tests run in the
# full suite (make python-test-live), which builds the engine.
python-sdk:
name: Python SDK
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

- name: Run unit tests
run: make python-test
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -382,7 +382,7 @@ python-test-live:
INPUTLAYER_USER=admin \
INPUTLAYER_PASSWORD=admin \
uv run --extra dev pytest \
tests/test_langchain.py tests/test_integration.py -v; \
tests/test_langchain.py tests/test_integration.py tests/test_connection_live.py -v; \
TEST_EXIT=$$?; \
cd - > /dev/null; \
kill $$SERVER_PID 2>/dev/null || true; \
Expand Down
25 changes: 23 additions & 2 deletions docs/content/docs/guides/python-sdk.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,30 @@ il = InputLayer(
username="admin",
password="admin",
auto_reconnect=True, # reconnect on disconnect (default: True)
reconnect_delay=1.0, # seconds between attempts (default: 1.0)
reconnect_delay=1.0, # first backoff in seconds, doubling to 30 with jitter (default: 1.0)
max_reconnect_attempts=10, # give up after N failures (default: 10)
default_timeout=30.0, # deadline of each call in seconds (default: 30.0)
keepalive=20.0, # ping after this many idle seconds (default: 20.0)
)
await il.connect()
# ... use the client ...
await il.close()
```

### How connections behave

- **One connection per knowledge graph.** `il.knowledge_graph(name)` gets its own connection, bound to that graph when it opens (on first use; a missing graph is created then). Handles never switch a shared connection between graphs, so one graph's subscriptions and session state survive calls on another.
- **Calls are pipelined.** Concurrent calls on one handle are sent without waiting for each other and their replies are matched by request id; at most 15 are unanswered at once, which keeps one of the server's 16 slots free for a cancel.
- **Deadlines.** Every call runs under a deadline (`default_timeout`, or `timeout=` on `kg.execute(iql, timeout=2.0)`) that the engine enforces (time spent waiting for a reconnect or a free request slot counts against it); past it the call raises `DeadlineExceeded` and nothing was applied. If no reply comes at all, the SDK cancels the request on the server: a query then raises `DeadlineExceeded`, but a write raises `OutcomeUnknownError` unless the server confirmed the cancel, because it may have committed, so read back before retrying it. Cancelling the awaiting task cancels the request too.
- **Live while idle.** One reader task per connection always runs, so notifications arrive while no call is in flight, and a ping every `keepalive` idle seconds keeps a listening client under the server's idle timeout.
- **Reconnects.** A dropped connection fails its pending calls with `ConnectionLost` (`may_have_committed` is set for a write that was sent: read back before retrying) and reconnects to the same knowledge graph, resuming notifications after the last `seq` seen (a reconnect with no `seq` to resume from reports a `notification_gap`). Session rules and facts do not survive it. `il.events` reports `disconnected`, `session_reset`, `reconnected`, `notification_gap` and `closed`:

```python
@il.events.on("session_reset")
async def recreate_session_rules(event):
await il.knowledge_graph(event.knowledge_graph).session.define_rules(MySessionView)
```

### Sync Client

If you're working in a script, notebook, or anywhere async isn't practical, use `InputLayerSync` instead. It wraps the async client and exposes the same API, just without `await`:
Expand Down Expand Up @@ -721,9 +737,14 @@ from inputlayer import (
ValidationError, # invalid data
QueryError, # base class for program and commit errors
StatementFailedError, # some statements of a program failed
OutcomeUnknownError, # commit outcome unknown; store may be read-only
OutcomeUnknownError, # write outcome unknown; read back before retrying
StoreReadOnlyError, # write refused until restart recovery
QueryTimeoutError, # query took too long
DeadlineExceeded, # the call's deadline passed; nothing applied
Cancelled, # the call was cancelled; nothing applied
ConnectionLost, # connection closed before the reply
RateLimited, # refused by the server's rate limit; already retried once
ProtocolError, # server could not read the request; an SDK bug
PermissionError, # insufficient permissions
KnowledgeGraphNotFoundError,
KnowledgeGraphExistsError,
Expand Down
44 changes: 44 additions & 0 deletions packages/conformance/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# SDK conformance fixtures

Frame sequences every InputLayer SDK replays through a mock server in its unit
tests, so the Python and TypeScript connection cores are held to the same
behaviour. The frames follow the `/ws` protocol (`ws-protocol/src`,
`docs/spec/asyncapi.yaml`).

## `connection/`

One JSON file per scenario:

| Field | Meaning |
|---|---|
| `name`, `description` | What the scenario proves. |
| `routes` | Subscription ids the client routes before its calls (the route takes its generation from the `.subscribe` reply). |
| `calls` | Requests the client issues in order without waiting for replies: `{"execute": program, "expect": ...}`. |
| `server` | What the mock server does once the client has authenticated, in order. |
| `expect` | What the client observed besides its calls' outcomes. |

Server steps:

- `{"recv": {...}, "as": "q"}`: read the next request (keepalive pings are answered and skipped); every field given must match. Its `id` is remembered as `q`.
- `{"send": frame}`: send a frame; a string value `"$q"` is replaced by the id remembered as `q`.
- `{"close": code}`: close the socket with that WebSocket close code.

A call's `expect` is `{"rows": [...], "columns": [...]}` for a result (`columns` optional), or `{"error": kind, "code": ..., "may_have_committed": ...}` (`code` and `may_have_committed` checked only when given). Error kinds:

| Kind | Python | TypeScript |
|---|---|---|
| `query` | `QueryError` (none of the kinds below) | `QueryError` |
| `statement_failed` | `StatementFailedError` | `StatementFailedError` |
| `deadline_exceeded` | `DeadlineExceeded` | `DeadlineExceeded` |
| `cancelled` | `Cancelled` | `Cancelled` |
| `outcome_unknown` | `OutcomeUnknownError` | `OutcomeUnknownError` |
| `protocol` | `ProtocolError` | `ProtocolError` |
| `internal` | `InternalError` | `InternalError` |
| `connection_lost` | `ConnectionLost` | `ConnectionLost` |

The top-level `expect` may hold `notifications` (the `seq`s delivered, in
order), `pushes` (`subscription`, `generation` and `seq` of each push routed to
a subscription), `stale_pushes` (pushes dropped as stale or unknown) and
`events` (connection event types emitted, in order).

The client runs with reconnecting off, so a `close` ends the scenario.
80 changes: 80 additions & 0 deletions packages/conformance/connection/chunked_result.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
{
"name": "chunked_result",
"description": "A streamed result is assembled in chunk order and checked against the counts its end announces.",
"calls": [
{
"execute": "?a(X)",
"expect": {
"rows": [
[
1
],
[
2
],
[
3
]
],
"columns": [
"x"
]
}
}
],
"server": [
{
"recv": {
"type": "execute"
},
"as": "q"
},
{
"send": {
"type": "result_start",
"id": "$q",
"columns": [
"x"
],
"total_count": 3,
"truncated": false,
"execution_time_ms": 1
}
},
{
"send": {
"type": "result_chunk",
"id": "$q",
"rows": [
[
1
],
[
2
]
],
"chunk_index": 0
}
},
{
"send": {
"type": "result_chunk",
"id": "$q",
"rows": [
[
3
]
],
"chunk_index": 1
}
},
{
"send": {
"type": "result_end",
"id": "$q",
"row_count": 3,
"chunk_count": 2
}
}
]
}
55 changes: 55 additions & 0 deletions packages/conformance/connection/chunked_result_count_mismatch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"name": "chunked_result_count_mismatch",
"description": "The end frame announces more rows than the chunks carried: an internal error, never a short result.",
"calls": [
{
"execute": "?a(X)",
"expect": {
"error": "internal"
}
}
],
"server": [
{
"recv": {
"type": "execute"
},
"as": "q"
},
{
"send": {
"type": "result_start",
"id": "$q",
"columns": [
"x"
],
"total_count": 3,
"truncated": false,
"execution_time_ms": 1
}
},
{
"send": {
"type": "result_chunk",
"id": "$q",
"rows": [
[
1
],
[
2
]
],
"chunk_index": 0
}
},
{
"send": {
"type": "result_end",
"id": "$q",
"row_count": 3,
"chunk_count": 1
}
}
]
}
100 changes: 100 additions & 0 deletions packages/conformance/connection/chunked_result_missing_chunk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
{
"name": "chunked_result_missing_chunk",
"description": "A chunk index skips one: the call fails as an internal error, and the connection keeps serving the next call.",
"calls": [
{
"execute": "?a(X)",
"expect": {
"error": "internal"
}
},
{
"execute": "?b(X)",
"expect": {
"rows": [
[
9
]
]
}
}
],
"server": [
{
"recv": {
"type": "execute",
"program": "?a(X)"
},
"as": "q"
},
{
"recv": {
"type": "execute",
"program": "?b(X)"
},
"as": "r"
},
{
"send": {
"type": "result_start",
"id": "$q",
"columns": [
"x"
],
"total_count": 3,
"truncated": false,
"execution_time_ms": 1
}
},
{
"send": {
"type": "result_chunk",
"id": "$q",
"rows": [
[
1
]
],
"chunk_index": 0
}
},
{
"send": {
"type": "result_chunk",
"id": "$q",
"rows": [
[
3
]
],
"chunk_index": 2
}
},
{
"send": {
"type": "result_end",
"id": "$q",
"row_count": 2,
"chunk_count": 2
}
},
{
"send": {
"type": "result",
"columns": [
"x"
],
"rows": [
[
9
]
],
"row_count": 1,
"total_count": 1,
"truncated": false,
"execution_time_ms": 0,
"id": "$r"
}
}
]
}
Loading
Loading