Skip to content

feat(gateway): record a client that leaves before its response exists - #54

Merged
fylorn merged 1 commit into
devfrom
feat/log-early-disconnects
Sep 24, 2026
Merged

fylorn merged 1 commit into
devfrom
feat/log-early-disconnects

Conversation

@fylorn

@fylorn fylorn commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

A disconnect was recorded only after a stream had started (StreamOutcome::ClientCancelled, 499). A client that left while its key's roles and limits loaded, the pre-flight stages ran, a route was picked, or a whole (non-streamed) answer was awaited left no row at all: hyper drops the handler future and nothing after the await point runs.

How

  • proxy::EarlyCancel — a drop guard. The API-key middleware arms it for the AI gateway surface as soon as the key row is known (a client that leaves before that is nobody yet and writes nothing), and disarms it after next.run returns — whatever the response, since every response and refusal path already writes its own row. The auth refusals after arming run inside the same scope, so they disarm too; only a dropped future leaves it armed.
  • Dropped armed, it writes one gateway_logs row: status 499, stream_outcome: client_cancelled (the marker cancelled streams already carry), cancelled_before: response, zero tokens, zero cost, no provider.
  • The handler fills in what it learns on the way through an EarlyCancelSlot request extension: the trace id and the aliased model. The fuller identity (email, IP) is filled in by the middleware once built.
  • No double logging: by the time a stream exists the handler has returned and the guard is disarmed; the stream's tail records its own cancel as before. WebSocket turns pass no slot — the connection's relay records a turn the client left (as a dropped stream).

Tests (early_cancel.rs)

  • Deterministic: a non-streamed request is dropped once the (60 s) upstream has received it → exactly one 499 row with the model, no tokens.
  • Leaving at once (5 ms client timeout, five times) during auth/limits/routing → at least one and at most one row per client, all client_cancelled, never a success.
  • A started stream the client leaves → exactly one row, without cancelled_before, and no second row later.
  • The first two fail on the old code (no rows) and pass now.

Local: fmt, clippy (--all-targets, --lib), 688 unit tests, full integration suite (345 passed) on own containers.

🤖 Generated with Claude Code

A disconnect was recorded only once a stream had started. A client that
left while its key's roles loaded, the limits ran, a route was picked or
a whole answer was awaited left no trace: hyper dropped the handler and
nothing after the await point ran.

The API-key middleware now arms a drop guard (proxy::EarlyCancel) for
the AI gateway as soon as the key is known, and disarms it when the
handler hands back any response, since every response path writes its
own row. Dropped still armed, it writes one gateway_logs row: status 499,
stream_outcome client_cancelled, cancelled_before response, no tokens,
no cost, no provider. The handler fills in the trace id and the model as
it learns them. A started stream is unaffected and still records its own
cancel, so nothing is logged twice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@fylorn
fylorn merged commit 7e95183 into dev Sep 24, 2026
6 checks passed
@fylorn
fylorn deleted the feat/log-early-disconnects branch September 24, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant