feat(gateway): record a client that leaves before its response exists - #54
Merged
Merged
Conversation
A disconnect was recorded only once a stream had started. A client that left while its key's roles loaded, the limits ran, a route was picked or a whole answer was awaited left no trace: hyper dropped the handler and nothing after the await point ran. The API-key middleware now arms a drop guard (proxy::EarlyCancel) for the AI gateway as soon as the key is known, and disarms it when the handler hands back any response, since every response path writes its own row. Dropped still armed, it writes one gateway_logs row: status 499, stream_outcome client_cancelled, cancelled_before response, no tokens, no cost, no provider. The handler fills in the trace id and the model as it learns them. A started stream is unaffected and still records its own cancel, so nothing is logged twice. Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A disconnect was recorded only after a stream had started (
StreamOutcome::ClientCancelled, 499). A client that left while its key's roles and limits loaded, the pre-flight stages ran, a route was picked, or a whole (non-streamed) answer was awaited left no row at all: hyper drops the handler future and nothing after the await point runs.How
proxy::EarlyCancel— a drop guard. The API-key middleware arms it for the AI gateway surface as soon as the key row is known (a client that leaves before that is nobody yet and writes nothing), and disarms it afternext.runreturns — whatever the response, since every response and refusal path already writes its own row. The auth refusals after arming run inside the same scope, so they disarm too; only a dropped future leaves it armed.gateway_logsrow: status 499,stream_outcome: client_cancelled(the marker cancelled streams already carry),cancelled_before: response, zero tokens, zero cost, no provider.EarlyCancelSlotrequest extension: the trace id and the aliased model. The fuller identity (email, IP) is filled in by the middleware once built.Tests (
early_cancel.rs)client_cancelled, never a success.cancelled_before, and no second row later.Local: fmt, clippy (
--all-targets,--lib), 688 unit tests, full integration suite (345 passed) on own containers.🤖 Generated with Claude Code