test: leave the cancelled stream after it has started, not on a timer - #52
Merged
Merged
Conversation
streaming_client_disconnect_emits_cancelled_gateway_log dropped the
request on a 150 ms client timeout. The gateway returns the SSE response
only after auth, limits and routing; when those took longer than 150 ms
under load, the client left before the stream existed, hyper dropped the
handler, and no gateway_logs row was ever written ("cancelled row never
landed"). Reproduced deterministically with a 5 ms timeout.
The client now waits for the response headers, which the gateway sends
before calling the upstream, and drops the response: the disconnect
always lands on a running stream, which is what the test is about. The
upstream's delay goes from 5 s to 60 s so it can never answer first.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause of the
streaming_client_disconnect_emits_cancelled_gateway_logflake ("gateway_logs cancelled row never landed").The test disconnected with a 150 ms client timeout. The gateway returns the SSE response only after the API-key middleware, rate limits/budget/access, and routing. When those took longer than 150 ms (a loaded machine), the client left before the stream existed: hyper dropped the handler future, the stream's tail never ran, and no row was written — which is exactly the reported failure. Reproduced deterministically by lowering the timeout to 5 ms (fails after the 10 s wait with the same message).
Fix: the client waits for the response headers — the gateway sends them before it calls the upstream (the upstream call starts on the body's first poll) — then drops the response. The disconnect now always lands on a running stream, which is what the test pins (499 +
client_cancelled). The upstream delay goes from 5 s to 60 s so it can never answer first. No retries or sleeps added.Not changed: a client that leaves during auth/limits/routing still leaves no
gateway_logsrow. That window is milliseconds and outside what the stream path records today; covering it would need a drop guard in the handler — a separate decision.Local: the test passes 3/3 with the fix; full integration suite unaffected (test-only change).
🤖 Generated with Claude Code