Repository navigation
fix: escape SSE-unsafe Unicode line separators in streamed payloads - #1433
devinhuang2001 wants to merge 1 commit into
Conversation
`json_utils.dumps()` serializes SSE `data:` payloads with `ensure_ascii=False`, so U+2028, U+2029 and U+0085 go on the wire as raw UTF-8. httpx's `aiter_lines()` follows `str.splitlines()`, which treats those three as line endings, so a single `data:` line is split and `json.loads()` fails with `JSONDecodeError`. Re-subscribing sends the same snapshot, so the task stays unreadable over streaming. Escape exactly those three characters as `\uXXXX` in `dumps()`. JSON escapes are ASCII, so the payload stays on one line; `json.loads()` restores the original characters, and all other non-ASCII text (CJK, emoji) still goes out raw. Verified against upstream main 28f8a62. See the PR body for the evidence table and the remaining client-side limitation. Fixes a2aproject#1432 Signed-off-by: devinhuang2001 <[email protected]>
871c16f to
fc3254d
Compare
🧪 Code Coverage (vs
|
|
Thanks, the diagnosis is spot on That behavior started after merging #1132, which was mine. We'd rather not take the server-side change though. Escaping in The fix belongs in Closing this one and #1432 stays open. Please let me know if you want to proceed with creating the next PR on client side Once again, thanks for contributing and the PR is good, but fixing the client would resolve the issue at its root |
json_utils.dumps()writes SSEdata:payloads withensure_ascii=False, soU+2028 / U+2029 / U+0085 go on the wire as raw UTF-8.
httpx.aiter_lines()uses
str.splitlines(), which treats those three as line breaks: onedata:line becomes several and
json.loads()raisesJSONDecodeError. Re-subscribingsends the same snapshot, so the task stays unreadable over streaming.
This change escapes exactly those three characters as
\uXXXXindumps().Escapes are ASCII, so the frame stays on one line,
json.loads()restores theoriginal text, and CJK / emoji still pass through unescaped.
dumps()is theserializer used by
rest_dispatcher.py,jsonrpc_dispatcher.pyand the v0.3REST adapter for streamed events, so all three streaming paths are covered.
Verification
tests/utils/test_json_utils.py+tests/client/transports/: 130 passed(the 12 existing
aiter_linesmocks are untouched and still pass).httpx.Responsethroughparse_sse_streamfor all three separators at chunk sizes 1 / 7 / 4096.
dumps()to upstream makes all 13 new assertionsfail; restoring it makes 26/26 pass in the two touched test files.
28f8a62f25. sse-starlette's ownframing was not re-tested here (the issue reporter covered it).
Known limitation
The client parser still uses
httpx.aiter_lines(), so a third-party A2A serverthat emits these characters raw can still split a line. Fixing that means
swapping to
aiter_text()plus a CR/LF-only decoder and migrating the existingaiter_linesmocks across the transport tests. I left that out of this bug-fixPR to keep the fix small; happy to follow up if maintainers prefer the
client-side approach instead.
Checklist
fix:)Lint Code Base(
ruff check,ruff format --check,ty check) passes on this head.The formatter was not run locally because the ruff wheel would not
download here; CI is the authoritative check.
Fixes #1432 🦕