fix(tools): reap ADK sessions of dead MCP connections in to_mcp_server - #7155
Open
harshal-96 wants to merge 5 commits into
Open
harshal-96 wants to merge 5 commits into
harshal-96 wants to merge 5 commits into
Conversation
to_mcp_server keeps one ADK session per MCP connection in a WeakKeyDictionary. When a connection is garbage-collected the map entry disappears, but the ADK session it pointed to stays in the session service forever, with its full event history. A long-running server therefore accumulates one dead conversation per closed connection, and a stateless streamable HTTP deployment, where the SDK builds a fresh transport for every request, leaks one session per tool call. Track the id of every session entered into the connection map and, at the start of each tool call, delete the sessions whose connection is no longer reachable. Reaping runs lazily from the tool call rather than a GC callback because finalizers may fire without a running event loop. Also document how a stateless streamable HTTP deployment behaves: each call is a fresh single-turn conversation whose session is reclaimed. Tested with mcp 1.26.0 and 2.2.0: 17 passed each.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Author
|
@googlebot I signed it! |
Callers who wire to_mcp_server to a persistent session service may want finished conversations to remain readable after their connection dies, e.g. for audit. delete_orphaned_sessions=False disables the reaping and leaves session lifecycle to the caller. The default stays True so a long-running server's memory is bounded out of the box.
The reap ran inline before the live tool call, so with a slow session service (e.g. database or Vertex backed) the first call after a burst of disconnects waited on every delete. Start it as a background task instead: - At most one reap runs at a time, and the task is referenced so it is not garbage-collected mid-run. A task left over from another event loop does not block reaping in the current one. - The task and its done callback are created in an empty contextvars Context. Both capture the current context by default, and a copy of the tool call's would keep the request, and on MCP SDK 1.x the connection, alive for as long as the task runs, so a stuck reap would pin the very connection whose session it should delete. - A done callback logs unexpected reap errors instead of leaving them for asyncio to report as never retrieved.
The background reap deleted orphaned sessions one at a time and a new reap only started once the previous one finished, so throughput was capped at one session per delete round trip. With a 50 ms delete and a call every 20 ms the service held a growing backlog that only drained when further calls arrived. Each reap now repeats until no orphan is left, picking up connections that close while it runs, and deletes each batch with a fixed pool of _MAX_CONCURRENT_DELETES workers sharing one iterator, so a backlog of any size costs a bounded number of tasks and a database-backed service sees a bounded number of concurrent deletes. Once as many deletes have failed as there are workers, the workers stop deleting and the pass ends, so an unavailable service sees fewer than twice the pool size of attempts and one summary warning per pass instead of one of each per waiting session, while a single session that keeps failing does not hold up the rest of the batch. Every id the pass did not delete is put back for a later call to retry. Each delete is bounded by _DELETE_TIMEOUT_SECONDS: only one reap runs at a time, so a delete that never returned would otherwise stop reaping for good.
A reap cancelled mid-pass, for example at event loop shutdown, dropped the ids it had claimed but not yet deleted: the in-flight ones and the ones no worker had started. They were never deleted afterwards. Put both back so a later reap retries them; deleting a session that is already gone is a no-op in the built-in session services. Each delete now waits with asyncio.wait instead of asyncio.wait_for. Before Python 3.12, wait_for can swallow a cancellation that arrives as the delete finishes, which let a cancelled reap keep draining its whole backlog. With 400 orphans and 50 ms deletes, cancelling took about 2.7 s on Python 3.10 and 3.11 and now returns immediately. Also document that only sessions created by this server process are tracked, so with a persistent session service, sessions orphaned before a restart are not deleted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #7154
Description
to_mcp_serverkeeps one ADK session per MCP connection in aweakref.WeakKeyDictionary. When a connection is garbage collected the map entry disappears, but the ADK session it pointed to stays in the session service forever, with its full event history. A long running server therefore accumulates one dead conversation per closed connection, and a stateless streamable HTTP deployment, where the MCP SDK builds a fresh transport for every request, leaks one session per tool call.This PR tracks the id of every session entered into the connection map. Tool calls start a background reap, at most one at a time, that deletes the sessions whose connection is no longer reachable through the weak map.
Design notes:
runners.pygives toolset cleanup. Only one reap runs at a time, so a delete that never returned would otherwise stop reaping for good. The wait usesasyncio.waitrather thanasyncio.wait_for: before Python 3.12,wait_forcan swallow a cancellation that arrives as the delete finishes.contextvars.Context. Both capture the current context by default, and on MCP SDK 1.x the request lives in a ContextVar that references the connection, so a copied context would keep the very connection whose session the reap should delete alive for as long as the task runs.to_mcp_server(..., delete_orphaned_sessions=False)disables reaping entirely and leaves session lifecycle to the caller, for deployments that read those records after the fact. Only sessions created by the running server process are tracked, so with a persistent session service, sessions orphaned before a restart are not deleted.to_mcp_serverdocstring now documents stateless streamable HTTP behavior: each call is a fresh single turn conversation whose session is reclaimed.Testing plan
18 new unit tests in
tests/unittests/tools/mcp_tool/test_agent_to_mcp.py:test_reap_deletes_only_sessions_no_longer_reachable: the reap deletes exactly the unreachable ids.test_session_of_a_collected_connection_is_reaped: a conversation does not outlive its connection.test_per_request_connections_do_not_accumulate_sessions: the stateless per request pattern stays flat instead of leaking one session per call.test_reap_failure_does_not_raise_and_is_retried: a session service outage does not fail the live tool call, and the orphan is deleted once the service recovers.test_reap_partial_failure_requeues_only_the_failed_ids: in a batch, a failed delete is retried later without undoing the deletes that were in flight and succeeded.test_reap_stops_after_a_failure_instead_of_hammering_the_service: with the session service down and 80 sessions waiting, a reap makes fewer than twice the pool size of delete attempts, logs one summary warning, and keeps every id for retry.test_one_failing_session_does_not_hold_up_the_rest: a session the service keeps rejecting does not stop the other 100 from being deleted in the same pass.test_cancelled_reap_stops_promptly_and_keeps_unfinished_ids: with 400 sessions waiting and 50 ms deletes, a reap cancelled after 0.1 s stops without draining the backlog, and every id is either deleted or still tracked.test_hung_delete_times_out_instead_of_stopping_reaping: a delete that never returns times out and is retried later, and the rest of the batch is still deleted.test_reap_task_count_stays_bounded_for_a_large_backlog: a backlog of 1000 is drained without a task per waiting session.test_slow_deletes_run_concurrently_up_to_the_cap: with a slow session service, a backlog is deleted concurrently and never with more than the cap in flight.test_reap_drains_orphans_that_appear_while_it_runs: sessions orphaned while a reap is deleting are picked up by the same reap.test_call_tool_reaps_conversation_of_closed_connection: end to end through a real in-memory MCP client and server.test_call_tool_retains_sessions_when_deletion_is_opted_out:delete_orphaned_sessions=Falsekeeps every session.test_call_tool_does_not_wait_on_slow_session_deletes: with the session service's delete blocked, the tool call still completes; it times out with an inline reap.test_background_reap_failure_is_logged_not_raised: an unexpected reap error is logged and the call succeeds.test_background_reap_does_not_inherit_the_callers_context: a ContextVar set around the call is not visible inside the reap.test_reap_is_not_blocked_by_a_task_from_another_event_loop: a reap stuck in a stopped event loop does not stop reaping in the next one.Each safeguard was checked by removing it and confirming its test fails, on mcp 1.26.0 and 2.2.0: putting back in-flight and unstarted ids on cancellation, the delete timeout, the failure threshold (both never stopping and stopping at the first failure), the bounded worker pool, concurrent deletes, the drain loop, per-id failure retry, background start, the done callback, the empty context, and the event loop check. Switching the delete back to
asyncio.wait_forfails the cancellation test in 4 of 5 runs on Python 3.10 and 3.11 (the race does not exist on 3.12).Results:
pytest tests/unittests/tools/mcp_tool/test_agent_to_mcp.py: 31 passed on Python 3.10, 3.11, and 3.12, each with mcp 1.24.0, 1.26.0, and 2.2.0 (the pin admits 1.x and 2.x); 20 consecutive runs stable on each of 3.10, 3.11, and 3.12.pytest tests/unittests/tools/mcp_tool: 389 passed (Python 3.12, mcp 2.2.0), and 443 passed on Python 3.10, 3.12 and 3.14 with the branch merged onto current main (f44d512). The fulltests/unittestssuite merged onto f44d512 gives 16573 passed versus 16555 on main: the 18 new tests, with the same 3 failures as main (unrelated, fixed in fix(live): stop an early-closed merged live stream from hanging #7358 and fix: keep google-auth transport imports out of Agent import #7359).run_streamable_http_async(stateless_http=True)on localhost, one freshstreamable_http_clientconnection per call. 9 calls leave 9 sessions in the service on main and 1 with this PR (the most recent call's session, reclaimed on the next call).Runnerand session services, mcp 1.26.0:InMemorySessionServiceandDatabaseSessionServiceon SQLite (also with 50 ms added per delete) hold 1 to 2 sessions over three runs, and no delete failed, so concurrent deletes do not trip SQLite locking. With the real Runner each call takes longer, so at this load the one-at-a-time version mostly kept up as well (2 to 6 sessions); the difference shows once calls outpace deletes, as in the stub run. With mcp 2.2.0 the sessions still held belong to connections not yet garbage collected, which are not orphans; after a collection, one call reaps them all.asyncio.wait_forit kept deleting for about 2.7 s on 3.10 and 3.11.sessions left in the service: 10on main,1with this PR.Formatting and checks: pyink 25.12, isort 8.0.1, ruff 0.15.17, codespell 2.4.2, and
scripts/compliance_checks.pyare clean;mypyreports no errors in the changed module, the same as main; pylint with the repositorypylintrcrates it 10.00/10.