Skip to content

feat(mcp): start MCP servers in bounded parallel batches - #358

Open
raymondginger2018-sudo wants to merge 2 commits into
lessweb:mainfrom
raymondginger2018-sudo:pr/mcp-parallel-startup
Open

raymondginger2018-sudo wants to merge 2 commits into
lessweb:mainfrom
raymondginger2018-sudo:pr/mcp-parallel-startup

Conversation

@raymondginger2018-sudo

Copy link
Copy Markdown

What

MCP server startup in McpManager.initialize() was strictly serial: each configured server paid its own spawn + handshake before the next one started, so startup time was the sum over all servers.

This starts servers in batches of 5 (Promise.allSettled), which cuts total startup to roughly 1/5 for typical configs while keeping the process/memory peak bounded — unlike launching every server at once.

Change

packages/core/src/mcp/mcp-manager.ts (+9/-2):

const entries = Object.entries(servers);
const CONCURRENCY = 5;
for (let i = 0; i < entries.length; i += CONCURRENCY) {
  if (this.disposed) break;
  const batch = entries.slice(i, i + CONCURRENCY);
  await Promise.allSettled(batch.map(([name, config]) => this.connectServer(name, config)));
}

Promise.allSettled also isolates failures: previously an exception thrown while connecting one server aborted the loop and left every later server unstarted. Per-server status reporting and crash handling (onServerCrash) are unchanged.

Why this is safe

connectServer mutates clients / tools / prompts / resources synchronously between its await points, and the namespaced-name reservation block contains no await, so concurrent callers cannot interleave and corrupt that state. The only observable difference is that status/log lines may now appear in a different order.

Tests

packages/core/src/tests/mcp-manager-parallel.test.ts (new, 2 tests, no child processes — connectServer is stubbed so the test exercises scheduling only):

  • all 12 configured servers start, with peak in flight > 1 and <= 5;
  • a disconnect() raised mid-startup starts no further batch.

Checked that the test has teeth: reverting the loop to the previous serial form makes the first test fail with expected overlapping connections, saw peak=1.

Verification (local, full CI equivalent)

npm run check (typecheck + eslint + prettier) · npm run build --workspace=@vegamo/deepcode-core · npm run bundle · npm run build:vscode · npm test — all pass (59 tests, 0 failures).

Overlap

Open PR #198 also edits mcp-manager.ts, but its hunks are disjoint from this one (import restructure + createMcpClient inside connectServer), so the two can land in either order.

Initialize configured MCP servers in batches of 5 instead of strictly
serial. Total startup drops to roughly 1/5 of the serial path while the
bounded batch avoids spawning every process at once (memory peak).
Promise.allSettled keeps a single failing server from blocking its batch;
per-server status/failure handling is unchanged.

Extracted from the earlier closed PR lessweb#266 (kept as a focused change).
Assert that initialize() runs connects concurrently (peak in flight > 1)
while staying inside the batch bound, and that a disconnect raised during
startup starts no further batch. Both use a stubbed connectServer so the
test exercises scheduling only and does not spawn child processes.

Reverting the batch loop back to serial makes the first test fail with
"expected overlapping connections, saw peak=1".
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant