Skip to content

Fix process resource leak and potential deadlock in CommandRunner - #795

Merged
seesharper merged 7 commits into
masterfrom
fix/commandrunner-process-leak
May 28, 2026
Merged

Fix process resource leak and potential deadlock in CommandRunner#795
seesharper merged 7 commits into
masterfrom
fix/commandrunner-process-leak

Conversation

@seesharper

Copy link
Copy Markdown
Collaborator

Summary

Fixes #794Cannot assign requested address on first script run.

  • Wrap Process objects in using in both Execute and Capture, so their underlying socket file descriptors are closed deterministically after each command rather than waiting for GC finalization
  • Read stdout and stderr concurrently in Capture via ReadToEndAsync to prevent a deadlock where a child that fills the stderr buffer stalls while the parent is blocked on ReadToEnd for stdout

Root cause

On macOS, .NET implements redirected stdio (triggered by RedirectStandardOutput = true) using socketpair(AF_UNIX, ...). Every call to CommandRunner.Execute or CommandRunner.Capture previously leaked the two read-end socket fds until the GC finalizer ran. On first script run, the compilation phase calls both of these methods (for dotnet restore and dotnet publish). When the GC finalizer later closes those stale fds, the OS can recycle the fd numbers. If CliWrap's script execution has allocated a new socket that received a recycled fd number, the finalizer tears it down mid-use, producing EADDRNOTAVAIL in PipeStream.ReadAsyncCore. On second run the cached DLL is used, no CommandRunner calls are made, no fds leak, so the issue disappears.

Test plan

  • Run the reproduction script from Cannot assign requested address #794 with a cleared cache and confirm it succeeds on first run
  • Confirm behaviour is unchanged on second run and with --disable-isolated-load-context

🤖 Generated with Claude Code

seesharper and others added 7 commits May 22, 2026 14:19
Dispose Process objects after use to release underlying socket file
descriptors immediately rather than waiting for GC finalization. On
macOS, .NET implements redirected stdio via socketpair(AF_UNIX), so
leaking Process objects leaves open socket fds that can be recycled and
torn down by a finalizer mid-use, causing EADDRNOTAVAIL in downstream
pipe consumers (e.g. CliWrap) on first run of a script.

Also read stdout and stderr concurrently in Capture to prevent the
deadlock where a child that fills the stderr socket buffer blocks
forever while the parent is stuck on ReadToEnd for stdout.

Fixes #794

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…d context

On macOS, AssemblyLoadContext.LoadFromAssemblyPath() creates SafeFileHandle
objects whose fds are explicitly closed but not GC.SuppressFinalize'd. On
non-cached runs, Roslyn compilation creates GC pressure that can trigger a
concurrent GC while CliWrap's child process socketpairs are in use. If a
stale finalizer fires for an fd that was recycled as a CliWrap socket, the
live socket is torn down → EADDRNOTAVAIL (errno 49).

Fix: force GC.Collect()/WaitForPendingFinalizers/GC.Collect() in
ScriptRunner.Execute() after all assembly loading but before script
invocation, draining stale SafeHandle finalizers before user code creates
any socket fds.

Also: do not pass AssemblyLoadContext to PublishCommandOptions in
ExecuteScriptCommand.GetLibrary() — the compile step needs only file
metadata; the correct ALC is applied at execution time. Passing it
forced all refs to MetadataReference.CreateFromFile() unnecessarily.

CommandRunner: switch from BeginOutputReadLine/BeginErrorReadLine to
ReadToEndAsync so Process.Dispose() correctly closes stdout/stderr fds
(async-read mode deliberately skips those in Dispose).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…cold cache

The previous fix placed GC.Collect() before script invocation, but this only
covered assemblies already loaded during setup (script.dll itself). Runtime
dependency assemblies are loaded lazily via the Resolving event handler as
user code accesses types — AFTER the GC sweep. On a cold cache (no net10.0
project folder), all runtime deps are loaded from the NuGet global cache,
creating many SafeFileHandle objects whose fd numbers are freed but not
GC.SuppressFinalize'd. If the script creates child processes (e.g. CliWrap
socketpairs) that recycle those fd numbers, the eventual finalizer closes the
live socket → EADDRNOTAVAIL.

Fix: iterate runtimeDepsMap and call assemblyLoadPal.LoadFrom() for every
runtime assembly before the GC sweep. This ensures all LoadFromAssemblyPath()
calls — and the SafeFileHandles they create — happen before the sweep, so
the GC cycle drains every pending finalizer before any user code runs.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…Unix

On Unix (macOS and Linux), anonymous pipes are backed by AF_UNIX socketpairs.
The .NET runtime's async socket I/O engine (SocketAsyncEngine) registers these
in a static epoll/kqueue-based dispatcher. Even after Process.Dispose(), the
engine holds socket references in its static state — no GC strategy (blocking
collect, NoInlining, Thread.Join, Sleep) can break those references. If a
script child process (e.g. CliWrap) later creates socketpairs that recycle the
same fd numbers, the stale engine state corrupts the new socket →
EADDRNOTAVAIL (errno 49) on full cold cache (first run after clearing the
dotnet-script cache folder).

Fix: skip stdout/stderr redirection in CommandRunner on non-Windows platforms
(RuntimeInformation.IsOSPlatform(OSPlatform.Windows)). Subprocess output goes
directly to the terminal on Unix, which is acceptable UX. On Windows,
anonymous pipes use CreatePipe (not sockets) so redirection is preserved.

Also improves EnsureSuccessfulExitCode() to emit a helpful message when stderr
was not captured, and updates the comment in ScriptRunner.Execute() to
reference the CommandRunner fix for the subprocess I/O handle path vs the
assembly-loading SafeFileHandle path.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
…estore output

On Unix, WaitForExit() internally drains redirected pipe streams asynchronously
(.NET 6+), registering AF_UNIX socketpair fds in SocketAsyncEngine's static
epoll/kqueue state even when callers use synchronous reads. After Process.Dispose()
the OS recycles those fd numbers; if a script child process (e.g. CliWrap) reuses
the same fds, the stale engine state corrupts the new socket → EADDRNOTAVAIL (49).

CommandRunner: disable stdout/stderr redirection on Unix to avoid creating the
socketpairs. DotnetRestorer: add -v q (quiet) to dotnet restore so it produces no
stdout on success, keeping the parent process's stdout clean (required for tests
that assert an empty stdout after running scripts with --no-cache).

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
On Unix, CommandRunner.Capture() runs without I/O redirection to prevent
EADDRNOTAVAIL from AF_UNIX socketpairs leaking into SocketAsyncEngine's static
state. This meant error output (e.g. NU1101 from a failed dotnet restore) was
never captured, breaking ShouldThrowExceptionOnRestoreError.

When the first (no-redirect) run exits with a non-zero code, Capture() now retries
with full redirection to collect the error details. Creating socketpairs in the
retry is safe: the command has already failed, so no user script or CliWrap will
run afterward — EADDRNOTAVAIL cannot occur on the error path.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
The retry-on-failure approach still let dotnet restore's error output bleed to the
parent process's stdout on the first (no-redirect) run, breaking
ShouldThrowExceptionWhenReferencingUnknownPackage which checks that the process
output starts with the exception message rather than a NuGet error path.

Replace with CaptureViaShell(): write a tiny #!/bin/sh script that redirects the
command's stdout and stderr to temp files, then exec it with no pipe redirection.
The shell forks+execs the target command with its I/O pointing at regular files —
no AF_UNIX socketpairs are created in the dotnet-script process, so SocketAsyncEngine
acquires no stale state. Output is read from the temp files after exit. Works for
both success and failure paths, and nothing bleeds to the parent's stdout/stderr.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@seesharper
seesharper merged commit ea2da11 into master May 28, 2026
6 checks passed
@filipw
filipw deleted the fix/commandrunner-process-leak branch May 28, 2026 12:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot assign requested address

2 participants