Repository navigation
find_nvidia_binary_utility() violates deterministic-search contract on Windows (CWD leakage via shutil.which) #2119
Description
Summary
cuda.pathfinder.find_nvidia_binary_utility() currently delegates to
shutil.which(name, path=trusted_dirs) after assembling a list of trusted
search directories (NVIDIA wheel bin/, CONDA_PREFIX, CUDA_PATH /
CUDA_HOME, etc.).
On Windows, CPython's shutil.which() may prepend the current working
directory to executable resolution even when the caller supplies an
explicit path=. As a result, a binary located in the process CWD can be
returned in preference to the trusted CUDA / Conda / wheel binary that
pathfinder is supposed to discover.
This violates the pathfinder contract — callers expect a deterministic
lookup against a documented, bounded set of trusted roots, not a
shell-style lookup that depends on ambient process state.
Reproduction (conceptual)
- On Windows, install
nvccvia the NVIDIA wheel (so it lives under a
site-packages/.../bindirectory pathfinder considers trusted). - Set CWD to a directory containing a different
nvcc.exe(e.g.
%CONDA_PREFIX%\Library\bin, or any attacker-writable location). - Call
cuda.pathfinder.find_nvidia_binary_utility("nvcc"). - Expected: the wheel-resolved
nvcc.exe(deterministic, contract-
honoring).
Actual: the CWDnvcc.exeis returned becauseshutil.which
prepended CWD ahead of the suppliedpath=.
Leo's framing (#2111 comment):
Assuming a pathological case where my CWD is
%CONDA_PREFIX%\Library\bin
[...] Following the pathfinder contract I would expect to get the latter
NVCC. But becauseshutil.whichprepends CWD (TIL!), I would actually
get the former.
Impact
- Contract correctness: the public API silently returns a binary that
was never in the documented trust set. - Security-adjacent: a planted
nvcc.exe(or other supported utility)
in an attacker-controlled CWD is executed downstream by callers that
invoke the returned path. Originally flagged by an internal security
scan. - Cross-platform inconsistency: Linux/macOS do not exhibit the CWD
prepend, so behavior diverges by platform in a way users cannot predict.
Root cause
shutil.which() is a "what would the shell run?" primitive, not a "search
exactly these directories" primitive. It carries ambient OS/Python policy:
- Windows: CWD prepended in certain Python versions (observed on 3.12).
PATHEXTexpansion is implicit and version-dependent.- Empty/relative entries in
path=become CWD-sensitive. - Return value is not guaranteed to be an absolute, normalized path.
None of these are appropriate for a deterministic discovery API.
Proposed direction
Replace the shutil.which delegation with an explicit, bounded directory
walk (see PR #2111 for a first cut). Key properties:
- Search only directories pathfinder constructs:
- NVIDIA Python wheel
bin/undersite-packages CONDA_PREFIXbin layout- Explicit
CUDA_PATH/CUDA_HOME - CTK root resolved via the existing canary mechanism, then CTK
bin
- NVIDIA Python wheel
- Preserve Windows extension normalization (
.exe,PATHEXThandling
done explicitly, not inherited from the environment). - Preserve Unix executable-bit checks.
- Return absolute, normalized paths.
- Do not consult ambient
PATHor CWD by default. - Document the search order and the non-inclusion of
PATH/CWD.
Open design question — ambient PATH
Keith raised (#2111 comment) that the official Linux CUDA install guide
instructs users to add the CTK bin/ to PATH:
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#environment-setup
Users who follow that guide — setting LD_LIBRARY_PATH for libraries and
PATH for executables, without setting CUDA_PATH/CUDA_HOME — would
need to set an explicit env var for pathfinder to discover their CTK
binaries once shutil.which is removed.
Options to weigh:
- No
PATHfallback (PR [codex] Avoid untrusted binary lookup paths #2111 as written). Maximally deterministic;
requires affected users to setCUDA_PATH/CUDA_HOME. Cleanest
contract. PATHfallback on Linux/macOS only. Sidesteps the Windows CWD
issue (the actual bug) while preserving the documented install path
for the platform where the docs apply. Asymmetric but pragmatic.- Opt-in
PATHsearch via parameter or dedicated env var (e.g.
CUDA_PATHFINDER_ALLOW_PATH=1). Explicit, non-default, auditable.
Leo's call requested before landing the fix.
Regression coverage
A test that plants a Windows CWD binary and asserts it is not returned
is required (already drafted in PR #2111).
References
- PR: [codex] Avoid untrusted binary lookup paths #2111
- Affected file:
cuda_pathfinder/cuda/pathfinder/_binaries/find_nvidia_binary_utility.py - Linux install guide: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/#environment-setup
Activity
feat(pathfinder): add CTK-root canary fallback for NVIDIA binaries (#…
feat(pathfinder): add CTK-root canary fallback for NVIDIA binaries (N…
feat(pathfinder): add CTK-root canary fallback for NVIDIA binaries (N…