Repository navigation
Avoid unnecessarily loading and checking driver version #962
Description
+ python -c 'import cuda.cccl.parallel'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/cuda/cccl/parallel/__init__.py", line 5, in <module>
from . import experimental
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/cuda/cccl/parallel/experimental/__init__.py", line 5, in <module>
from .algorithms import (
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/cuda/cccl/parallel/experimental/algorithms/__init__.py", line 6, in <module>
from ._merge_sort import merge_sort as merge_sort
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/cuda/cccl/parallel/experimental/algorithms/_merge_sort.py", line 11, in <module>
from .. import _cccl_interop as cccl
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/cuda/cccl/parallel/experimental/_cccl_interop.py", line 17, in <module>
from numba import cuda, types
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/numba_cuda/numba/cuda/__init__.py", line 73, in <module>
from .device_init import *
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/numba_cuda/numba/cuda/device_init.py", line 71, in <module>
from .intrinsic_wrapper import (
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/numba_cuda/numba/cuda/intrinsic_wrapper.py", line 8, in <module>
@jit(device=True)
^^^^^^^^^^^^^^^^
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/numba_cuda/numba/cuda/decorators.py", line 151, in jit
lto = _have_nvjitlink() and not debug
^^^^^^^^^^^^^^^^^
File "/home/conda/staged-recipes/build_artifacts/cccl-python_1757530738279/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib/python3.11/site-packages/numba_cuda/numba/cuda/cudadrv/driver.py", line 137, in _have_nvjitlink
nvjitlink_internal._inspect_function_pointer("__nvJitLinkVersion")
File "cuda/bindings/_internal/nvjitlink.pyx", line 247, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
File "cuda/bindings/_internal/nvjitlink.pyx", line 250, in cuda.bindings._internal.nvjitlink._inspect_function_pointer
File "cuda/bindings/_internal/nvjitlink.pyx", line 198, in cuda.bindings._internal.nvjitlink._inspect_function_pointers
File "cuda/bindings/_internal/nvjitlink.pyx", line 70, in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
File "cuda/bindings/_internal/nvjitlink.pyx", line 83, in cuda.bindings._internal.nvjitlink._check_or_init_nvjitlink
RuntimeError: something went wrong
Ideally, there'd be a way (perhaps something analogous to strerror?) to grab the CUDA error message, if such a thing exists, and throw that into the exception.
Activity
cpcloud commentedon Sep 10, 2025
Actually, it looks like this code was very recently merged into cybind. @mdboom Can you take a look at this one?
[-]Error messages from bindings code are difficult to interpret[/-][+]Avoid unnecessarily loading and checking driver version[/+]leofang commentedon Sep 10, 2025
As discussed internally, we don't need this driver version check in all modules. The check was there before the pathfinder landed, and was meant for the pathfinder to pick up (#654). At the call site to the pathfinder we no longer need this.
mdboom commentedon Sep 15, 2025
Assigning myself on this one. To confirm I understand -- there are some modules where we need to get the CUDA version in order to load the correct library. There are others where this version check is performed (and it's been that way for some time), but the result is never used (I think these are the majority of them). We should remove the version check in these cases. Is that correct?
leofang commentedon Sep 15, 2025
Yes. We should remove the unused checks. They were used at one point to decide which DSO to load, similar to what was done in nvmath-python:
https://github.com/NVIDIA/nvmath-python/blob/1cbc60be436fe98e8a08f38e9867bc6fba90f007/nvmath/bindings/_internal/utils.pyx#L126-L179
Since the advent of the pathfinder, this is no longer the responsibility of the internal layer. The pathfinder should be responsible for checking the driver version, if needed. So in all cases we just call the pathfinder API, and now the driver check becomes obsolete.
Fix NVIDIA#962: Don't perform unnecessary version checks
Fix NVIDIA#962: Don't perform unnecessary version checks (NVIDIA#969)
Backport NVIDIA#962 and NVIDIA#969 to 12.9.x
Backport #962 and #969 to 12.9.x (#992)