Repository navigation
[RESEARCH] finding libnvvm.so #441
Description
Creating this as a place for myself to collect information related to finding libnvvm.so
Note that libnvvm.so is unusual: while most CTK .so files live under <loc>/lib or <loc>/lib64, it lives under <loc>/nvvm/lib64.
Metadata
Metadata
Assignees
Type
Projects
- Status
Activity
rwgk commentedon Feb 7, 2025
rwgk commentedon Feb 10, 2025
Inside a
cccl [Dev Container: cuda12.8-gcc13 @ ... ]rwgk commentedon Feb 10, 2025
Ubuntu 24.04workstation withCTK 12.6.1installed into/usr/localrwgk commentedon Feb 11, 2025
Using a
venv(namedscratchenvhere):rwgk commentedon Feb 11, 2025
When using conda (miniforge), this appears to always exist, after installing
cuda-nvcc:With these files:
I tried:
conda install cuda-version=12.8 cuda-nvccinto thebaseenvironment.conda create --yes -n scratchenv python=3.12 cuda-version=12.8 cuda-nvccAdditionally,
$CONDA_PREFIX/lib/python3.12/site-packagesappears to always be included insys.path.rwgk commentedon Feb 11, 2025
For the record, I also experimented with this script:
https://github.com/rwgk/stuff/blob/2398710e6be12f65c044b91c8cb105093c720f43/cuda-python/cuda_bindings_nvvm_proc_self_maps.py
Example:
Into a
scratchvenv:Example output WITHOUT setting LD_LIBRARY_PATH:
After
export LD_LIBRARY_PATH=/usr/local/cuda/nvvm/lib64the output changes:rwgk commentedon Feb 12, 2025
Possible approaches for locating libnvvm.so:
Rely on the existing
$ORIGIN/../../../nvidia/cuda_nvcc/nvvm/lib64rpathaddition to discoversite-packages/nvidia/cuda_nvcc/nvvm/lib64.Traverse
sys.path, look forlibsubdirectories, look fornvvm/lib64in their parent directories. — This will work for conda-based installations with separatevenv. — We wouldn't need the$ORIGIN/../../../../../../nvvm/lib64rpathaddition anymore, although it would be fine to keep. — Note that this is similar to looking for$CONDA_PREFIX/nvvm/lib64, but without requiring that environment variable.Look for
$CUDA_HOME/nvvm/lib64or$CUDA_PATH/nvvm/lib64as the last resort, e.g. to find/usr/local/cuda/nvvm/lib64.In theory we could look for
/usr/local/cuda/nvvm/lib64as the very last resort, but is that wise?kkraus14 commentedon Feb 12, 2025
@rwgk could probably also learn from how Numba finds
nvvmtoday as well: https://github.com/NVIDIA/numba-cuda/blob/bf487d78a40eea87f009d636882a5000a7524c95/numba_cuda/numba/cuda/cudadrv/libs.py#L54rwgk commentedon Mar 11, 2025
Some ChatGPT queries, for more datapoints:
For easy reference, copy-pasting summaries:
General:
cuda-toolkit-configif available (it's direct and robust).which nvccorCUDA_HOMEfor general compatibility.FindCUDA.Python-based:
numba.cuda.runtime.get_libdevice()– if numba is installed, this directly returns the path to libnvvm.cupy.get_cuda_path()– cleanest if you're already using CuPy.torch.utils.cpp_extension.CUDA_HOME– if PyTorch is in use.os.getenv()– safest for broad compatibility.site-packages/nvidia:
importliborpkg_resourcessys.path+globCUDA_PATHkkraus14 commentedon Mar 11, 2025
I assume this is referring to the
aptpackagecuda-toolkit-config-commonand isn't an actually usable tool as far as I'm aware. There's too many different OS package managers where interacting directly with them is probably a bad idea.CUDA_HOMEisn't an officially supported environment variable and is just something that multiple OSS projects have adopted as a "failsafe" for when they can't find things in expected locations or if a user has a non-standard install location.which nvccwill not work in situations where the user doesn't have nvcc installed, i.e. if they're using wheel or conda packages.Can't rely on having CMake installed and
FindCUDAis deprecated whereFindCUDAToolkitis the more updated version, but it still doesn't support finding libraries in wheel packages and doesn't work if annvccpackage isn't installed.This is an option, but we need to handle more than Conda. We need to handle OS package managers putting things in standard locations found by the loader / linker, as well as the situation where users have installed the CUDA toolkit into a manual location and have set things like
LD_LIBRARY_PATHandPATHappropriately.We intend to move
numba.cudato depend oncuda.corefor its internal usage of CUDA libraries, so this is a non-starter.We hope to have
cupyandpytorchadopt and depend oncuda.corein the future so these are non-starters.There isn't any officially supported environment variable to indicate the CUDA toolkit location and different package managers can have different layouts for the various libraries of the CUDA toolkit.
There isn't really a python package within the current CUDA wheels. In theory there's a
site-packages/nvidiafolder that you can import as a no-op and then you can checknvidia.__dir__()and check if there's acuda_nvccentry in the list (thecuda-nvccpackage is what contains the nvvm DSO).If a user is using wheels I don't think this helps us versus just importing the
nvidianamespace package / directory as described above.CUDA_PATHisn't an officially supported environment variable similar toCUDA_HOME.rwgk commentedon Mar 11, 2025
Thanks @kkraus14, that's very useful, for me to better understand the environment.
These are interesting to me, though, and I already looked pretty closely at the numba code, after you pointed me to it a month ago (your comment here from Feb 12, my #447 experiment). I figure, roughly, we need to provide a functional union of what
numba.cuda,cupy, andpytorchdo for discovering CUDA libraries and headers. I want to spend a small/modest amount of time to also look at the relevant code incupyandpytorch.rwgk commentedon Apr 11, 2025
Content moved here: #451 (comment)