cuda.cudart.getLocalRuntimeVersion() raises RuntimeError: Failed to dlopen libcudart.so.12

Is this a bug? getLocalRuntimeVersion() fails for me in cuda 11.8 environment. I'm asking because I see that the API call is in the cuda-python 11.8 release notes.

In the source code, we're hard coding libcudart.so.12. Is that right?

Repro

In [1]: from cuda import cudart

In [2]: cudart.getLocalRuntimeVersion()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[2], line 1
----> 1 cudart.getLocalRuntimeVersion()

File ~/.conda/envs/rapids/lib/python3.11/site-packages/cuda/cudart.pyx:24961, in cuda.cudart.getLocalRuntimeVersion()

File ~/.conda/envs/rapids/lib/python3.11/site-packages/cuda/ccudart.pyx:2365, in cuda.ccudart.getLocalRuntimeVersion()

File ~/.conda/envs/rapids/lib/python3.11/site-packages/cuda/_lib/ccudart/ccudart.pyx:2121, in cuda._lib.ccudart.ccudart._getLocalRuntimeVersion()

RuntimeError: Failed to dlopen libcudart.so.12

xref rmm/1675

@Matt711

It seems to be a backport mistake that we should fix:

cuda-python/cuda/_lib/ccudart/ccudart.pyx.in

Lines 2451 to 2457 in 64cc9ae

    
           # Load 
        
           handle = dlfcn.dlopen('libcudart.so.12', dlfcn.RTLD_NOW) 
        
           if handle == NULL: 
        
               with gil: 
        
                   raise RuntimeError(f'Failed to dlopen libcudart.so.12') 
        
           __cudaRuntimeGetVersion = dlfcn.dlsym(handle, 'cudaRuntimeGetVersion')

@Matt711 how urgent is this?

@leofang

@Matt711 how urgent is this?

Not urgent. We already have a workaround using numba.cuda. I also don't mind working on this @leofang, if you could point me in the right direction.

@Matt711

Thanks, @Matt711. The offending code that I linked to above is from the 11.8.x branch, so ideally we can just fix the lines referencing libcudart.so.12 to .11. But we're transitioning to a new development/release process so let me check with @vzhurba01 later today first, and get back to you later.

@Matt711

@Matt711 we discussed and will try to get a new 11.8.x release out next week, with this bug fixed and perhaps also #75 backported.

@leofang

Thanks @leofang

@leofang

@leofang, @vzhurba01 did this backport/release occur?

Not yet. The wheels and conda packages are currently going through pre-release validation. I'll update this issue once posting is complete.

FYI I've updated the repo with the fix under the patch release 11.8.4 (tag v11.8.4).

I created new issue #139 to track the wheels/conda uploads for this patch release. I'm thinking of keeping this current issue open though until they are uploaded, and then give a notice before closing.

https://pypi.org/project/cuda-python/11.8.4/
https://anaconda.org/nvidia/cuda-python/files?version=11.8.4

Issue #139 is now complete as both PYPI and Conda (nvidia channel) are now updated with the 11.8.4 patch. Thank you all for your patience.

Fixes from windows testing (NVIDIA#89)

Update `cuda.bindings` to 13.0.0 (#792)

Navigation Menu

Repository navigation

`cuda.cudart.getLocalRuntimeVersion()` raises `RuntimeError: Failed to dlopen libcudart.so.12` #89

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	# Load
	handle = dlfcn.dlopen('libcudart.so.12', dlfcn.RTLD_NOW)
	if handle == NULL:
	with gil:
	raise RuntimeError(f'Failed to dlopen libcudart.so.12')

	__cudaRuntimeGetVersion = dlfcn.dlsym(handle, 'cudaRuntimeGetVersion')

cuda.cudart.getLocalRuntimeVersion() raises RuntimeError: Failed to dlopen libcudart.so.12 #89

Description

Activity

Matt711 commented on Sep 10, 2024

leofang commented on Sep 10, 2024

Matt711 commented on Sep 10, 2024

leofang commented on Sep 10, 2024

leofang commented on Sep 10, 2024

Matt711 commented on Sep 11, 2024

wence- commented on Sep 30, 2024

vzhurba01 commented on Sep 30, 2024

vzhurba01 commented on Oct 7, 2024

vzhurba01 commented on Nov 4, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`cuda.cudart.getLocalRuntimeVersion()` raises `RuntimeError: Failed to dlopen libcudart.so.12` #89