Repository navigation
[BUG]: cuda-core 1.2.0 breaks Buffer use for non-CUDA memory implementations #2769
Description
Is this a duplicate?
- I confirmed there appear to be no duplicate issues for this bug and that I agree to the Code of Conduct
Type of Bug
Runtime Error
Component
cuda.core
Describe the bug
nvmath-python provides NVPL-backed host APIs; this means that we run in environments where the CUDA driver is not present or the context is not initialized.
I have implemented a MemoryResource sub-class that uses Numpy as the allocator. This is because I want to use the same interfaces for both CUDA and Non-CUDA when performing internal memory allocations.
Starting in cuda-core 1.2.0, it is now not possible to inherit from MemoryResource/Buffer without running cuInit? For example, I get the following error when running tests that do not use CUDA.
nvmath/_internal/workspace.py:519: in allocate
return Buffer.from_handle(ptr=ptr, size=size, mr=self)
cuda/core/_memory/_buffer.pyx:379: in cuda.core._memory._buffer.Buffer.from_handle
???
cuda/core/_memory/_buffer.pyx:310: in cuda.core._memory._buffer.Buffer._init
???
cuda/core/_utils/cuda_utils.pxd:23: in cuda.core._utils.cuda_utils.HANDLE_RETURN
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E cuda.core._utils.cuda_utils.CUDAError: CUDA_ERROR_NOT_INITIALIZED: This indicates that the CUDA driver has not been initialized with cuInit() or that initialization has failed.
How to Reproduce
import numpy as np
from cuda.core import MemoryResource, Buffer, Stream
class NumpyMemoryResource(MemoryResource):
"""Host :class:`cuda.core.MemoryResource` backed by :func:`numpy.empty`.
Allocates plain (non-pinned) host memory via NumPy. Unlike
:class:`PinnedMemoryResource` / :class:`LegacyPinnedMemoryResource`, this
resource does not call into the CUDA driver, so it is safe to use in
environments where no CUDA driver/device is available (CPU-only test runs,
pure-host scratch space for parameter struct staging, etc.).
Pair with ``Workspace(NumpyMemoryResource(), logger)`` and pass
``stream_holder=None`` to ``allocate_perhaps`` for a fully CUDA-free path
(``device_id`` is read from the resource and is ``-1``).
Not suitable as the host half of a :class:`MirroredWorkspace`: ``cudaMemcpyAsync``
requires page-locked memory for asynchronous correctness.
"""
def __init__(self) -> None:
# Strong refs keyed by ptr — Buffer carries only the int pointer.
self._held: dict[int, np.ndarray] = {}
def allocate(self, size: int, *, stream: Stream | None = None) -> Buffer:
arr = np.empty(size, dtype=np.uint8)
ptr = int(arr.ctypes.data)
self._held[ptr] = arr
return Buffer.from_handle(ptr=ptr, size=size, mr=self)
def deallocate(self, ptr: int, size: int, stream: Stream | None = None) -> None:
# `stream` must NOT be keyword-only: cuda.core < 1 calls deallocate with all
# arguments positionally (Buffer.close -> mr.deallocate(ptr, size, stream)).
self._held.pop(int(ptr), None)
@property
def is_host_accessible(self) -> bool:
return True
@property
def is_device_accessible(self) -> bool:
return False
@property
def is_managed(self) -> bool:
return False
@property
def device_id(self) -> int:
return -1
if __name__ == "__main__":
mr = NumpyMemoryResource()
b = mr.allocate(64, stream=None)Expected behavior
No runtime error from Buffer constructor if the CUDA runtime hasn't been initialized.
Operating System
WSL2 Ubuntu 24.04
nvidia-smi output
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.79 Driver Version: 596.59 CUDA Version: 13.2 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA RTX 1000 Ada Gene... On | 00000000:01:00.0 Off | N/A |
| N/A 52C P3 11W / 44W | 0MiB / 6141MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
Activity
Andy-Jost commentedon Sep 4, 2026
Thanks for the detailed report and the reproducer.
Root cause
Since 1.2.0,
cuda.corerecords a deallocation stream when aBufferis created withmr=(#2526). When nostream=is given, the default stream token is used. That token carries no context, soBuffer.from_handlelooks up the current context withcuCtxGetCurrentand pins it. This lets the later free run in the allocation context even when the buffer is released from another thread or after the current context changed (#2497).Two things go wrong in your scenario:
Buffer, including buffers from a memory resource that never touches CUDA. Memory that the device cannot access is not a CUDA allocation, and freeing it needs no context, so the lookup is unnecessary there.from_handletranslates into the descriptiveRuntimeError("no CUDA context is current ..."), so the rawCUDAErrorescapes instead.The documented remedies in 1.2.0 (
Device.set_current()or a non-defaultstream=) both require an initialized driver, so a CUDA-free process has no way to create an owningBuffertoday. That is a regression from 1.1.x.Fix
When
mr.is_device_accessibleisFalse,Buffer.from_handlerecords no deallocation stream and does not call the driver.mr.deallocate()still receives the default stream, as in 1.1. Device-accessible resources are unchanged. The change is in #2773, with tests that run a NumPy-backed resource like yours in a process that never initializes CUDA.Workaround for 1.2.0
Pass
owner=arrinstead ofmr=self; the array's refcount frees the memory.buf.handleandbuf.sizework, butis_host_accessible,is_device_accessible, anddevice_idquery the driver when no memory resource is attached, so avoid them in CUDA-free code. Alternatively, pincuda-core<1.2.cuda.core: move the NVIDIA#2769 release note into the 1.3.0 notes
cuda.core: host-only memory resources need no CUDA context
cuda.core: host-only memory resources need no CUDA context (NVIDIA#2773)
Prepare cuda.core v1.2.1 with Windows on ARM support (#2935)