Repository navigation
[FEA]: Update the memory module to use __dealloc__. #1063
Description
Is this a duplicate?
- I confirmed there appear to be no duplicate issues for this request and that I agree to the Code of Conduct
Area
cuda.core
Is your feature request related to a problem? Please describe.
I would like abnormal process shutdown to complete gracefully. When testing memory IPC, it was observed that a killed child process sometimes appears to invoke Buffer.__del__ after the CUDA context has been destroyed (see additional context section for error message). This has been observed to result in hard errors including floating point exceptions and segmentation violations.
Describe the solution you'd like
Update Buffer.__del__ (and similar methods) to use Buffer.__dealloc__.
Describe alternatives you've considered
Solutions based on sys.is_finalizing appear to be unreliable.
Additional context
The following messages were output from a killed child process while running cuda_core/tests/memory_ipc tests:
/opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/_pytest/unraisableexception.py:67: PytestUnraisableExceptionWarning: Exception ignored in: 'cuda.core.experimental._memory.Buffer.__del__'
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.12.11/x64/lib/python3.12/site-packages/cuda/core/experimental/_device.py", line 991, in __new__
devices = _tls.devices
^^^^^^^^^^^^
AttributeError: '_thread._local' object has no attribute 'devices'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "cuda/core/experimental/_memory.pyx", line 82, in cuda.core.experimental._memory.Buffer._shutdown_safe_close
self._mr.deallocate(self._ptr, self._size, stream)
File "cuda/core/experimental/_memory.pyx", line 801, in cuda.core.experimental._memory.DeviceMemoryResource.deallocate
raise_if_driver_error(err)
File "cuda/core/experimental/_utils/cuda_utils.pyx", line 69, in cuda.core.experimental._utils.cuda_utils._check_driver_error
raise CUDAError(f"{name}: {expl}")
cuda.core.experimental._utils.cuda_utils.CUDAError: CUDA_ERROR_INVALID_CONTEXT: This most frequently indicates that there is no context bound to the current thread. This can also be returned if the context passed to an API call is not a valid handle (such as a context that has had ::cuCtxDestroy() invoked on it). This can also be returned if a user mixes different API versions (i.e. 3010 context with 3020 API calls). See ::cuCtxGetApiVersion() for more details. This can also be returned if the green context passed to an API call was not converted to a ::CUcontext using ::cuCtxFromGreenCtx API.
Activity
leofang commentedon Oct 1, 2025
Blocked by #1041.
cpcloud commentedon Oct 3, 2025
How do we know that implementing
__dealloc__fixes the problem?Since
__dealloc__(corresponding to the C API'stp_deallocslot) is always called after__del__(if it's defined), it would continue to be invoked after the CUDA context has been destroyed.So, how does
__dealloc__address the problem?cpcloud commentedon Oct 3, 2025
Additionally, the Cython documentation for
__dealloc__explicitly states:(The bolding is mine)
leofang commentedon Oct 4, 2025
I can't find any up-to-date discussion on this, so let me summarize my understanding here for future reference. If we can prove or disapprove it one day, either way it'd be great.
My observation has been that if C functions are
cimport'd and in__dealloc__we only call those C functions to do cleanup, we do not run into any attribute errors at shutdown, because all function access can be done through pointer chasing without calling into Python states. Furthermore, if the same implementation is done with only__dealloc__replaced by__del__, I've see it failed before. Ideally this should not matter and the same implementation should work for both situations, but it does not seem to be the case. Finally, I did see__dealloc__raising attribute not found errors if Python access was done in its function body, so to be precise using__dealloc__is not a silver bullet, we also have to avoid Python access.Why it behaves like this has been unclear to me for years, just like it's been a mystery why PEP 442 did not fix this problem once and for all back in Python 3.4. I simply stopped asking questions related to this, and just stick to the good practices that are known to work 🥲
rwgk commentedon Oct 5, 2025
This is far over my head TBH, but maybe this is useful?
https://chatgpt.com/share/68e2a72e-a090-8008-91eb-8c265b8d7cf8
(It automatically "Thought for 59s")