Repository navigation
fix(cuda.core): allow NUMA pools on older drivers - #2858
aryanputta wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
Thanks for the fix. The diagnosis in #2795 is right: a driver older than 12.9 cannot answer CU_DEVICE_ATTRIBUTE_HOST_NUMA_MEMORY_POOLS_SUPPORTED, and cuda.core reads the error as False.
Removing the check from allocate() goes too far, though. A 12.9+ driver can answer the query, and a False answer there is real. #2487 added the check because a pool on an unsupported device can return buffers that fail at first use (#2486).
A simpler fix is to repair the property itself. allocate() already reads it, so allocate() then needs no change, and host_numa_memory_pools_supported also stops reporting False on 12.6–12.8 drivers. For example, in _device.pyx (with cy_driver_version cimported from cuda.core._utils.version):
@property
def host_numa_memory_pools_supported(self) -> bool:
if cy_driver_version() >= (12, 9, 0):
return bool(
self._get_cached_attribute(driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_HOST_NUMA_MEMORY_POOLS_SUPPORTED)
)
# The attribute arrived in CUDA 12.9, HOST_NUMA pools in 12.6 (#2795).
return cy_driver_version() >= (12, 6, 0) and self.memory_pools_supported and not self.integratedThe integrated clause keeps today's answer on Jetson. The evidence in #2795 comes from a discrete GPU, and #2486 showed host pools on Jetson returning unusable buffers.
Please also:
- Revert the change to
allocate(). - Add a test that simulates a pre-12.9 driver.
- Say in the property's docstring that the value is inferred on drivers before 12.9.
- Rebase on main.
When the update is in, I will start the GPU CI.
|
I revised my review above: please repair the property instead of changing |
Signed-off-by: Aryan <[email protected]>
Signed-off-by: Aryan <[email protected]>
7be3e09 to
5b7cd96
Compare
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Summary
Priority: ➖ Normal Change: Bug fix · Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to Explicit NUMA pools can reach the driver on supported older versions, while ordinary host pools retain their existing check. No issue identified here requires a change before merge. ✨ Finishing Touches 💡 1
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: NVIDIA/cuda-python/.coderabbit.yaml
- Review profile: CHILL
- Plan: Enterprise
- Run ID:
6e65bbdc-a1ad-4b22-96d8-b427516ea9e2
📒 Files selected for processing (3)
cuda_core/cuda/core/_device.pyicuda_core/cuda/core/_device.pyxcuda_core/tests/test_device.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.
Signed-off-by: Aryan <[email protected]>
Fixes #2795
Summary
Allow explicit HOST_NUMA allocations to reach the CUDA driver when the newer capability attribute is unavailable, while preserving early rejection for unsupported ordinary HOST pools.
CUDA 12.6-12.8 support HOST_NUMA pools, but the capability query returns CUDA_ERROR_INVALID_VALUE. The previous check exposed that as false and rejected valid pools.
Validation
Checklist