Fix device attribute handling

@leofang

I think this was an oversight when the new attributes were added. We should not always return 0 because it'd shadow the actual issue (driver version insufficient) and our standard treatment is to add version checks in both the codebase and the corresponding test suite to ensure cuda-core runs with any CUDA driver 12.x and 13.x. This needs to be revisited before the next release is out.

Originally posted by @leofang in #1409 (comment)

Do we really want to issue an error when a user asks about feature enablement?

To take an example, CU_DEVICE_ATTRIBUTE_HOST_NUMA_MEMORY_POOLS_SUPPORTED is a CUDA 13 feature. Say a user application supports CUDA 12 or 13 and contains feature testing based on the attribute host_numa_memory_pools_supported.

Don't we want this to return False in a CUDA 12 environment? Otherwise, the user needs to add additional logic or fencing to first say "can I ask about host NUMA memory pool support" and then separately "are host NUMA memory pools supported?" That would complicate the application logic, and needlessly so it seems to me.

Wouldn't it be more convenient to always report that future features are unsupported?

Wouldn't it be more convenient to always report that future features are unsupported?

Yes, but the current implementation doesn't discriminate cleanly between "there is an issue" and "everything is OK but it's just a future feature".

Although I'm not sure how exactly this shakes out in practice:

How likely is it that CUDA_ERROR_INVALID_VALUE is something other than "future feature"?

Another thought: honestly reporting "we don't know" is generally better in my experience. Basically that's a variation of the zen item: In the face of ambiguity, refuse the temptation to guess.

Two alternative approaches to that:

Return None instead of 0
Raise AttributeError

I'd probably go for the AttributeError.

How likely is it that CUDA_ERROR_INVALID_VALUE is something other than "future feature"?

I think there is zero chance of this. cuDeviceGetAttribute relies on an enum. Over time, the driver team extends that enum and we wrap the new values. The only way to get CUDA_ERROR_INVALID_VALUE through one of our wrappers is by calling into an old driver.

Another thought: honestly reporting "we don't know" is generally better in my experience

For the attributes ending with "supported" I don't think we encounter this situation. We know for certain that the application is asking about a newer feature that the driver does not support.

The attribute enum values contain only two types:

Enablement queries (end with "supported").
Value queries.

Most of the values are guarded by an enablement test. There are a few that could cause trouble by combining a value with an enablement test, such as:

CU_DEVICE_ATTRIBUTE_HOST_NUMA_ID = 134
NUMA ID of the host node closest to the device. Returns -1 when system does not support NUMA.

I think there is zero chance of this.

Excellent.

What do you think about this kind of approach, to make the substitutions explicit and customizable?

Look for subst_val:

    cdef inline _get_attribute(self, cydriver.CUdevice_attribute attr, subst_val):
        """Retrieve the attribute value directly from the driver."""
        cdef int val
        cdef cydriver.CUresult err
        with nogil:
            err = cydriver.cuDeviceGetAttribute(&val, attr, self._handle)
        if err == cydriver.CUresult.CUDA_ERROR_INVALID_VALUE and subst_val is not None:
            return subst_val
        HANDLE_RETURN(err)
        return val

    cdef _get_cached_attribute(self, attr, subst_val):
        """Retrieve the attribute value, using cache if applicable."""
        if attr not in self._cache:
            self._cache[attr] = self._get_attribute(attr, subst_val)
        return self._cache[attr]

    @property
    def max_threads_per_block(self) -> int:
        """int: Maximum number of threads per block."""
        return self._get_cached_attribute(driver.CUdevice_attribute.CU_DEVICE_ATTRIBUTE_MAX_THREADS_PER_BLOCK, 0)

I'm in favor of that. Maybe with 0/False as a default, since it will be used in almost every case.

I'm in favor of that. Maybe with 0/False as a default, since it will be used in almost every case.

I recommend not doing that: oversights will slip in too easily, and the overhead for making it explicit is just a very small number of characters. I'd even make the keyword required, so that people see e.g. subst=0 when they copy-paste a line, which will probably trigger a "what's that and is it correct?" thought, right when we want it.

I forgot to add: maybe subst=None as default is useful, combined with the idea that we don't need to specify another value for all attributes that exist already in CTK 12.0, which is maybe most?

This seems like a good place to use a default. There are 143 enum values. CUDA 11 ends at 124 to there's no need to look at anything earlier than that. From 125-143 there is exactly one value that needs a nonzero/false default, which is the aforementioned CU_DEVICE_ATTRIBUTE_HOST_NUMA_ID.

This seems like a good place to use a default. There are 143 enum values. CUDA 11 ends at 124 to there's no need to look at anything earlier than that. From 125-143 there is exactly one value that needs a nonzero/false default, which is the aforementioned CU_DEVICE_ATTRIBUTE_HOST_NUMA_ID.

Hm, up to you. My suggestions were based on the belief that being explicit will safe time (in the future), by reducing the potential for oversights/accidents, but I'm fine if you think slightly less code is better here than slightly better future proofing.

I'm looking into an implementation of this. Ran into another interesting case.

Both CU_DEVICE_ATTRIBUTE_GPU_PCI_DEVICE_ID and CU_DEVICE_ATTRIBUTE_GPU_PCI_SUBSYSTEM_ID were added in CUDA 12.8.

Someone could build cuda-core against CUDA 12.9 but use a CUDA 12+ driver prior to 12.8. I think we support that configuration. But that would mean that these device queries raise CUDA_ERROR_INVALID_VALUE.

I think the right thing here is to default to return 0 (an invalid PCI device ID) and add a note to the docstrings:

    @property
    def gpu_pci_device_id(self) -> int:
        """int: The combined 16-bit PCI device ID and 16-bit PCI vendor ID.

        Returns 0 if the driver does not support this query.
        """

This is the first place I'm aware of where the cuda-core device attribute docstrings differ from the CUDA driver manual, but it seems like the right thing to do. User code can play defense by checking for 0.

User code can play defense by checking for 0.

That'll be very tricky to discover; probably people will get there only after some sort of accident.

Cleanest I think (will probably fail noisily, which is better than failing silently):

    @property
    def gpu_pci_device_id(self) -> int | None:
        """int: The combined 16-bit PCI device ID and 16-bit PCI vendor ID.

        Returns None if the driver does not support this query.
        """

Maybe (more forgiving, but also more obvious if a human looks at the values):

    @property
    def gpu_pci_device_id(self) -> int:
        """int: The combined 16-bit PCI device ID and 16-bit PCI vendor ID.

        Returns -1 if the driver does not support this query.
        """

User code can play defense by checking for 0.

That'll be very tricky to discover; probably people will get there only after some sort of accident.

Cleanest I think (will probably fail noisily, which is better than failing silently):
    @property
    def gpu_pci_device_id(self) -> int | None:
        """int: The combined 16-bit PCI device ID and 16-bit PCI vendor ID.

        Returns None if the driver does not support this query.
        """
Maybe (more forgiving, but also more obvious if a human looks at the values):
    @property
    def gpu_pci_device_id(self) -> int:
        """int: The combined 16-bit PCI device ID and 16-bit PCI vendor ID.

        Returns -1 if the driver does not support this query.
        """

+1 The situation where you don't want or its not obvious what to allocate as errors within a range of values from the returned number range. Optional types were built specifically for this situation. If successful, you get the full int space for values... and if its not valid you don't have a valid object to look at and potentially misinterpret.

Fix device attribute handling for older drivers (NVIDIA#1437)

Navigation Menu

Repository navigation

Fix device attribute handling #1420

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Fix device attribute handling #1420

Description

Activity

Andy-Jost commented on Jan 5, 2026

rwgk commented on Jan 5, 2026

Andy-Jost commented on Jan 5, 2026

rwgk commented on Jan 6, 2026

Andy-Jost commented on Jan 6, 2026

rwgk commented on Jan 6, 2026

rwgk commented on Jan 6, 2026

Andy-Jost commented on Jan 6, 2026

rwgk commented on Jan 6, 2026

Andy-Jost commented on Jan 7, 2026

rwgk commented on Jan 7, 2026

rparolin commented on Jan 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions