Vendor CCCL and use it to reactivate tensormap by seberg · Pull Request #2940 · NVIDIA/cuda-python

seberg · 2026-09-23T11:58:58Z

I'll guess that this'll need some discussion. Doing the CCCL and TMA at the same time is nice for testing, but I can split it out for easier review: My understanding is that we expect more usage out of a CCCL submodule (otherwise this would be dubious to me).

Description

This now does two things. First, it vendors CCCL into cuda-core (largely agent driven). Two notes on this:

I removed CCCL_HOME env var as it seems the submodule should cover this
The sdist bloats a lot (I believe to 12.5MiB) due to this. This seemed fine since we never intend to ship it?
On a whim: added to dependabot to update monthly.

The other thing is that the TMA code now actually uses the CCCL path (enabled by having the newest version with multiple fixes). Note that even with this there are some few paths that cannot take this path due to DLPack exchange limitations, unfortunately. Which makes this all slightly unclear.

Notes on the TMA changes:

I moved everything to Cython, C++ is nice, but just adds a layer here.
I moved a lot of things to be proper cdef (changed some tests)
Since the CCCL path is now used and I moved things to Cython, the error normalization is slightly less precise. Raising a CUDAError with the CCCL string and also raising it for the bad compute capability check included there (probably also a cuda error on the driver path, though).

This seems likely not the last word here...

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

github-actions · 2026-09-23T14:28:39Z

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-2940/
https://nvidia.github.io/cuda-python/pr-preview/pr-2940/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-2940/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-2940/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

leofang · 2026-10-06T01:06:05Z

@seberg sorry for the huge conflicts and long wait, @juenglin just finished refactoring the build system significantly. Could you have your agent rebase or start over? 🙇

This now does two things. First, it vendors CCCL into cuda-core (largely agent driven). Two notes on this: * I removed CCCL_HOME env var as it seems the submodule should cover this * The `sdist` bloats a lot (12.5MiB) due to this. This seemed fine since we never intend to ship it? * On a whim: added to dependabot to update monthly. The other thing is that the TMA code now actually uses the CCCL path (enabled by having the newest version with multiple fixes). Note that even with this there are some few paths that cannot take this path due to DLPack exchange limitations, unfortunately. Which makes this all slightly unclear. Notes on the TMA changes: * I moved everything to Cython, C++ is nice, but just adds a layer here. * I moved a lot of things to be proper `cdef` (changed some tests) * Since the CCCL path is now used and I moved things to Cython, the error normalization is slightly less precise. Raising a CUDAError with the CCCL string and also raising it for the bad compute capability check included there (probably also a cuda error on the driver path, though). This seems likely not the last word here...

…der)

coderabbitai · 2026-10-13T03:17:12Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/cuda-python/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: 5e19a33d-b205-49e4-a672-6c6a71ca561f

📥 Commits

Reviewing files that changed from the base of the PR and between 31adc57 and a7aba60.

📒 Files selected for processing (1)

cuda_core/MANIFEST.in

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

📝 Summary

Summary by CodeRabbit

New Features
- Tensor-map creation supports additional data types through direct CUDA driver encoding. Tiled and im2col descriptors use tensor layout information directly.
Build & Installation
- Source builds require the pinned CCCL headers. Clone instructions include the required submodule, with guidance for existing checkouts.
- Source distributions include headers and license files needed to build cuda.core.
Bug Fixes
- Tensor-map validation now reports errors for unsupported data types, mismatched devices, and invalid layout, box-size, or stride parameters.

The pull request adds a pinned CCCL submodule and uses its headers in cuda.core builds. Tensor-map construction now reads typed memory-view layouts and uses updated descriptor encoding paths. Build workflows, packaging, setup guidance, and tests are updated for these changes.

Changes: CCCL-backed Tensor Map Build

Layer / File(s)	Summary
Pin CCCL and initialize submodules `.gitmodules`, `cuda_core/third_party/cccl`, `.github/dependabot.yml`, `.github/workflows/*`	The repository pins CCCL, schedules monthly submodule updates, and initializes submodules in build, coverage, and source-distribution workflows.
Use CCCL headers in source builds `cuda_core/build_hooks.py`, `cuda_core/MANIFEST.in`, `cuda_core/NOTICE`, `CONTRIBUTING.md`, `cuda_core/AGENTS.md`, `cuda_core/docs/source/install.rst`, `cuda_core/tests/test_build_hooks.py`, `.pre-commit-config.yaml`, `.spdx-ignore`, `ruff.toml`	Build hooks use the pinned `libcudacxx/include` directory and raise an error when it is absent. Packaging, setup guidance, notices, exclusions, and build-hook tests are updated for the submodule.
Update tensor-map construction `cuda_core/cuda/core/_tensor_map.pyx`, `cuda_core/cuda/core/_tensor_map.pyi`, `cuda_core/cuda/core/_dlpack.pxd`, `cuda_core/cuda/core/_cpp/tensor_map.cpp`, `cuda_core/cuda/core/_cpp/tensor_map_cccl.h`	Tensor-map constructors use typed memory views and layout-backed shape and stride data. Tiled descriptors use `make_tma_descriptor` for DLPack-compatible dtypes and the direct CUDA driver encoder for other dtypes. The previous CCCL C wrapper and its declarations are removed.
Validate tensor-map behavior `cuda_core/tests/test_tensor_map.py`	Tests add coverage for box-size, element-stride, dtype, and device validation. Tests for removed internal helpers are deleted.

Suggested reviewers: juenglin

Priority: ⬇️ Low

Change: Feature

Merge Risk: ⚪ Minimal · up to a7aba

This manifest change limits the source distribution to the needed headers and licenses. No merge-blocking risk is identified in the reviewed change.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

leofang

Thanks, @seberg! LGTM. I asked my bot to give it a pass. It left a bunch of nitpicks that are safe to ignore. Up to you.

leofang · 2026-10-13T03:26:41Z

+            except RuntimeError as err:
+                # RuntimeErrors here should in practice be CUDAErrors.
+                # (either driver error or compute capability check).
+                raise CUDAError(str(err)) from err


You call this out in the PR body, but worth capturing a decision inline: everything CCCL throws as a std::runtime_error — the driver-return path and the compute-capability guard — collapses to CUDAError here. CUDAError is defined elsewhere as "a CUDA driver call returned non-success," so a host-side CC rejection carrying that type is a semantic regression vs. the old C++ shim, and users keying off exception type to distinguish "driver said no" from "my GPU is too old" lose that signal.

One option: catch the two upstream shapes separately — let CC check surface as RuntimeError (or a dedicated UnsupportedComputeCapabilityError) and keep CUDAError for genuine driver returns. If upstream makes that hard to tell apart from the exception alone, a cheap pre-check against view.__dlpack_device__() → device CC before invoking CCCL would also work.

leofang · 2026-10-13T03:26:54Z

+    def test_swizzle_via_options(self, dev, skip_if_no_tma):
+        # SWIZZLE_128B caps the innermost box at 128 bytes; 64 float32 is 256.
+        buf = dev.allocate(1024 * 4, stream=dev.default_stream)
+        with pytest.raises(ValueError, match="bytes128"):


match="bytes128" here, and match="multiple of 16" further down at line 320, are both matching substrings of CCCL's own _CCCL_THROW(std::invalid_argument, …) message strings (see cuda_core/third_party/cccl/libcudacxx/include/cuda/__tma/make_tma_descriptor.h). With dependabot bumping CCCL monthly, any upstream wording tweak breaks these without a real behavior change.

Either match on the exception type alone and assert the behavioral invariant another way, or pick a tighter semantic token the upstream maintainers are likely to keep stable (e.g. the enum spelling tma_swizzle::bytes128, or box_sizes).

right... I think I like matching something and this seems pretty sensible for "probably stable" already.

As to the CUDAError, yes we could try to narrow it, but part of why to use CUDAError is that it is what the other branch currently will give.

leofang · 2026-10-13T03:27:02Z

+[submodule "cuda_core/third_party/cccl"]
+	path = cuda_core/third_party/cccl
+	url = https://github.com/NVIDIA/cccl.git
+	branch = main


branch = main + the dependabot entry means we're floating on CCCL's dev tip. The current pin (ba8eb88f) is 544 commits past v3.6.0 because the TMA fixes you need aren't in a tagged release yet — understandable — but once they are, I'd advocate switching to the release tag and treating the dependabot PRs as "take the latest tag" rather than "take latest main". Either way, worth adding a one-line comment here capturing the policy so the next maintainer isn't guessing.

Hmmm, yeah, I guess we may want to change this to not track main but rather the latest release tag eventually.
(ignored the "add comment", though)

leofang · 2026-10-13T03:27:11Z

+# setuptools-scm only records the submodule gitlink, so the checkout must be
+# grafted or wheel-from-sdist has no CCCL headers. Exclude .git so the sdist
+# does not ship a pointer into this repository.
+graft third_party/cccl


graft third_party/cccl ships the entire CCCL repo (~47 MiB on disk; cub, thrust, cudax, docs, python, c, ci) when _get_cccl_include_dirs() only consumes libcudacxx/include (~16 MiB on disk). Narrowing to

graft third_party/cccl/libcudacxx/include include third_party/cccl/LICENSE include third_party/cccl/libcudacxx/LICENSE.TXT

should shrink the sdist by roughly 2/3 without changing the build. You acknowledged the 12.5 MiB bloat in the PR body — since no sdist is published to PyPI today, this is CI-internal weight only, but trimming it also removes the question of whether we're redistributing cub/thrust under their own license terms. Fine to defer; worth deciding one way or the other before merge.

seberg self-assigned this Sep 23, 2026

github-actions Bot added CI/CD CI/CD infrastructure cuda.core Everything related to the cuda.core module labels Sep 23, 2026

seberg mentioned this pull request Sep 23, 2026

Make sure CCCL TMA is compiled and use it (very narrowly) #2428

Closed

seberg added the P1 Medium priority - Should do label Sep 23, 2026

seberg added this to the cuda.core next milestone Sep 23, 2026

seberg added 4 commits October 6, 2026 13:55

Cast enum values directly and outside nogil block (and test option or…

8fe505d

…der)

Keep build macro tests independent of the CCCL checkout

2c59eba

Exclude the CCCL git pointer from source distributions

a75c3b6

seberg force-pushed the tensor-map-submodule-cccl branch from 5503921 to a75c3b6 Compare October 6, 2026 12:09

leofang self-requested a review October 7, 2026 15:23

leofang modified the milestones: cuda.core next, cuda.core 1.3.0 Oct 7, 2026

Merge branch 'main' into tensor-map-submodule-cccl

31adc57

leofang approved these changes Oct 13, 2026

View reviewed changes

Only graft CCCL headers and license, not full repo

a7aba60

leofang merged commit 30754f8 into NVIDIA:main Oct 13, 2026
111 checks passed

seberg deleted the tensor-map-submodule-cccl branch October 13, 2026 15:55

Andy-Jost mentioned this pull request Oct 13, 2026

cuda.core: import fails with undefined symbol cudaGetErrorString when _tensor_map is built with -z now (after #2940) #3098

Open

rwgk mentioned this pull request Oct 13, 2026

ci: initialize CCCL for rendered Lychee checks #3099

Draft

2 tasks done

Navigation Menu

Repository navigation

Vendor CCCL and use it to reactivate tensormap - #2940

Vendor CCCL and use it to reactivate tensormap#2940
leofang merged 6 commits into
NVIDIA:mainfrom
seberg:tensor-map-submodule-cccl

seberg commented Sep 23, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 23, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

leofang commented Oct 6, 2026

Uh oh!

coderabbitai Bot commented Oct 13, 2026 •

edited

Loading

Summary by CodeRabbit

Uh oh!

leofang left a comment

Uh oh!

leofang Oct 13, 2026

Uh oh!

leofang Oct 13, 2026

Uh oh!

seberg Oct 13, 2026

Uh oh!

leofang Oct 13, 2026

Uh oh!

seberg Oct 13, 2026

Uh oh!

leofang Oct 13, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

seberg commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

github-actions Bot commented Sep 23, 2026

Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

leofang commented Oct 6, 2026

Uh oh!

coderabbitai Bot commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

leofang left a comment

Choose a reason for hiding this comment

Uh oh!

leofang Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

leofang Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

seberg Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

leofang Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

seberg Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

leofang Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

seberg commented Sep 23, 2026 •

edited

Loading

coderabbitai Bot commented Oct 13, 2026 •

edited

Loading