Repository navigation
EPIC: Setup cuda-python integration tests #1955
Description
Motivation
The cuda-python monorepo contains four published packages with an explicit dependency graph:
cuda_pathfinder → (no internal deps)
cuda_bindings → cuda_pathfinder
cuda_core → cuda_pathfinder, cuda_bindings
cuda_python → cuda_bindings (meta package)
Today, CI tests each sub-package in isolation against its own unit/functional tests (see ci/tools/run-tests). What we lack are integration tests that exercise cross-package interactions and end-to-end workflows using the published packages together — the way a real user would install and use them.
Gaps in current testing
-
No cross-package compatibility tests —
cuda.coreinternally callscuda.bindingsAPIs (e.g., driver/runtime bindings). Today each package is tested in isolation, so a breaking change incuda.bindings(changed signature, renamed enum, different return type) could silently breakcuda.corewithout CI catching it. Integration tests would install both built wheels together and verifycuda.coreoperations that depend oncuda.bindingsstill work end-to-end. -
No end-to-end user workflow tests — We don't test realistic user scenarios like:
pip install cuda-python[cu13]→ import all sub-packages → run a simple kernel- Install
cuda-corefrom PyPI wheel → usecuda.core.Programto compile and launch a kernel - Install from source (editable mode) → run tests
-
No downstream consumer smoke tests — We don't validate that packages like CuPy, numba-cuda, or Triton can still import and use cuda-python APIs after a change.
-
No pathfinder ↔ bindings ↔ core version-matrix tests — The backport-branch mechanism (
12.9.x) tests old bindings with new core, but we don't systematically test version compatibility boundaries (e.g.,cuda-pathfinder1.4.x vs 1.5.x withcuda-bindings13.x).
Proposed scope
Phase 1: Cross-package integration tests (in-repo)
- Add a new top-level
tests/integration/directory (orci/integration-tests/) - Tests install the built wheels from CI artifacts (same as current test jobs) but exercise cross-package scenarios:
- Import chain:
cuda.pathfinder→cuda.bindings→cuda.core→cuda.python - Compile-and-launch: use
cuda.coreAPIs that internally callcuda.bindingsto compile a trivial kernel and launch it - Pathfinder resolution: verify
cuda.pathfindercorrectly discovers CTK components thatcuda.bindingsandcuda.coreneed - Version consistency: all installed sub-packages report compatible versions
- Import chain:
- Wire into CI as a new step in
test-wheel-linux.yml/test-wheel-windows.ymlthat runs after individual module tests pass
Phase 2: End-to-end user scenario tests
- Simulate
pip install cuda-python[cu13]from scratch in a clean venv - Simulate editable installs (
pip install -e .) and verify tests pass - Test with both local CTK and wheel-based CTK
Phase 3: Downstream consumer smoke tests
- Minimal import/smoke tests for key consumers (CuPy, numba-cuda)
- Can run on a schedule (nightly) rather than per-PR to limit CI cost
Activity
mdboom commentedon Apr 22, 2026
I don't think (1) is quite true -- we don't test
cuda-corein isolation. We test two ways:cuda-coremain withcuda-bindingsmain andcuda-bindingslatest stable. So we can catch whencuda-bindingsbreakscuda-core. Here's a proof of concept of that, which fails as expected. Additionally, this setup allows us to develop features that require changes to both packages in parallel. I would describe the gap as this:For (2) I wouldn't spend too much time on editable builds. Editable builds are only really useful for pure Python packages. Since we have so much compiled code, we always need to recompile anyway, so editable builds IME don't provide any value and only represent a weird corner case. (If someone else is finding them useful, I'd love to stand corrected).
For (3), we used to run into this problem with CPython all the time. Particularly CPython main breaking projects with close ties to the internals of CPython, such as Cython and greenlets. The solution we took was not to own that integration ourselves but to make it easier for the downstream projects to test against nightlies of CPython. We already publish artifacts of the latest main -- I would work with these downstream projects to add one configuration that pulls cuda-core/cuda-bindings from there, rather than PyPI, just as they are likely already testing on multiple versions of Python today. The feedback would be a little delayed (it's not going to run on every cuda-python PR, for example), but should give us some feedback in advance of releases.
[-]Setup cuda-python integration tests[/-][+]EPIC: Setup cuda-python integration tests[/+]leofang commentedon Jun 8, 2026
On the one hand I agree with this. I haven't seen major projects testing this capability TBH. OTOH we did have issues in the past that we had to fix (#1073) though it's mostly impacting ourselves as maintainers, not our users.
leofang commentedon Jun 30, 2026
I think I misremembered what this issue is about. Let me reopen (and change priority/milestone). Sorry, Emily!
CI: nightly-cuda-core + nightly-numba-cuda-mlir + PTDS fix (#2283)