Repository navigation
Bump tensor bridge version cap for PyTorch 2.12 #2089
Description
Summary
The tensor bridge's _torch_version_check() in _memoryview.pyx caps the supported PyTorch version range at (2, 3) <= (major, minor) <= (2, 11). PyTorch 2.12 was released on ~May 14, 2026, causing the version check to fail. When this happens, _is_torch_tensor() returns False and torch tensors fall through to the generic CAI/DLPack paths instead of the fast AOTI tensor bridge.
For from_cuda_array_interface, this causes a hard failure because PyTorch reports __cuda_array_interface__ version 2, while cuda-python requires version 3+:
BufferError: only CUDA Array Interface v3 or above is supported
Root cause
# _memoryview.pyx
cdef inline bint _torch_version_check():
...
_torch_version_ok = (2, 3) <= (major, minor) <= (2, 11) # ← needs bumpingThe upper bound exists because the tensor bridge relies on undocumented PyTorch internals (THPVariable struct layout, AtenTensorHandle == at::Tensor*). The docstring says: "Bump the upper bound after verifying a new PyTorch release."
Action needed
- Verify PyTorch 2.12's
THPVariablestruct layout is unchanged (checktorch/csrc/autograd/python_variable.h) - Bump the upper bound to
(2, 12) - Re-add
latestto the nightly CI pytorch test matrix (currently pinned to2.11.0as a workaround in Add nightly CI for optional-dependency testing (PyTorch, numba-cuda) #1987)
Discovered by
Nightly optional-dependency CI (#1987) — the latest torch entry started failing when PyTorch 2.12 was released.
-- Leo's bot
Activity
cuda.core: bump tensor bridge PyTorch upper-bound to 2.12 (#2099)
leofang commentedon May 18, 2026
I am reopening this issue because the work is not done yet (see "Action needed"). We need to actually verify this (and we can now!) in the CI, and then bump the nightly pinning on PyTorch version.
rwgk commentedon May 18, 2026
Before merging #2099, I used Cursor for a verification based on inspecting the sources, which I believe is the most conclusive approach. Runtime tests may pass just by luck (very common for UB situations).
That's quite a bit of churn going forward. Did you see this comment:
#2099 (review)
After adding the suggested feature, we could run the nightly CI with
export CUDA_PYTHON_TORCH_TENSOR_BRIDGE_COMPATIBILITY_CHECK=WARN, plus some easy alert mechanism similar to what I proposed under #1943, then all we'd need to do in almost all cases (assuming long-term ABI stability) is one one-line PR per pytorch release to silence the warning.leofang commentedon May 18, 2026
Code inspection (especially with agents) is actually the least safe approach, otherwise we wouldn't have to have follow-up fixes after the first PR #1894 was merged. The CI tests actually revealed extra issues that could not be caught by reading the code.
I did, but I am against adding any new env vars for such a niche feature. If it is worth fixing at all, we should just fix the fallback path so that any new PyTorch release would not fail at the CAI test, or just skip the test for PyTorch, or address it upstream once and for all (#1894 (comment)). We signed up for this churn (which, by the way, is not coming from user requests, but from our own CI testing) and I don't think this is urgent enough to address. I will handle the rest of this issue.
rwgk commentedon May 19, 2026
That makes sense. I agree CI is the right way to catch integration issues like the Windows DLL loading problem, CPU-only/CUDA package differences, and missed test paths like #1999.
The point I was making was narrower: for the specific 2.12 version-cap bump, the core question was whether the private layout assumptions changed (
THPVariableandAtenTensorHandle). For that question, I trust a source diff/inspection more than runtime tests alone, since tests can pass even when an unsafe layout assumption is only accidentally still working.So I think we want both: CI for the integration matrix, and source inspection for the private ABI/layout assumption before bumping the cap.
Thanks, that makes sense. I agree the best outcome is to address this upstream so we can stop relying on the private
THPVariable/AtenTensorHandleassumptions and eventually remove most of this release-by-release churn. I posted a comment on the PyTorch issue to help narrow thePyObject*ABI concern:AtenTensorHandlefrom the tensor object via AOTI stable C API pytorch/pytorch#180107 (comment)I see what you mean about the fallback path: supporting PyTorch there would mean accepting its CAI v2 export instead of rejecting it before it can be useful. The gotcha is that PyTorch’s CAI v2 omits
stream, so that fallback would still need a clear stream-ordering story.