[BUG]: CI may build cuda.core against published cuda-bindings instead of the same-commit wheel #2468

Activity

1. rwgk

   added

   bugSomething isn't workingSomething isn't working

   P0High priority - Must do!High priority - Must do!

   CI/CDCI/CD infrastructureCI/CD infrastructure

   on Jul 31, 2026
2. rwgk

   added the

   Bug

   issue type on Jul 31, 2026
3. rwgk commented on Aug 5, 2026

   

   rwgk

   on Aug 5, 2026

   ContributorAuthor

   More actions

   This is not a new problem, although the earlier cases were not identical.

   In January, #1454 documented a closely related provenance failure: during an isolated build, a local or just-built cuda-bindings could be displaced by a published PyPI candidate. That PR included a proposed safeguard in cuda_core/build_hooks.py, but the broader change did not land. The review also identified an important weakness in that implementation: passively inspecting whether cuda-bindings was already installed would rarely work under normal PEP 517 build isolation.

   The same broader incident also produced #1470, which explored fail-fast validation for invalid development versions. Both #1454 and #1470 were closed without merging. A narrower wheel-install CI correction was extracted into #1493 and did merge. Those changes addressed parts of the incident, but the more general dependency-provenance risk remained. I think this history is useful to consider because it shows both that accidental PyPI substitution is a recurring risk and that the solution must operate correctly under build isolation.

   We now have two PRs addressing the immediate problem in this issue:

   • ci: add selective wheel build plumbing #2464 uses exact, direct-reference pip constraints for local cuda-pathfinder and cuda-bindings wheels. It keeps artifact-selection policy in CI and integrates it with the selective-build work.
   • ci: build cuda.core against the exact same-checkout cuda-bindings wheel #2470 makes the dynamic cuda-bindings build requirement point directly at a selected local wheel, applies that protection to wheel, sdist, and coverage builds, and reports the imported bindings version and location.

   Both approaches improve materially on PIP_FIND_LINKS and PIP_PRE, neither of which identifies the exact artifact that CI intends to validate. #2464 has the cleaner CI-owned selection mechanism and also covers cuda-pathfinder, but its current exact-artifact protection is confined to the wheel workflow. #2470 has more complete workflow coverage and diagnostics, but puts CI-specific artifact-selection behavior into the distributed cuda_core build backend and addresses only cuda-bindings.

   Rather than choosing one PR wholesale, I think we should take the historical incident, this issue, and the complementary strengths of #2464 and #2470 into account in a focused alternative. The goal would be to make accidental PyPI substitution less likely more generally.

   The immediate failure in #2468 concerns cuda.core compiling against the wrong cuda-bindings, but the same resolver pattern can affect any CI stage that builds one same-checkout CUDA Python package against another while a public index remains available. We should therefore minimize accidental PyPI substitution more generally, within a clear boundary.

   Core invariant

   In CI intended to test same-checkout CUDA Python packages, once a local internal artifact is selected, pip must consume that exact artifact or fail. Public indexes remain available for third-party dependencies and for workflows explicitly testing PyPI or TestPyPI.

   What to combine

   From #2464, I propose retaining:

   • exact direct-reference constraints for both cuda-pathfinder and cuda-bindings;
   • stage-specific selection of current- and previous-major artifacts;
   • compatibility with artifacts that were either built in the current run or downloaded for selective reuse;
   • CI ownership of dependency-resolution policy.

   From #2470, I propose retaining:

   • coverage of current- and previous-major wheel builds, Linux and Windows sdist builds, and Linux and Windows coverage builds;
   • fail-closed behavior when the expected local artifact cannot be identified;
   • the build-time diagnostic showing the imported cuda-bindings version and location.

   I propose using exact CI-supplied constraints as the single artifact-selection mechanism. The normal cuda.core build-backend requirement would remain unchanged, while the CI caller would constrain that requirement to an exact local wheel. The build hook would retain the provenance diagnostic, but would not need a CI-specific wheel-directory selection contract.

   PIP_FIND_LINKS may remain useful for discovery in other contexts, but it should not be considered sufficient protection against substitution. PIP_PRE=1 is also insufficient because it only makes prereleases eligible; it does not select the intended artifact.

   Proposed implementation shape

   1. Add one small, cross-platform, tested helper that generates constraint files from selected local wheel artifacts.

      The helper should:

      • emit PEP 508 direct references such as cuda-bindings @ file:///.../cuda_bindings-....whl;
      • use proper URI construction so paths containing spaces work;
      • validate the distribution name and, for bindings, the expected CUDA major;
      • require exactly one matching artifact and fail with a descriptive error for missing or ambiguous inputs;
      • print the selected package, version, and wheel path.

   2. Generate constraints for each dependency edge rather than one global file. For example:

      • building cuda-bindings: constrain cuda-pathfinder;
      • building current cuda-core: constrain current cuda-pathfinder and current cuda-bindings;
      • building previous-major cuda-core: constrain the selected pathfinder and previous-major bindings artifacts.

   3. Supply both PIP_BUILD_CONSTRAINT and PIP_CONSTRAINT to the relevant build and install steps. The protected workflows should explicitly provide pip 25.3 or newer, where dedicated build constraints are supported.

   4. Apply the mechanism to every same-checkout build path, including:

      • current- and previous-major production wheel builds;
      • Linux and Windows wheel-from-sdist builds;
      • Linux and Windows coverage builds;
      • any metapackage or test-environment installation that is expected to consume the selected local wheels.

      Workflows whose purpose is to verify packages from PyPI or TestPyPI should remain explicitly outside this policy.

   5. Preserve the cuda_core/build_hooks.py diagnostic after cuda.bindings is imported:

      print(
          f"Using cuda-bindings {cuda.bindings.__version__} from {bindings_path}",
          file=sys.stderr,
      )

      This records what Cython actually consumed, complementing the helper's record of the wheel selected for installation.

   6. Test the helper's successful and failure paths, including missing, ambiguous, wrong-distribution, wrong-major, and spaces-in-path cases. The real Linux and Windows wheel, sdist, and coverage jobs should then provide integration coverage that pip honors the generated constraints.

   7. Keep public indexes enabled. Globally setting PIP_NO_INDEX would be unnecessarily broad because these environments still need third-party packages and some workflows intentionally verify released CUDA Python packages. Exact constraints should protect only the internal artifacts selected for same-checkout testing.

   Proposed PR strategy

   I propose preparing a new draft PR from current main, focused specifically on exact same-checkout dependency provenance.

   • The PR would credit and cross-reference Add __version__ validation: import-time assertions, unit tests, PyPI fallback prevention, CI hardening #1454, Add version validation to prevent silent setuptools-scm fallback versions #1470, ci: add selective wheel build plumbing #2464, ci: build cuda.core against the exact same-checkout cuda-bindings wheel #2470, and this issue.
   • It would take the exact-constraint mechanism from ci: add selective wheel build plumbing #2464 and the workflow coverage and diagnostics from ci: build cuda.core against the exact same-checkout cuda-bindings wheel #2470, without absorbing the broader selective-CI feature set.
   • ci: add selective wheel build plumbing #2464 and ci: build cuda.core against the exact same-checkout cuda-bindings wheel #2470 would remain open while the alternative is reviewed and exercised in CI; there is no need to decide between them before the combined design has concrete code and test results.
   • The authors and reviewers involved in both approaches would be asked to review the alternative explicitly, especially the CI-versus-build-backend boundary and whether every intended same-checkout path is covered.
   • If the alternative is accepted, ci: build cuda.core against the exact same-checkout cuda-bindings wheel #2470 could be closed as superseded, and ci: add selective wheel build plumbing #2464 could be rebased so its selective-build work reuses the common exact-artifact mechanism instead of carrying overlapping resolver logic.
4. rwgk commented on Aug 6, 2026

   

   rwgk

   on Aug 6, 2026

   ContributorAuthor

   More actions

   Analysis of PR 2509 provenance output in main CI before PR 2510

   Conclusion

   The new diagnostics from PR 2509 make the accidental package-index substitution unambiguous.

   In the successful main CI run 31071375906, the previous-CTK cuda.core build had a local cuda-bindings 12.9.8.dev9+ga4d338b9a wheel available, but 18 matrix jobs reported that the build actually imported released cuda-bindings 12.9.7. The configured index is NVIDIA's PyPI proxy/cache, so this is more precisely an index-sourced released wheel rather than the provided local CI artifact; saying "from PyPI" is reasonable shorthand.

   The run also shows the same problem even more consistently for cuda-pathfinder: every provenance print reports released 1.6.0, although the run built 1.6.1.dev44+g979b65fb2 from the checkout.

   This is a strong before-state validation for PR 2510. It does not, of course, exercise PR 2510 itself.

   Run inspected

   • Workflow: CI, triggered by a push to main
   • Result: successful
   • Head: 979b65f (#2512, merged after PR 2509)
   • PR 2509 squash commit: 7e7d3c1, present in the run's history
   • PR 2510 commit: f39b4c0, not present in the run's history

   I inspected all 24 wheel-build jobs: eight interpreter variants (3.10, 3.11, 3.12, 3.13, 3.14, 3.14t, 3.15, and 3.15t) on each of linux-64, linux-aarch64, and win-64.

   Direct cuda-bindings evidence

   Every wheel-build job had the previous-branch artifact cuda-bindings 12.9.8.dev9+ga4d338b9a available locally before building cuda.core against the previous CTK. The build requirement was cuda-bindings==12.*, and the local artifact directory was supplied through PIP_FIND_LINKS.

   The result split by interpreter compatibility:

   Matrix rows	Count	Version reported by PR 2509
   Python 3.10 through 3.14, including 3.14t, on all three platforms	18	cuda-bindings 12.9.7
   Python 3.15 and 3.15t on all three platforms	6	cuda-bindings 12.9.8.dev9+ga4d338b9a

   The Linux x86-64 Python 3.10 job provides a compact example:

   cuda_bindings-12.9.8.dev9+ga4d338b9a-cp310-cp310-manylinux_..._x86_64.whl
   ...
   - cuda-bindings==12.*
   ...
   Using cuda-bindings 12.9.7 from /tmp/build-env-_cwv7nwr/lib/python3.10/site-packages/cuda/bindings
   

   See Build linux-64, CUDA 13.3.0 / py3.10.

   Windows shows the same result:

   cuda_bindings-12.9.8.dev9+ga4d338b9a-cp310-cp310-win_amd64.whl
   ...
   - cuda-bindings==12.*
   ...
   Using cuda-bindings 12.9.7 from C:\Users\runneradmin\AppData\Local\Temp\build-env-v03d7oio\lib\site-packages\cuda\bindings
   

   See Build win-64, CUDA 13.3.0 / py3.10. The Linux aarch64 Python 3.10 job reports the same 12.9.7 result in job 92527767778.

   The Python 3.15 rows are a useful control. They report the exact local dev version, for example:

   Using cuda-bindings 12.9.8.dev9+ga4d338b9a from /tmp/build-env-rieig1hv/lib/python3.15/site-packages/cuda/bindings
   

   See Build linux-64, CUDA 13.3.0 / py3.15. This is consistent with no compatible 12.9.7 wheel being available for Python 3.15, leaving the local dev wheel as the usable candidate. The logs establish the selection, not that resolver rationale. This contrast is especially helpful: CI provenance currently depends on what compatible releases happen to exist on the package index.

   The temporary build-env path alone would not prove the source, because a local wheel would also be installed into that isolated environment. The decisive evidence is the version mismatch: 12.9.7 was imported even though the local artifact was 12.9.8.dev9+ga4d338b9a.

   Why the pre-2510 behavior occurs

   PIP_FIND_LINKS adds a directory to pip's candidate sources; it does not make the directory exclusive or give its candidates provenance priority over the configured index. The checkout and previous-branch wheels use development versions, while stable released candidates are present on the index. With requirements such as cuda-bindings==12.*, pip can therefore choose the released candidate instead of the local dev wheel.

   PIP_PRE=1 would make prereleases eligible, but it would still not encode the desired provenance invariant. It would also broaden prerelease selection for dependencies generally.

   PR 2510 instead:

   • reads the local wheel metadata and creates exact direct-reference constraints such as cuda-bindings @ file:///.../cuda_bindings-...whl;
   • fails closed when the expected wheel is missing, ambiguous, for the wrong project, or for the wrong CUDA major;
   • supplies the constraints through both PIP_BUILD_CONSTRAINT and PIP_CONSTRAINT so isolated PEP 517 build dependencies are covered;
   • applies the mechanism to cuda-pathfinder when building cuda-bindings, and to both internal wheels when building current- and previous-CTK cuda.core, including the sdist and coverage workflows;
   • leaves the package index available for unrelated external dependencies.

   Diagnostic caveats

   There are two limitations in what this run displays:

   1. None of the 24 current-CTK builds emits a Using cuda-bindings 13.x line. They request cuda-bindings==13.* and build successfully, but PR 2509's bindings print is inside a best-effort import guarded by except ImportError. The logs do not establish why that import did not produce a line, and they do not directly identify the selected 13.x version. The previous-CTK 12.x result and the pathfinder results are nevertheless conclusive evidence of substitution.
   2. The two sdist jobs do not surface the provenance messages at their default frontend verbosity. Their pre-2510 workflows also use PIP_FIND_LINKS, so PR 2510 appropriately covers them, but the counts above (96 messages total) refer only to the 24 wheel-build jobs.

   The missing 13.x print is worth remembering when reading future logs: the absence of a provenance line does not establish that the local wheel was selected. PR 2510 improves observability here as well because its constraint-generation step prints the exact wheel selected before the build, independently of whether import cuda.bindings succeeds in the backend hook.

   Recommendation

   The run provides exactly the before-state evidence that PR 2510 is intended to address. I see no reason in these logs to delay the already-approved PR 2510.

   After PR 2510 is merged, I would verify one main run for:

   • Selected cuda-pathfinder==1.6.1.dev... from ... as file://... in the constraint-generation steps;
   • Selected cuda-bindings==13.3.2.dev... for the current-CTK build;
   • Selected cuda-bindings==12.9.8.dev... for the previous-CTK build;
   • matching build-hook provenance versions wherever those hooks can import and report the packages.

   That follow-up would confirm the after-state; this run already documents the problem very clearly.

   ---

   cuda-pathfinder baseline and broader PR 2510 policy

   The pre-2510 production-wheel result is uniform:

   • Each of the 24 jobs built cuda-pathfinder 1.6.1.dev44+g979b65fb2 from the checked-out main commit.
   • There are 72 pathfinder provenance messages, three per job: during the cuda-bindings build, the current-CTK cuda.core build, and the previous-CTK cuda.core build.
   • All 72 report cuda-pathfinder 1.6.0 from a temporary PEP 517 build environment.
   • The surrounding pip output confirms that the cached cuda_pathfinder-1.6.0 wheel from the configured package index was selected.

   For example, the Linux x86-64 Python 3.10 job on main first builds 1.6.1.dev44+g979b65fb2, but the three dependent builds report:

   Using cuda-pathfinder 1.6.0 from /tmp/build-env-_wgc_ix0/lib/python3.10/site-packages/cuda/pathfinder
   ...
   Using cuda-pathfinder 1.6.0 from /tmp/build-env-zsnrkfr6/lib/python3.10/site-packages/cuda/pathfinder
   ...
   Using cuda-pathfinder 1.6.0 from /tmp/build-env-_cwv7nwr/lib/python3.10/site-packages/cuda/pathfinder
   

   These messages are useful provenance, but this was not another instance of the issue 2468 failure under the pre-2510 workflow contract. In these production-wheel jobs, the workflow did not make the just-built pathfinder wheel available to the isolated cuda-bindings and cuda.core builds, so resolving cuda-pathfinder>=1.5 from the configured index was expected. That differs from the problematic cuda-bindings edge: a local bindings wheel directory was supplied, yet pip was still free to choose a released wheel from PyPI.

   Build isolation explains why the imported package lives under /tmp/build-env-*; it does not require that the package originate on PyPI. A build frontend can install either an index candidate or an exact local-wheel reference into the isolated environment.

   PR 2510 intentionally broadens the same-checkout policy to include pathfinder. In the representative PR 2510 Linux x86-64 Python 3.10 job, each of the three protected dependency edges selects cuda-pathfinder 1.6.1.dev45+gf39b4c008 through an exact file:///host/... wheel constraint. Each corresponding build-hook message then reports that same version from its isolated build environment. The same constraint mechanism selects the appropriate exact local cuda-bindings wheel for the current- and previous-CTK cuda.core builds.

   The before-state therefore documents an expected index-based pathfinder baseline, while the PR 2510 job demonstrates a deliberate policy change: dependencies produced by the same checkout are selected exactly and cannot be silently replaced by similarly named packages from an index. Build isolation is preserved; dependency provenance becomes deterministic. If compatibility with a released cuda-pathfinder is important, it should be exercised by an explicit test configuration rather than as an incidental resolver outcome in the same-checkout build.
5. rwgk commented on Aug 6, 2026

   

   rwgk

   on Aug 6, 2026

   ContributorAuthor

   More actions

   Before/after CI evidence for exact local-wheel constraints

   This is a follow-up to the earlier pre-2510 log analysis. I compared two adjacent points in the current stack:

   • Before enforcement: PR build: report cuda-bindings provenance in PEP 517 builds #2520, run 31091062660, head 9ef67b6. This includes the build-hook provenance improvement, but not ci: constrain internal builds to exact local wheels #2510's exact-wheel constraints.
   • After enforcement: PR ci: constrain internal builds to exact local wheels #2510, run 31090411239, head 5fcc5fd. Because ci: constrain internal builds to exact local wheels #2510 is stacked directly on build: report cuda-bindings provenance in PEP 517 builds #2520, it has the same provenance prints plus the exact local-wheel policy.

   This gives us a useful controlled comparison: #2520 shows what the isolated build frontend selected under the old policy, and #2510 shows what changes when each internal build dependency is constrained to one exact wheel file.

   Scope

   I counted isolated build-dependency consumers, not the downstream jobs that test already-built artifacts:

   • 24 production wheel jobs: 3 platforms (linux-64, linux-aarch64, and win-64) x 8 Python variants (3.10, 3.11, 3.12, 3.13, 3.14, 3.14t, 3.15, and 3.15t).
   • Each wheel job builds current-CTK cuda.bindings, current-CTK cuda.core, and previous-CTK cuda.core: 72 isolated consumer builds and 120 internal dependency resolutions in total.
   • 2 sdist jobs (Linux and Windows), each rebuilding cuda.bindings and cuda.core from their sdists: 4 additional consumer builds and 6 internal dependency resolutions.
   • Coverage is a separate workflow and is qualified separately below.

   Results at a glance

   Consumer	Executions	Before: PR #2520	After: PR #2510
   Current-CTK cuda.bindings	24	24/24 selected released cuda-pathfinder 1.6.0, not the wheel built in that job	24/24 selected the exact job-built pathfinder wheel
   Current-CTK cuda.core	24	Pathfinder: 24/24 released 1.6.0. Bindings: 18/24 released 13.3.1; 6/24 local dev wheel	24/24 selected the exact job-built pathfinder and current-bindings wheels
   Previous-CTK cuda.core	24	Pathfinder: 24/24 released 1.6.0. Bindings: 18/24 released 12.9.7; 6/24 intended previous-branch dev wheel	24/24 selected the exact job-built pathfinder wheel and exact downloaded previous-branch bindings wheel
   Wheel-from-sdist cuda.bindings	2	Successful, but selected dependency was not visible in non-verbose pip wheel output	2/2 constrained to the exact local pathfinder wheel
   Wheel-from-sdist cuda.core	2	Successful, but selected dependencies were not visible in non-verbose pip wheel output	2/2 constrained to the exact local pathfinder and bindings wheels

   For the 24-job wheel matrix, this can be summarized as follows:

   Evidence	Before: PR #2520	After: PR #2510
   Constraint-generator Selected ... lines	0	120
   Backend Using cuda-pathfinder ... lines	72	72
   Backend Using cuda-bindings ... lines	48	48
   Dependency resolutions matching the intended artifact	12/120	120/120
   Released/index substitutions	108/120	0/120

   The 12 incidental matches in the before run are six current-core and six previous-core bindings resolutions for Python 3.15 and 3.15t. They matched because there was no compatible released wheel available, not because the workflow required the local artifact.

   Before: compatible candidates could still win

   PR #2520 built local cuda-pathfinder 1.6.1.dev45+g9ef67b610 and current cuda-bindings 13.3.2.dev163+g9ef67b610. It also downloaded previous-branch cuda-bindings 12.9.8.dev9+ga4d338b9a for the previous-CTK core build.

   Across all 24 wheel jobs, the backend provenance was completely consistent:

   • All 72/72 pathfinder imports used released 1.6.0, covering the bindings build plus both core builds in every job.
   • For Python 3.10 through 3.14t on all three platforms, 18/24 current-core builds used released cuda-bindings 13.3.1, and 18/24 previous-core builds used released 12.9.7.
   • Only Python 3.15 and 3.15t, 6/24 jobs in each bindings group, used the available dev artifacts.

   The pattern was identical on Linux x86-64, Linux aarch64, and Windows. Representative examples are Linux x86-64 / Python 3.10, Linux aarch64 / Python 3.10, Windows / Python 3.10, and Linux x86-64 / Python 3.15.

   The imported packages correctly appear under temporary PEP 517 build-environment site-packages directories. The evidence of substitution is the version mismatch against the wheel produced or downloaded by that job, not the temporary import path itself.

   This confirms the underlying problem: PIP_FIND_LINKS made the local wheel available as another candidate, but did not require it. A compatible released distribution could silently win. PIP_PRE=1 would broaden prerelease eligibility, but would still not express the required artifact identity.

   After: exact selection and backend import agree

   PR #2510 built local cuda-pathfinder 1.6.1.dev47+g5fcc5fd8d and current cuda-bindings 13.3.2.dev165+g5fcc5fd8d, and downloaded previous-branch cuda-bindings 12.9.8.dev9+ga4d338b9a into a separate directory.

   All 24 production jobs succeeded. Each job emitted exactly five constraint-generator selection lines and five matching backend provenance lines:

   1. The bindings build selected and imported the exact current-job pathfinder wheel.
   2. The current-core build selected and imported the exact current-job pathfinder and bindings wheels.
   3. The previous-core build selected and imported the exact current-job pathfinder wheel and exact downloaded previous-branch bindings wheel.

   That is 120 exact artifact selections and 120 matching backend imports, with no version or ordering anomalies across the full platform/Python matrix. For example, the Linux x86-64 / Python 3.10 job reports direct-file selections for all five dependencies, followed by Using ... lines with the same versions from the isolated build environments. Linux container paths are mapped to file:///host/...; Windows uses file:///D:/a/... paths.

   The two evidence channels are separate and complementary:

   1. write_wheel_constraints.py reports which exact local artifact it wrote into the constraint file.
   2. The build hook reports the package version actually imported inside the isolated PEP 517 backend.

   Agreement between them across all 120 wheel-matrix dependency resolutions is the strongest runtime evidence that the policy is working as intended.

   sdist builds

   Both before-run sdist jobs succeeded (Linux, Windows), but their non-verbose pip wheel commands suppressed successful backend stderr. Their zero Using ... lines are therefore an observability limitation, not evidence for or against substitution.

   After #2510, the corresponding Linux and Windows jobs each emitted three constraint-generator selection lines: exact local pathfinder for the bindings build, followed by exact local pathfinder and bindings for the core build. All 6/6 selections became direct-reference constraint entries, and both jobs completed successfully under those constraints. Backend Using ... lines remain suppressed at the current pip verbosity, but exact selection is now explicit before each build starts.

   Coverage qualification

   Neither PR CI run executes .github/workflows/coverage.yml; that workflow runs on a schedule or by manual dispatch. The latest relevant pre-fix coverage run 31060413129 independently demonstrated the same risk: Linux selected released pathfinder/bindings packages, and Windows downloaded public cuda-bindings 13.4.0b1 despite a local 13.3.2.dev160+g418595b78 wheel being available. The subsequent Windows core compile failure is consistent with that unintended version mismatch, although the log alone does not prove causality.

   PR #2510 updates all four relevant coverage consumers (Linux bindings/core and Windows bindings/core) to use the same exact-wheel mechanism. There is no coverage run at the PR head, however, so runtime confirmation for this separate workflow remains outstanding. I have not included coverage in the 120 production-wheel or 6 sdist dependency counts.

   CI status

   • PR build: report cuda-bindings provenance in PEP 517 builds #2520's full CI run completed successfully. Its separate known-unrelated build smoke (cu13, linux-64, CPU) failure is not part of this analysis.
   • All 24 production wheel jobs and both sdist jobs in PR ci: constrain internal builds to exact local wheels #2510 completed successfully.
   • PR ci: constrain internal builds to exact local wheels #2510's Windows Python 3.15 GPU job initially failed while actions/setup-python looked for ~3.15.0-0 in the runner's local tool cache. The same matrix cell passed in earlier runs, so I reran only that job; the rerun completed successfully.
   • PR ci: constrain internal builds to exact local wheels #2510's separate known-unrelated source-build smoke failure was intentionally left untouched.

   At the final check, the only failed check reported for either PR was its respective instance of that excluded source-build smoke failure.

   Conclusion

   The before run shows that successful builds can silently substitute compatible released packages for artifacts built in the same workflow. The after run changes the policy from "consider these wheels" to "use these exact files," and the production-wheel logs cross-check each selected file against what the isolated backend actually imported.

   For every wheel and sdist consumer exercised here, the enforced policy now satisfies the core invariant: an internal build dependency resolves to the exact wheel produced or downloaded for that workflow, not merely to any compatible distribution available from an index.

   The one remaining runtime-validation item is the independently scheduled coverage workflow.
6. rparolin

   assigned

   rwgk

   on Aug 6, 2026
7. rparolin

   added this to the cuda.bindings 13.4.0 & 12.9.8 milestone on Aug 6, 2026
8. rwgk

   closed this as completedin #2510on Aug 7, 2026