Skip to content

Better support for PEP-517 build backends #2060

Closed

Description

@dalcinl

I have a project that builds CUDA code into shared library. This shared library is then loaded with ctypes to be used from pure-Python code passing buffers from CuPy arrays.

My pyproject.toml uses the Cmake-based scikit-build-core build system, configured as per the nowadays widespread mechanism of PEP-517.

Ideally, I would like to have something minimal like:

[build-system]
build-backend = "scikit_build_core.build"
requires = [
  "cuda-toolkit[cccl,nvcc,cudart]==13.*",
  "scikit-build-core",
]

to configure the build-system dependencies and a companion CMakeLists.txt for CMake to handle the build of the shared library.

Unfortunately, this does not work out of the box. The reason is simple: CMake knows nothing about the install layout of the various NVIDIA/CUDA packages withing a virtual environment, thus it fails to find the nvcc compiler located in <venv>/lib/pythonX.Y/site-packages/nvidia/cuZZ/bin/nvcc.

I managed to find a workaround that could become a sensible feature addition. All what would be needed is to install a *.pth file in the virtual environment. This file would look for NVCC and then prepend <bindir> to the process $PATH.
A proof of concept is available here: https://github.com/dalcinl/skbuild-cuda-demo/

Please not this line https://github.com/dalcinl/skbuild-cuda-demo/blob/master/pyproject.toml#L18. In there, I'm adding a local project cuda-activate as an additional build dependency. This project is the one that manages to eventually install a *.pth file.

After cloning the git repo, you can check that everything works smoothly using uv (otherwise, do export PROJECT_ROOT=$PWD first):

$ uv run --group test pytest

If you comment out this line https://github.com/dalcinl/skbuild-cuda-demo/blob/master/pyproject.toml#L18 then things will no longer work, CMake will not find nvcc.

Long story short: I would like to propose an opt-in and backward compatible mechanism (via a new small package) such PEP-517 build backends would find NVIDIA tools in automatically. One possible API for this opt-in mechanism could be

[build-system]
requires = [
  "cuda-toolkit[build-system]",
  ...
]

where the extra build-system (or any name you see fit) would bring in an extra package that install *.pth file in charge of extending $PATH (i.e os.environ["PATH"]).
A prototype for this new package is what I wrote for my demo: https://github.com/dalcinl/skbuild-cuda-demo/tree/master/src/cuda-activate which could be of course renamed, e.g cuda-build-system for consistency with the extras name I proposed avobe.

However, there are a few things that I do not like in the code, like depending on finding nvcc vs possibly other tools. I do not know the exact layout and contents of all the new packages in the cuda-python ecosystem, therefore would need some advice here.

Thoughts?

Activity

  1. self-assigned this
    on May 11, 2026
  2. added
    packagingAnything related to wheels or Conda packages
    on May 11, 2026
  3. leofang commented on May 13, 2026

    @leofang
    Member

    Thanks, Lisandro! Will bring this issue to our internal packaging meeting, as discussed. Though I just realized: PyCon US starts tomorrow and several of our people will be at the conference, so please expect slower-than-usual response from us 😅

  4. leofang commented on Jun 4, 2026

    @leofang
    Member

    Update: The packaging wg reviewed and agreed we want to address this. Filed KITMAKER-4373 to track.

  5. rparolin commented on Jun 17, 2026

    @rparolin
    Collaborator

    @leofang Did we want to close this issue since its been forwarded to the Kitmaker tracker?

  6. dalcinl commented on Jun 17, 2026

    @dalcinl
    Author

    Any chance someone can explain here what exactly the proposed resolution would be?

  7. leofang commented on Jul 9, 2026

    @leofang
    Member

    We discussed adding an entry-point script to each wheel that contains executables. But last time I checked no work has started yet.

  8. removed their assignment
    on Jul 9, 2026
  9. dalcinl commented on Jul 9, 2026

    @dalcinl
    Author

    Do you mean a Python entry point? So every invocation o let say nvcc would spawn a Python process that runs an entry point script that then finds nvcc within the package to finally execute it?

  10. leofang commented on Jul 10, 2026

    @leofang
    Member

    Yes, to my understanding.

  11. leofang commented on Aug 28, 2026

    @leofang
    Member

    I see that this is implemented but I dunno at which CTK version this feature would be published. Will follow up internally. In any case, this isn't a task for us so let me move this thread to GH Discussion.

  12. locked and limited conversation to collaborators on Aug 28, 2026
  13. converted this issue into a discussion #2732 on Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

featureNew feature or requestpackagingAnything related to wheels or Conda packagestriageNeeds the team's attention

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions