CUDA graph phase N - explicit graph construction #1317

Activity

1. leofang

   added this to the cuda.core backlog milestone on Dec 4, 2025
2. leofang

   added a parent issue on Dec 4, 2025

   • [EPIC] Support CUDA graphs #111
3. leofang

   added

   enhancementAny code-related improvementsAny code-related improvements

   triageNeeds the team's attentionNeeds the team's attention

   cuda.coreEverything related to the cuda.core moduleEverything related to the cuda.core module

   on Dec 4, 2025
4. rparolin

   removed

   triageNeeds the team's attentionNeeds the team's attention

   on Jan 19, 2026
5. Andy-Jost

   self-assigned this

   on Mar 3, 2026
6. Andy-Jost

   modified the milestones: cuda.core backlog, cuda.core v0.7.0 on Mar 3, 2026
7. Andy-Jost commented on Mar 3, 2026

   

   Andy-Jost

   on Mar 3, 2026

   Contributor

   More actions

   I'm picking this up and trying to nail down the scope. The issue describes an alternative GraphBuilder backend that uses explicit graph construction (cuGraphCreate + cuGraphAddNode) instead of stream capture.

   Although the graph interface is mostly independent of the construction method, I see two conceptual leaks:

   • The GraphBuilder.stream property is tightly coupled to capture. The entire usage pattern is kernel.launch(builder.stream, ...).
   • begin_building(mode=...) takes a mode parameter that maps directly to CUstreamCaptureMode.

   If I understand the proposal correctly, we want to retain the same user-facing API while changing the construction method internally. That makes this a large cross-cutting change: every capturable operation needs a dispatch branch. For example, kernel launch becomes something like:

   class Kernel:
       def launch(self, stream, ...):
           graph_builder = _get_active_builder(stream)
           if graph_builder is not None:
               graph_builder._add_kernel_node(self, ...)
           else:
               cuLaunchKernel(...)

   This would add overhead to the hot paths and would largely duplicate logic already handled by the driver's own graph capture machinery, in addition to creating new maintenance burdens.

   Before starting the implementation, I'd like to understand the motivation better. What use cases or user stories would explicit construction enable that stream capture doesn't handle well today? What does a "win" look like here? A concrete example would help a lot.
8. Andy-Jost commented on Mar 3, 2026

   

   Andy-Jost

   on Mar 3, 2026

   Contributor

   More actions

   After discussion the plan is:

   1. Develop a private module for explicit graph construction (cuGraphCreate + cuGraphAddNode), not exposed as public API.
   2. Test it in isolation.
   3. Wire it into the public GraphBuilder interface in a follow-up.

   Stream capture remains the public-facing mechanism for now. Explicit construction becomes an internal implementation detail that enables a unified interface later.
9. Andy-Jost

   added 4 commits that reference this issue on Mar 6, 2026

   Convert _graph.py to _graph/ package for explicit graph construction …

   ...

   478101c

   Convert _graph.py to _graph/ package for explicit graph construction …

   ...

   188d3d6

   Add RAII infrastructure for explicit graph construction

   ...

   52d6f75
10. Andy-Jost

    closed this as completedin #1772on Mar 17, 2026