Design of graph support - Phase 1 #114

Activity

1. github-actions

   added

   triageNeeds the team's attentionNeeds the team's attention

   on Sep 19, 2024
2. leofang

   added

   cuda.py beta2

   on Sep 19, 2024
3. leofang

   added this to the cuda.core beta 2 milestone on Oct 9, 2024
4. leofang

   added

   P1Medium priority - Should doMedium priority - Should do

   featureNew feature or requestNew feature or request

   cuda.coreEverything related to the cuda.core moduleEverything related to the cuda.core module

   and removed

   cuda.core beta2

   triageNeeds the team's attentionNeeds the team's attention

   on Oct 9, 2024
5. leofang

   assigned

   leofang

   and

   vzhurba01

   on Oct 29, 2024
6. leofang commented on Nov 12, 2024

   

   leofang

   on Nov 12, 2024

   Last edited by leofang

   MemberAuthor

   More actions

   A few pointers to consider when we design this:

   • Add support for conditional nodes in cuda graphs. pytorch/pytorch#130386
   • Initial work on "cuda graphs with arguments". pytorch/pytorch#137318
   • Add CUDA Graphs conditional node support and user-friendly APIs cupy/cupy#8615
   • cuda graph api proposal numba/numba#4182
7. leofang

   added

   P0High priority - Must do!High priority - Must do!

   and removed

   P1Medium priority - Should doMedium priority - Should do

   on Nov 14, 2024
8. leofang

   removed their assignment

   on Nov 14, 2024
9. leofang

   modified the milestones: cuda.core beta 2, cuda.core beta 3 on Dec 11, 2024
10. leofang commented on Jan 23, 2025

    

    leofang

    on Jan 23, 2025

    MemberAuthor

    More actions

    Discussed internally. With all things considered will take a multi-phase approach to iteratively enhance the CUDA graph coverage. Below is the phase-1 design considerations:

    • Only cover stream capture (no explicit graph construction)
    • Exclude memory allocation/deallocation steps, and assume when entering the capturing context all needed memory are already allocated by the user
    • Exclude host callback operations
    • Basic coverage for conditional nodes
    • The resulting graph should be replay-able, meaning
      • user objects' lifetimes are properly managed
      • ...
11. leofang

    changed the title [-]Design of graph support[/-] [+]Design of graph support - Phase 1[/+] on Mar 3, 2025
12. leofang commented on Mar 24, 2025

    

    leofang

    on Mar 24, 2025

    MemberAuthor

    More actions

    Design is being wrapped up with a prototype (#455). Moving this to beta 4.
13. leofang

    modified the milestones: cuda.core beta 3, cuda.core beta 4 on Mar 24, 2025
14. leofang

    closed this as completedin #455on Jun 10, 2025