Repository navigation
cuda.core: add append-only attachment ownership for executable graphs #2353
Description
Background
Part of #1330. Depends on #2350.
CUDA propagates user objects from a CUgraph when it creates or updates a CUgraphExec, but provides no API for directly retaining or releasing user objects on an existing executable graph. Individual executable-node updates may introduce new callbacks, buffers, events, kernels, or argument owners whose lifetimes must extend through future launches.
Add one exec-private, append-only attachment accumulator that CUDA retains as a user object. Whole-graph updates replace it with a fresh accumulator, providing a safe reclamation boundary.
Design
GraphExecBox
CUgraphExec
CUuserObject attachment_object # non-owning handle
ExecAttachments* attachments # non-owning pointer
ExecAttachments
vector<OpaqueHandle>
The CUDA user object directly owns ExecAttachments. GraphExecBox stores a raw pointer because its CUgraphExec guarantees the user object remains alive while the box is usable. The box must not access the pointer after calling cuGraphExecDestroy.
The exec does not retain its source CUgraph. Source identity and node-view lifetime are part of #2354. This preserves normal CUDA behavior: after instantiation or whole-graph update returns, callers may destroy the source graph without affecting the executable.
Instantiation
Every managed instantiation creates a unique accumulator:
- Allocate an empty
ExecAttachmentsand create its user object. - Temporarily retain that user object on the source
CUgraph. - Instantiate the graph; CUDA propagates the reference to the new
CUgraphExec. - Release the source graph's temporary reference.
- Store the user-object handle and accumulator pointer in
GraphExecBox.
If attachment or instantiation fails, release the temporary graph reference and destroy any partially created exec. Multiple execs instantiated from one definition receive independent accumulators.
Whole-graph update
Every Graph.update(source) uses a fresh accumulator:
- Create a new empty accumulator user object.
- Temporarily retain it on the update source graph.
- Call
cuGraphExecUpdate. - On success, replace the box's accumulator handle and pointer.
- Release the source graph's temporary reference.
CUDA replaces the exec's old user-object set with the source graph's objects and the fresh accumulator. It retains the old accumulator until all earlier launches finish, then destroys it asynchronously.
If the update fails, release the fresh accumulator and leave the exec and its existing accumulator unchanged.
Internal append transaction
Provide an internal operation for appending one or more OpaqueHandle owners to the current exec accumulator. It must support:
- Recording the original accumulator size.
- Appending every replacement owner before changing CUDA node parameters.
- Keeping the additions after driver success.
- Restoring the original size after driver failure.
Successful additions are never removed individually. They remain until the next successful whole-graph update or final exec destruction.
Destruction
GraphExecBox destruction calls cuGraphExecDestroy and then discards its non-owning accumulator fields. CUDA may destroy the accumulator immediately or defer destruction until an in-flight launch completes.
The CUDA user-object callback must not release Python-backed owners or call CUDA APIs. It schedules ExecAttachments for the existing deferred cleanup path, which releases its OpaqueHandle entries from a safe Python thread.
Acceptance criteria
- Every managed exec receives a unique accumulator propagated through CUDA user-object retention.
- The exec does not retain its source
CUgraph; destroying the source after instantiation or update does not affect the accumulator or executable behavior. - Destroying an exec with an in-flight launch does not release accumulated owners before launch completion.
- A successful
Graph.update()installs a fresh accumulator and releases the old one only when CUDA determines it is safe. - A failed
Graph.update()preserves the old accumulator and releases the unused replacement. - Multiple execs instantiated from one definition do not share accumulated update owners.
- Internal append rollback leaves no retained owner after a failed transaction.
- Existing graph instantiation, update, launch, upload, and close behavior remains unchanged.
Non-goals
- Source-graph identity, source-node lifetime, and public executable-node update methods, covered by cuda.core: add atomic update methods for executable graph nodes #2354.
- Removing individual accumulated owners.
- Tracking or synchronizing launch streams.
- Definition-level attachment ownership, covered by cuda.core: retain graph node attachments as per-node CUDA user objects #2350.
- Definition-level node updates, covered by cuda.core: add atomic update methods for graph definition nodes #2352.
Activity