Skip to content

[FEA]: Support CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig #2628

Closed
Closed
[FEA]: Support CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig#2628
Assignees
atiaomar1978-hublijinf2
Labels
cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Description

@lijinf2
lijinf2
opened on Aug 14, 2026

Is this a duplicate?

Area

cuda.core

Is your feature request related to a problem? Please describe.

LaunchConfig does not expose CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY, which overrides the CPU wait policy (SPIN / YIELD / BLOCKING_SYNC) for a launch.

Parent tracking: #496.

Describe the solution you'd like

  • Users can set CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY via LaunchConfig.
  • Mapping to the native launch attribute is tested.

Describe alternatives you've considered

  1. Use cuda.bindings.driver (CUlaunchAttribute / CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY) and cuLaunchKernelEx directly.
  2. Set the stream attribute via cuStreamSetAttribute(..., CU_STREAM_ATTRIBUTE_SYNCHRONIZATION_POLICY, ...) instead of a per-launch attribute.

Downside of (1)/(2): leaves cuda.core users without a first-class LaunchConfig API; they must drop to low-level bindings and lose consistency with other LaunchConfig launch attributes (is_cooperative, programmatic_stream_serialization, etc.).

Additional context

Incremental LaunchConfig attribute coverage under #496 (same approach as #1334 for programmatic_stream_serialization).

Example (C API) — agent-generated; reviewed but not executed:

// Per-launch SPIN policy (lowest-latency host sync)
CUlaunchAttribute syncAttr = {
    .id    = CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY,
    .value = { .syncPolicy = CU_SYNC_POLICY_SPIN }
};

CUlaunchConfig config = {
    .gridDimX = 128, .gridDimY = 1, .gridDimZ = 1,
    .blockDimX = 256, .blockDimY = 1, .blockDimZ = 1,
    .sharedMemBytes = 0,
    .hStream = latencyCriticalStream,
    .attrs = &syncAttr,
    .numAttrs = 1
};

cuLaunchKernelEx(&config, kernel, NULL, NULL);

// Subsequent sync on this stream busy-waits on CPU
cuStreamSynchronize(latencyCriticalStream);

Activity

  1. lijinf2
    added
    featureNew feature or request
    cuda.coreEverything related to the cuda.core module
    on Aug 14, 2026
  2. lijinf2
    added a parent issue on Aug 14, 2026
  3. github-project-automation
    added this to CCCLon Aug 14, 2026
  4. github-project-automation
    moved this to Todo in CCCLon Aug 14, 2026
  5. lijinf2
    changed the title [-]Add support for CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig[/-] [+][FEA]: Add support for CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig[/+] on Aug 14, 2026
  6. github-project-automation
    moved this from Todo to In Progress in CCCLon Aug 14, 2026

  7. atiaomar1978-hub commented on Aug 14, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 14, 2026
    Contributor

    Implementation available in PR #2637

    I implemented the requested LaunchConfig support for CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY.

    What changed

    • New LaunchConfig.synchronization_policy parameter (optional, default None)
    • New public enum cuda.core.typing.SynchronizationPolicyType with AUTO, SPIN, YIELD, and BLOCKING_SYNC, aligned with cuda.bindings.driver.CUsynchronizationPolicy
    • Native mapping to CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in _to_native_launch_config() (both the cdef method used by launch() and the cpdef helper used in tests)

    Tests added (tests/test_launcher.py)

    • Attribute mapping for all four policies (including acceptance of driver enum values)
    • Invalid input rejection
    • Default None produces no launch attribute
    • Combination with is_cooperative
    • GPU launch smoke test (skipped when CUDA 13 bindings run on a CUDA 12 driver)

    GPU validation (RunPod RTX A5000)

    pytest cuda_core/tests/test_launcher.py -k "synchronization_policy" -v
# 10 passed, 4 skipped, 0 failed

    Full details: #2637 (comment)

    This satisfies both checklist items from this issue: users can set the policy via LaunchConfig, and mapping to the native launch attribute is tested.

  8. atiaomar1978-hub
    added a commit that references this issue on Aug 14, 2026

    feat(cuda.core): add synchronization_policy to LaunchConfig

    df550c9
  9. rparolin
    assigned
    lijinf2
    on Aug 14, 2026
  10. atiaomar1978-hub
    added 2 commits that reference this issue on Aug 15, 2026

    style: sort imports in test_launcher.py (NVIDIA#2628)

    dd69219

    feat(cuda.core): add synchronization_policy to LaunchConfig

    ce87db2

  11. lijinf2 commented on Aug 17, 2026

    @lijinf2
    lijinf2
    on Aug 17, 2026
    ContributorAuthor

    Hi @atiaomar1978-hub ,

    I've added the labels and kicked off CI. Once the checks finish, please take a look and fix any failures if needed. Let's get CI green first, then we can start the code review.

    Thanks!

  12. atiaomar1978-hub commented on Aug 17, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 17, 2026
    Contributor

    Thanks @lijinf2 I will keep an eye.

  13. atiaomar1978-hub
    added 2 commits that reference this issue on Aug 18, 2026

    revert(cuda.core): drop LaunchConfig.synchronization_policy (NVIDIA#2637

    71763c5

    feat(cuda.core): add Stream.synchronization_policy via cuStreamSetAtt…

    64b318a

  14. atiaomar1978-hub commented on Aug 18, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 18, 2026
    Contributor

    Direction update (PR #2637)

    We originally tried LaunchConfig.synchronization_policy via cuLaunchKernelEx, but the driver rejects CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY at launch time (CUDA_ERROR_INVALID_VALUE on every GPU runner). CUDA docs in this repo mark it "Valid for streams", not launches — cuStreamSetAttribute accepts it.

    Implemented instead: Stream.synchronization_policy (get/set via cuStreamGetAttribute / cuStreamSetAttribute), plus optional StreamOptions.synchronization_policy at stream creation and SynchronizationPolicyType in cuda.core.typing.

    Validation: all 13 synchronization_policy stream tests pass on RunPod with CI-like CUDA 13.3 setup and driver 580+ (including launch+sync cases that require CUDA 13 drivers).

    Happy to retitle/re-scope this issue to the stream-level API if that works for maintainers.

  15. lijinf2
    changed the title [-][FEA]: Add support for CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig[/-] [+][FEA]: Support CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in LaunchConfig[/+] on Aug 18, 2026

  16. atiaomar1978-hub commented on Aug 19, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 19, 2026
    Contributor

    @lijinf2 What's needed for #2628

    Checked upstream/main:

    Surface Status
    (2) Stream attr Already in cuda.bindings.driver (cuStreamSetAttribute, CU_STREAM_ATTRIBUTE_SYNCHRONIZATION_POLICY, CUsynchronizationPolicy)
    (1) LaunchConfig Not in cuda.core. We implemented it in PR #2637 first: LaunchConfig.synchronization_policy → CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY in CUlaunchConfig.attrs, passed to cuLaunchKernelEx. Fails: every policy (AUTO/SPIN/YIELD/BLOCKING_SYNC) returns CUDA_ERROR_INVALID_VALUE (CI GPUs + RunPod). Control: same noop kernel + CU_LAUNCH_ATTRIBUTE_COOPERATIVE succeeds; cuStreamSetAttribute(..., SYNCHRONIZATION_POLICY, ...) succeeds with identical values. Vendored docs: attribute is "Valid for streams", not launches.
    cuda.core wrapper Not on main (PR #2637 only)

    So (2) is already available via bindings; a cuda.core wrapper would duplicate that. (1) is what this issue asks for, but the driver rejects it today — not a cuda-python mapping bug.

    Please advise: driver fix / supported launch path for (1), or close/re-scope #2628.

  17. atiaomar1978-hub commented on Aug 19, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 19, 2026
    Contributor

    RunPod repro — option (1) still fails on CI-like stack (driver 580 + CUDA 13.3)

    RTX 4090 · driver 580.95.05 · mini-CTK 13.3.0 · cuda-bindings==13.3.1

    Path Result
    cuLaunchKernelEx + SYNCHRONIZATION_POLICY (all 4 policies) CUDA_ERROR_INVALID_VALUE (1)
    cuLaunchKernelEx + COOPERATIVE (control) CUDA_SUCCESS (0)
    cuStreamSetAttribute + same policies CUDA_SUCCESS (0)

    Docs mark this attribute "Valid for streams" — driver rejects it at launch time, not a cuda-python mapping bug. Driver 580 does not fix option (1); would need future driver support for launch-time acceptance.

    Full probe log 
    ############ ISSUE #2628 OPTION (1) PROBE ############
========================================================================
Issue #2628 — LaunchConfig option (1) driver probe
Timestamp (UTC): 2026-08-19T05:21:13.630835+00:00
========================================================================

--- Environment (CI-like) ---
CUDA_HOME=/workspace/cuda-python/cuda_toolkit
CUDA_CORE_BUILD_MAJOR=13
cuda-bindings=13.3.1
#define CUDA_VERSION 13030

--- GPU / driver ---
Wed Aug 19 05:21:13 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.95.05              Driver Version: 580.95.05      CUDA Version: 13.0     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4090        On  |   00000000:81:00.0 Off |                  Off |
| 30%   25C    P8             51W /  450W |       0MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

GPU: NVIDIA GeForce RTX 4090�����������������������������������������                                                                
Driver: CUDA 13.0 (cuDriverGetVersion=13000)
Compute capability: sm_89

--- Documentation ---
CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY: Valid for streams (driver.rst). CU_LAUNCH_ATTRIBUTE_COOPERATIVE: Valid for graph nodes, launches.

--- Option (1): cuLaunchKernelEx + CU_LAUNCH_ATTRIBUTE_* ---
  A. baseline (no attrs)
    -> CUDA_SUCCESS (numeric=0, repr=<CUresult.CUDA_SUCCESS: 0>)
  B. CU_LAUNCH_ATTRIBUTE_COOPERATIVE (documented for launches)
    -> CUDA_SUCCESS (numeric=0, repr=<CUresult.CUDA_SUCCESS: 0>)
  C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_AUTO
    -> CUDA_ERROR_INVALID_VALUE (numeric=1, repr=<CUresult.CUDA_ERROR_INVALID_VALUE: 1>)
  C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_SPIN
    -> CUDA_ERROR_INVALID_VALUE (numeric=1, repr=<CUresult.CUDA_ERROR_INVALID_VALUE: 1>)
  C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_YIELD
    -> CUDA_ERROR_INVALID_VALUE (numeric=1, repr=<CUresult.CUDA_ERROR_INVALID_VALUE: 1>)
  C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_BLOCKING_SYNC
    -> CUDA_ERROR_INVALID_VALUE (numeric=1, repr=<CUresult.CUDA_ERROR_INVALID_VALUE: 1>)

--- Option (2): cuStreamSetAttribute (documented path) ---
  D. cuStreamSetAttribute SYNC_POLICY=CU_SYNC_POLICY_SPIN
    -> CUDA_SUCCESS (numeric=0, repr=<CUresult.CUDA_SUCCESS: 0>)
  D. cuStreamSetAttribute SYNC_POLICY=CU_SYNC_POLICY_BLOCKING_SYNC
    -> CUDA_SUCCESS (numeric=0, repr=<CUresult.CUDA_SUCCESS: 0>)

========================================================================
VERDICT — why option (1) cannot ship as LaunchConfig today
========================================================================
  cuLaunchKernelEx + SYNCHRONIZATION_POLICY: ALL FAIL -> CUDA_ERROR_INVALID_VALUE (1)
    policies tested: ['CU_SYNC_POLICY_AUTO', 'CU_SYNC_POLICY_SPIN', 'CU_SYNC_POLICY_YIELD', 'CU_SYNC_POLICY_BLOCKING_SYNC']
  cuLaunchKernelEx + COOPERATIVE (control):    PASS
  cuStreamSetAttribute + SYNCHRONIZATION_POLICY: ALL PASS

  Root cause: driver rejects launch-time attribute; docs mark it stream-only.
  Not a cuda-python mapping bug — identical values succeed on cuStreamSetAttribute.

  Driver update needed for option (1) IF NVIDIA intends launch-time support:
    - Header/bindings expose CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY for launches
    - Driver must accept it in CUlaunchConfig.attrs passed to cuLaunchKernelEx
    - Until then, use option (2): cuStreamSetAttribute / Stream.synchronization_policy
========================================================================
    JSON report 
    {
  "issue": "NVIDIA/cuda-python#2628",
  "option_1": "LaunchConfig.synchronization_policy -> cuLaunchKernelEx",
  "option_2": "Stream.synchronization_policy -> cuStreamSetAttribute",
  "environment": {
    "gpu": "NVIDIA GeForce RTX 4090",
    "compute_capability": "8.9",
    "driver_version_raw": 13000,
    "driver_version_human": "CUDA 13.0",
    "bindings_version": "13.3.1",
    "cuda_header": "#define CUDA_VERSION 13030",
    "ci_note": "cuda-python CI linux-64 GPU jobs use CUDA 13.3 mini-CTK + driver 580+"
  },
  "documentation": "CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY: Valid for streams (driver.rst). CU_LAUNCH_ATTRIBUTE_COOPERATIVE: Valid for graph nodes, launches.",
  "launches": [
    {
      "label": "A. baseline (no attrs)",
      "result": {
        "name": "CUDA_SUCCESS",
        "value": 0,
        "repr": "<CUresult.CUDA_SUCCESS: 0>"
      },
      "success": true
    },
    {
      "label": "B. CU_LAUNCH_ATTRIBUTE_COOPERATIVE (documented for launches)",
      "result": {
        "name": "CUDA_SUCCESS",
        "value": 0,
        "repr": "<CUresult.CUDA_SUCCESS: 0>"
      },
      "success": true
    },
    {
      "label": "C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_AUTO",
      "result": {
        "name": "CUDA_ERROR_INVALID_VALUE",
        "value": 1,
        "repr": "<CUresult.CUDA_ERROR_INVALID_VALUE: 1>"
      },
      "success": false
    },
    {
      "label": "C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_SPIN",
      "result": {
        "name": "CUDA_ERROR_INVALID_VALUE",
        "value": 1,
        "repr": "<CUresult.CUDA_ERROR_INVALID_VALUE: 1>"
      },
      "success": false
    },
    {
      "label": "C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_YIELD",
      "result": {
        "name": "CUDA_ERROR_INVALID_VALUE",
        "value": 1,
        "repr": "<CUresult.CUDA_ERROR_INVALID_VALUE: 1>"
      },
      "success": false
    },
    {
      "label": "C. CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY=CU_SYNC_POLICY_BLOCKING_SYNC",
      "result": {
        "name": "CUDA_ERROR_INVALID_VALUE",
        "value": 1,
        "repr": "<CUresult.CUDA_ERROR_INVALID_VALUE: 1>"
      },
      "success": false
    }
  ],
  "streams": [
    {
      "label": "D. cuStreamSetAttribute SYNC_POLICY=CU_SYNC_POLICY_SPIN",
      "result": {
        "name": "CUDA_SUCCESS",
        "value": 0,
        "repr": "<CUresult.CUDA_SUCCESS: 0>"
      },
      "success": true
    },
    {
      "label": "D. cuStreamSetAttribute SYNC_POLICY=CU_SYNC_POLICY_BLOCKING_SYNC",
      "result": {
        "name": "CUDA_SUCCESS",
        "value": 0,
        "repr": "<CUresult.CUDA_SUCCESS: 0>"
      },
      "success": true
    }
  ],
  "verdict": {
    "launch_sync_policy_all_invalid_value": true,
    "launch_cooperative_control_ok": true,
    "stream_set_attribute_ok": true,
    "blocking_error_code": "CUDA_ERROR_INVALID_VALUE",
    "blocking_error_numeric": 1
  }
}
  18. lijinf2
    assigned
    atiaomar1978-hub
    on Aug 19, 2026

  19. lijinf2 commented on Aug 19, 2026

    @lijinf2
    lijinf2
    on Aug 19, 2026
    ContributorAuthor

    Hi @atiaomar1978-hub — thank you for the thorough investigation, the stream-side implementation in #2637, and the RunPod repro. That work made the real constraint obvious.

    I independently checked the C++ cuda.h APIs as well. Same result: CU_LAUNCH_ATTRIBUTE_SYNCHRONIZATION_POLICY on a launch config is rejected with CUDA_ERROR_INVALID_VALUE for all four policies, while cuStreamSetAttribute succeeds. This is not a Python mapping issue.

    I was wrong to treat this as a LaunchConfig attribute. The CU_LAUNCH_ATTRIBUTE_* name is easy to misread, and the cuda.h comment only says Valid for streams — it never explicitly says not for launches. I took the enum name plus the union field as “this belongs on cuLaunchKernelEx.” Sorry for the extra churn.

    I’ll close this issue. I also added you as an assignee on the ticket. Further suggestions welcome.

  20. lijinf2
    closed this as completedon Aug 19, 2026

  21. atiaomar1978-hub commented on Aug 19, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 19, 2026
    Contributor

    Thanks @lijinf2 — really appreciate you validating this independently on the C++ side and for the clear write-up. The "Valid for streams" wording in cuda.h is exactly what tripped us up too; the enum living under CU_LAUNCH_ATTRIBUTE_* makes the launch path look plausible until you hit the driver.

    Agree closing #2628 makes sense. The stream-side path in #2637 is the right surface for cuda.core users until launch-time support exists in the driver.

    Happy to help on any follow-up if useful. Thanks again for the review and for adding me on the ticket.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Type

No type

Projects

  • CCCL
    Status
    In Progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions