Skip to content

[FEA]: Support CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig #2629

Closed
#2703
Closed
[FEA]: Support CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig#2629
#2703
Assignees
atiaomar1978-hublijinf2
Labels
cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Description

@lijinf2
lijinf2
opened on Aug 14, 2026

Is this a duplicate?

Area

cuda.core

Is your feature request related to a problem? Please describe.

LaunchConfig does not expose CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE, which sets the cluster scheduling policy preference (CUclusterSchedulingPolicy) for a launch.

Note: Kernel.cluster_scheduling_policy_preference exists as a function attribute getter; this request is for the launch attribute on LaunchConfig.

Parent tracking: #496.

Describe the solution you'd like

  • Users can set CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE via LaunchConfig.
  • Mapping to the native launch attribute is tested.

Describe alternatives you've considered

Use cuda.bindings.driver (CUlaunchAttribute / CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE) and cuLaunchKernelEx directly.

Downside: leaves cuda.core users without a first-class LaunchConfig API; they must drop to low-level bindings, manage the attribute array themselves, and lose consistency with other LaunchConfig launch attributes (cluster, is_cooperative, programmatic_stream_serialization, etc.).

Additional context

Incremental LaunchConfig attribute coverage under #496 (same approach as #1334 for programmatic_stream_serialization).

No example code in the design notes for this attribute.

Activity

  1. lijinf2
    added
    featureNew feature or request
    cuda.coreEverything related to the cuda.core module
    on Aug 14, 2026
  2. lijinf2
    added a parent issue on Aug 14, 2026
  3. github-project-automation
    moved this to Todo in CCCLon Aug 14, 2026
  4. github-project-automation
    added this to CCCLon Aug 14, 2026
  5. lijinf2
    changed the title [-]Add support for CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig[/-] [+][FEA]: Add support for CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig[/+] on Aug 14, 2026
  6. rparolin
    assigned
    lijinf2
    on Aug 14, 2026
  7. lijinf2
    changed the title [-][FEA]: Add support for CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig[/-] [+][FEA]: Support CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig[/+] on Aug 18, 2026
  8. github-project-automation
    moved this from Todo to In Progress in CCCLon Aug 18, 2026

  9. atiaomar1978-hub commented on Aug 18, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 18, 2026
    Contributor

    Implementation PR: #2668

    Adds LaunchConfig.cluster_scheduling_policy_preference via ClusterSchedulingPolicyType (DEFAULT, SPREAD, LOAD_BALANCING), mapping to CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE. Unlike #2628 sync policy (stream-only), this attribute is documented as valid for launches/graph nodes, so LaunchConfig is the correct surface.

    Tests cover native attribute mapping, validation, CC >= 9.0 gating, and a Hopper+ launch smoke test.

  10. atiaomar1978-hub commented on Aug 19, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 19, 2026
    Contributor

    Hi @lijinf2 — following your guidance on #2668, I'm holding off on reopening the PR until the issue is assigned to me.

    I've already implemented LaunchConfig.cluster_scheduling_policy_preference (ClusterSchedulingPolicyType, CC >= 9.0 gating, unit tests, and Hopper launch smoke coverage) in the closed PR. GPU validation passed on RunPod (CI-like CUDA 13.3 + driver 580).

    When you have a moment, could you please assign #2629 to me (@atiaomar1978-hub)? I'll reopen/update PR #2668 once assigned so review stays aligned with your workflow.

    Thanks again for the clear process — happy to adjust anything before resubmitting.

  11. lijinf2
    assigned
    atiaomar1978-hub
    on Aug 21, 2026

  12. lijinf2 commented on Aug 21, 2026

    @lijinf2
    lijinf2
    on Aug 21, 2026
    ContributorAuthor

    Hi @lijinf2 — following your guidance on #2668, I'm holding off on reopening the PR until the issue is assigned to me.

    I've already implemented LaunchConfig.cluster_scheduling_policy_preference (ClusterSchedulingPolicyType, CC >= 9.0 gating, unit tests, and Hopper launch smoke coverage) in the closed PR. GPU validation passed on RunPod (CI-like CUDA 13.3 + driver 580).

    When you have a moment, could you please assign #2629 to me (@atiaomar1978-hub)? I'll reopen/update PR #2668 once assigned so review stays aligned with your workflow.

    Thanks again for the clear process — happy to adjust anything before resubmitting.

    Hi @atiaomar1978-hub — before coding, could you briefly note how you plan to implement and test this?

    • API on LaunchConfig and mapping to CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE (same pattern as programmatic_stream_serialization.
    • Tests for that mapping (see test_launcher.py), plus skip conditions if hardware support is required.

  13. atiaomar1978-hub commented on Aug 22, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 22, 2026
    Contributor

    Hi @lijinf2 — thanks for the guidance. Brief plan:

    Unlike #2628 (sync policy, stream-only), this attribute is documented for launches and graph nodes, so LaunchConfig is the right surface.

    API / mapping (same pattern as programmatic_stream_serialization):

    • Add optional LaunchConfig.cluster_scheduling_policy_preference: ClusterSchedulingPolicyType | None
    • Enum in cuda.core.typing: DEFAULT, SPREAD, LOAD_BALANCING (maps to CUclusterSchedulingPolicy)
    • In _to_native_launch_config, when set, append CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE with clusterSchedulingPolicyPreference
    • Accept None (driver/kernel default), enum members, or raw CUclusterSchedulingPolicy ints
    • Gate on CC >= 9.0 (same check as LaunchConfig.cluster)

    Tests (test_launcher.py):

    • Parametrized mapping test: each policy → correct CUlaunchAttributeID + value
    • Validation: invalid int → ValueError; CC < 9.0 → CUDAError
    • Combined attrs: cluster + policy → two launch attributes
    • GPU smoke on Hopper+: launch() with cluster=(2,1,1) + LOAD_BALANCING (skip if CC < 9.0)

    I already have a prototype in closed PR #2668; happy to reopen/adjust once #2629 is assigned. Let me know if you'd prefer a different enum name or stricter CC gating.

    Thanks!

  14. atiaomar1978-hub commented on Aug 22, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 22, 2026
    Contributor

    Hi @lijinf2 — a bit more context on what cluster_scheduling_policy_preference controls, in case it helps review.

    Unlike #2628 (sync policy, stream-only), this attribute is documented for launches and graph nodes, so LaunchConfig is the right surface. At a high level:

    • LaunchConfig (CPU) → _to_native_launch_config() → CUlaunchConfig (driver) → cuLaunchKernelEx (GPU)
    • This field only matters when launching with cluster=(…) (Hopper+, CC >= 9.0)

    cluster_scheduling_policy_preference

    How the driver places blocks inside a thread-block cluster on SMs. Set on LaunchConfig at launch time; None leaves the kernel/driver default.

    Value Driver enum What it means (GPU) When to use
    DEFAULT CU_CLUSTER_SCHEDULING_POLICY_DEFAULT Driver/kernel default placement Don't care; let hardware/driver decide
    SPREAD CU_CLUSTER_SCHEDULING_POLICY_SPREAD Spread cluster blocks across SMs in the cluster Reduce contention; spread work evenly
    LOAD_BALANCING CU_CLUSTER_SCHEDULING_POLICY_LOAD_BALANCING Allow hardware load-balancing of cluster blocks Uneven work per block; let GPU rebalance

    Metaphor

    • LaunchConfig = flight plan written in the office (CPU)
    • CUlaunchConfig = form the control tower uses (driver)
    • Kernel launch = plane actually flying the route (GPU SMs running threads)

    For this attribute: the flight plan includes how to seat passengers (blocks) across rows (SMs) within one cabin section (cluster) — SPREAD vs LOAD_BALANCING vs "use airline default."

    Implementation plan is in my previous comment; prototype remains in closed PR #2668 pending assignment.

  15. lijinf2 commented on Aug 25, 2026

    @lijinf2
    lijinf2
    on Aug 25, 2026
    ContributorAuthor

    @atiaomar1978-hub

    Thanks for the plan! The implementation part looks good to me. The testing part it looks like there is no way to test the actual scheduling policy at runtime. It should be good enough to test that getter/setter can work properly, and test that application code runs through when setting policy to DEFAULT/SPREAD/LOAD_BALANCING. Feel free to put up a PR if aligned, or let me know if you have different thoughts.

  16. atiaomar1978-hub
    added a commit that references this issue on Aug 25, 2026

    test(cuda.core): cover cluster policy getter/setter and launch run-th…

    dfbec3a

  17. atiaomar1978-hub commented on Aug 25, 2026

    @atiaomar1978-hub
    atiaomar1978-hub
    on Aug 25, 2026
    Contributor

    Hi @lijinf2 — thanks, the plan is aligned with your note.

    GitHub would not reopen closed PR #2668. I first opened #2702 (unsigned commits), then replaced it with a signed re-issue on current main:

    #2703

    Please review #2703. Same change as #2702: LaunchConfig.cluster_scheduling_policy_preference maps to CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE.

    Tests match your guidance (no attempt to observe the live scheduler):

    • getter/setter round-trip for DEFAULT / SPREAD / LOAD_BALANCING
    • native mapping + CC >= 9.0 gate
    • launch() run-through for all three policies on Hopper+ (skip on CC < 9.0)

    RunPod (2026-08-25), NVIDIA H100 80GB HBM3, CC 9.0, driver 580.126.09, CTK 13.3: 13 passed, 0 skipped.

    test_to_native_launch_config_cluster_scheduling_policy[0/1/2] PASSED
test_to_native_launch_config_cluster_scheduling_policy_accepts_driver_enum PASSED
test_launch_config_cluster_scheduling_policy_invalid_value_rejected PASSED
test_launch_config_cluster_scheduling_policy_requires_cc_90 PASSED
test_to_native_launch_config_cluster_and_scheduling_policy PASSED
test_launch_config_cluster_scheduling_policy_getter_setter[DEFAULT/SPREAD/LOAD_BALANCING] PASSED
test_launch_cluster_scheduling_policy_smoke[DEFAULT/SPREAD/LOAD_BALANCING] PASSED
  18. atiaomar1978-hub
    added 2 commits that reference this issue on Aug 26, 2026

    feat(cuda.core): add cluster scheduling policy to LaunchConfig

    3374af4

    test(cuda.core): cover cluster policy getter/setter and launch run-th…

    c6c283b
  19. lijinf2
    added a commit that references this issue on Sep 15, 2026

    feat(cuda.core): add cluster scheduling policy to LaunchConfig (#2703)

    ca91432
  20. lijinf2
    closed this as completedin #2703on Sep 15, 2026
  21. github-project-automation
    moved this from In Progress to Done in CCCLon Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Type

No type

Projects

  • CCCL
    Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions