Skip to content

Publish Windows CUDA bundles and leave the CUDA runtime to the host - #25

Merged
danielhanchen merged 3 commits into
masterfrom
unsloth-cuda-shared-runtime
Oct 11, 2026
Merged

danielhanchen merged 3 commits into
masterfrom
unsloth-cuda-shared-runtime

Conversation

@danielhanchen

@danielhanchen danielhanchen commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Studio currently has no Windows CUDA build of sd.cpp from this mirror, so a Windows NVIDIA host that runs the native image engine falls back to an older upstream release (unslothai/unsloth#12470). This adds Windows CUDA bundles and makes every CUDA bundle much smaller by leaving the CUDA runtime to the host.

What changes

  • build-windows-cuda (new): win-cuda12-x64 and win-cuda13-x64, MSVC + Ninja, signed like the other Windows bundles, continue-on-error like the Linux CUDA leg.
  • build-linux-cuda: now a cuda12 / cuda13 matrix (Linux-Ubuntu-22.04-x86_64-cuda12, -cuda13).
  • CUDA toolkits come from NVIDIA's redist CDN with the same component tables as unslothai/llama.cpp's unsloth-cuda-{linux,windows}-setup, so both projects link the same CUDA sonames.
  • No CUDA runtime in any bundle. Studio always installs unslothai/llama.cpp, whose CUDA build already needs cudart / cublas / cublasLt and gets them from the venv's torch and nvidia-* wheels, CUDA_PATH, or the paired cudart archive on Windows. Studio picks the bundle whose CUDA major matches that llama.cpp build and gives sd-cli the same library directories (Studio: pin sd.cpp to the mirror build with Windows CUDA, sharing llama.cpp's CUDA runtime unsloth#13175). A step fails the build if a runtime library ends up in the bundle, or if any binary or library links a CUDA library outside the host set for its major (driver, cudart, cublas, cublasLt; no nvrtc, cuDNN or other major).
  • ggml is a shared library (SD_BUILD_SHARED_GGML_LIB), so sd-cli and sd-server share one copy of the CUDA kernels. The stable-diffusion library itself stays static, so the architecture markers Studio scans for stay in the executables.
  • Linux binaries and libs get DT_RPATH $ORIGIN (not RUNPATH): it is searched before LD_LIBRARY_PATH, so the bundle's libggml*.so always win over llama.cpp's same-named ones.
  • package_bundle.py ships each shared library once, under its soname. zipfile follows symlinks, so a shared-ggml build otherwise stores the CUDA library three times. Paths are compared resolved, so a relative BIN_DIR packages the same, and other symlinked binaries or libraries still ship under their own names as before.

Sizes (test build, not published)

bundle before after
Linux x86_64 cuda12 1168 MiB (828 MiB of CUDA runtime, kernels linked into both executables) 328 MiB
Linux x86_64 cuda13 n/a 292 MiB
Windows x64 cuda12 n/a (hosts used upstream: 322 MiB + 537 MiB cudart archive) 324 MiB
Windows x64 cuda13 n/a 287 MiB

Verification

  • Bundle checks on the CI artifacts: no runtime libraries, libggml-cuda.so.0 needs libcudart/libcublas/libcublasLt.so.<major>, every Linux ELF has RPATH $ORIGIN; win-cuda12-x64 ggml-cuda.dll imports cudart64_12.dll, cublas64_12.dll, cublasLt64_12.dll; win-cuda13-x64 imports cublas64_13.dll and cublasLt64_13.dll only (CUDA 13 links cudart statically, as unslothai/llama.cpp's cuda13 build does); every DLL / EXE in both carries an Authenticode signature.
  • Qwen-Image-2.1 Q4_K_M rendered through Studio's native engine with both CI Linux bundles on a B200; the running sd-server mapped libcudart.so.13 / libcublas.so.13 from the venv's nvidia/cu13/lib (the files llama-server uses).
  • Without the ld.so cache and without Studio's environment the runtime-less binary refuses to start (libcublasLt.so.13: cannot open shared object file); with Studio's environment it loads and lists the GPU. With llama.cpp's build dir placed first on LD_LIBRARY_PATH it still loads its own libggml*.so.
  • The two runtime-policy steps, run from this workflow's text against the CI bundles: pass on all four; fail as intended when told the wrong CUDA major and when a library is made to link nvrtc (8 negative controls).
  • Not run: no Windows GPU was available, so the Windows bundles are verified by contents, imports and signatures only.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for security reviews. Please try again later.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T05:16:53.204707Z c03c43b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@danielhanchen danielhanchen self-assigned this Oct 11, 2026
@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: 7003ade779

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@danielhanchen

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: c03c43bc1f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch was successfully deployed

1 active deployment
release-signing — c03c43bc Deployed Oct 11, 2026 by danielhanchen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant