Repository navigation
Sync with upstream master-968-8cd731e (FLUX.3 Image Klein) - #33
Conversation
Brings in 17 upstream commits up to leejet/stable-diffusion.cpp 8cd731e (FLUX.3 Image Klein and Klein Base, leejet#2143).
…e flash attention Upstream leejet#2132 maps --diffusion-fa to --flash-attn off,diffusion=on and no longer sets the legacy diffusion_flash_attn flag, so the H3 VAE flash-attention rule (CUDA/ROCm by default) never fired after the sync. Key it on the resolved per-module state instead: diffusion FA on, VAE FA off, and no explicit vae= assignment.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca7d7af876
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| sd_version_is_minimax_h3(version)) { | ||
| // Keyed on the resolved per-module state: the CLI now spells --diffusion-fa as --flash-attn off,diffusion=on and | ||
| // never sets the legacy diffusion_flash_attn flag. An explicit vae= assignment is the user's choice and wins. | ||
| const bool vae_fa_assigned = std::string(SAFE_STR(sd_ctx_params->flash_attn_mode)).find("vae=") != std::string::npos; |
There was a problem hiding this comment.
On MiniMax-H3 with CUDA/ROCm, this raw substring check recognizes only the exact spelling vae=. The assignment parser also accepts whitespace, case differences, and aliases such as first_stage, autoencoder, and tae, so a valid setting like --flash-attn 'off,diffusion=on,first_stage=off' is parsed as an explicit VAE disable but is not detected here; the following block then enables VAE flash attention anyway. Check the parsed flash_attn_assignment.module_names for SDBackendModule::VAE rather than searching the original string.
Useful? React with 👍 / 👎.
8 participants
Syncs the fork with upstream
master-968-8cd731e(2026-10-16), which adds FLUX.3 Image Klein and Klein Base (leejet#2143). The last sync, #24, brought the tree tomaster-951-f89d9b1, so this adds 17 upstream commits. The ggml submodule is unchanged (d25b121), and all 13 patches inscripts/unsloth/ggml-patchesstill apply.Merge this with "Create a merge commit", not squash or rebase. The prebuilt workflow names a release after the highest upstream
master-*tag reachable from the tree. If this PR is squashed,8cd731eis not reachable and the next release is still namedmaster-951-.... The upstream tagmaster-968-8cd731ealso has to be in this repository (git push origin refs/tags/master-968-8cd731e), asmaster-951-f89d9b1is today.Upstream commits brought in
flux3.hpp, Qwen3-VL-4B conditioning (layers 4..32, padded to multiples of 80), a flow-map denoiser for the distilled model, up to 10 reference images.auto.--diffusion-fa,--fa,--diffusion-conv-directand--vae-conv-directstill work, now as deprecated aliases (--diffusion-fa=--flash-attn off,diffusion=on).0e1d733).Conflicts
Two files conflicted.
.github/workflows/build.yml, Windows ROCm job. Took upstream's ROCm 10.1.0. Kept the fork'sGPU_TARGETSenv name, which is what the job's cmake line reads, andCCACHE_COMPILERCHECK: content. Kept the fork's SHA-pinned actions and themax-size: 2Gcomment. Added upstream's "Tune ccache for reinstalled ROCm toolchain" step. Every action in the workflows is still pinned to a commit.src/core/ggml_extend_backend.cpp. An include clash: kept both<tuple>(fork) and<utility>(upstream).Beyond the textual merge
ca7d7af). Upstream feat: add per-module automatic flash attention selection leejet/stable-diffusion.cpp#2132 rewrote--diffusion-faas a module assignment and no longer sets the legacydiffusion_flash_attnflag. The fork's rule ("on CUDA and ROCm,--diffusion-faalso covers the H3 video VAE") read that flag, so it would never have fired again and H3 decode would have silently lost its flash attention. It now keys on the resolved per-module state: diffusion FA on and VAE FA off, unless the user gave an explicitvae=assignment.SD_H3_VAE_FLASH_ATTNbehaves as before. The log line "Using flash attention in the MiniMax-H3 video VAE" appears in both master and this branch.Behaviour that changes with upstream
--flash-attn off --conv-direct offis pixel-identical to master's default (CPU and CUDA, see below), so this is the whole difference.--diffusion-fa --diffusion-conv-direct. With those flags, CPU output is pixel-identical between master and this branch. On CUDA the VAE also gets upstream's auto direct conv, which master never enabled (about 60 dB from master, LPIPS 0.0001 to 0.0002).Verification
Builds (CUDA 13.1, sm_100, cuDNN on, shared ggml; CPU
GGML_NATIVE=OFFwith all CPU variants andGGML_BACKEND_DL, as the prebuilt) all succeed:sd-cliandsd-serverfor master11761efand for this branch.The A/B compares master
11761efagainst this branch on the same host and GPU, interleaved, with seed 42. Assets are those Studio ships for each family.Assets per family:
unsloth/Z-Image-Turbo-ComfyUIAE + Qwen3-4B, 8 steps.unsloth/FLUX.2-VAE+ Qwen3-4B, 4 steps.unsloth/Qwen-Image-ComfyUIVAE + Qwen2.5-VL-7B Q4_K_M, euler, shift 3, 20 steps, CFG 2.5.unsloth/flux-text-encoders, 4 steps. The FLUX.1 AE comes fromunsloth/Z-Image-Turbo-ComfyUI, becauseblack-forest-labs/FLUX.1-schnellis gated for our token. It is the same AE.--diffusion-fa.Images are 1024x1024. Every arm repeated twice and is bit-identical to itself (A/A 0) on CPU and CUDA.
CUDA (B200, shared with other jobs)
--diffusion-fa --diffusion-conv-direct)--flash-attn off --conv-direct offvs master defaults--diffusion-fa)--conv-direct off)CPU (32 threads,
GGML_NATIVE=OFF+ all CPU variants, 512x512, 4 steps)Timing is from a host shared with other GPU and CPU jobs (Qwen-Image ranged 30 to 103 s for one arm), so it only supports "no slowdown". The branch was at or below master in every pair.
FLUX.3 (this branch only): the unsloth/flux-3-image-klein[-base]-GGUF denoisers (UD-Q4_K_XL, Q4_K_M, Q8_0, BF16) load as published. The VAE is
vae/diffusion_pytorch_model.safetensorsfrom the FP8 repos and the encoder is Qwen3-VL-4B-Instruct GGUF. At 1024, 16 steps on a B200, the distilled model takes about 9 to 11 s/img and the base (25 steps, CFG 3.5) 16 to 19 s/img. A reference edit with one-rimage works without the vision projector. CPU output matches CUDA (LPIPS 0.0008 to 0.006 for the same cell).