fix(cli,server): handle UTF-8 file paths on Windows by therealshulgin · Pull Request #2145 · leejet/stable-diffusion.cpp

therealshulgin · 2026-10-15T19:52:12Z

Companion of #2144 (the same fix in the library, for model files). The two PRs are independent and each builds on its own.

Problem

On Windows the CLI reads most options as UTF-8 (argv_to_utf8()), but:

outputs and previews are built with std::filesystem::path(std::string)
and written through path::string(), std::ofstream(std::string) and
stb without STBI_WINDOWS_UTF8/STBIW_WINDOWS_UTF8, all of which use the
ANSI code page. -o "D:\essai_été\café.png" logs
save result image 0 to '...\essai_été\café.png' (success) but the file
is written to ...\essai_Ã©tÃ©\cafÃ©.png;
-r, --ref-video, --ref-video-audio, --ref-audio, --audio,
--prompt-file, --negative-prompt-file and --imat-in take the raw
argv[index], which is in the ANSI code page. They work for characters
that code page can represent and fail for the others (-r "...\参照\写真.png" →
load image from '...\??\??.png' failed). They would also disagree with
every UTF-8 path once those are fixed.

Changes

Path-taking option callbacks use argv_to_utf8(); it is declared in
common.h so --imat-in in cli/main.cpp can use it too.
media_io.cpp defines STBI_WINDOWS_UTF8 and STBIW_WINDOWS_UTF8
(stb only acts on them under _WIN32).
Paths are built with fs::u8path(...) and turned back into strings
with .u8string() (output, preview, sidecar audio, video, image
directories, embeddings, LoRA and model directories, prompt files,
server model/LoRA/upscaler listings, serve_html_path).
A LoRA name taken from the prompt text that is not valid UTF-8 (a prompt
file in another encoding) is reported and skipped instead of throwing.

No change on Linux/macOS.

Verification

Windows 11 (ANSI code page 1252), MSVC 14.44, -DSD_CUDA=OFF, master
8cd731e, FLUX.2 klein 4B Q4_0 (256x256, 4 steps, seed 1):

case	before	after
`-o ...\essai_été\café.png`	written to `essai_Ã©tÃ©\cafÃ©.png`	written to `essai_été\café.png`
`-o ...\東京\画像.png`	written to a mangled name	written to `東京\画像.png`
`-r ...\参照\写真.png`	`load image ... '??\??.png' failed`, exit 1	reference loaded
`-r ...\entrée\référence.png`	loaded (cp1252 covers é)	loaded
ASCII `-o ...\plain\coffee.png`	written	written, identical image

🤖 Generated with Claude Code

Summary by CodeRabbit

Bug Fixes
- Improved handling of non-ASCII file and directory names across command-line tools and the server, including model, image, audio, and HTML paths.

Outputs and previews were built with std::filesystem::path(std::string) and written through path::string(), std::ofstream(std::string) and stb without STBI_WINDOWS_UTF8, which all use the ANSI code page: "café.png" was written as "cafÃ©.png". -r, --ref-video, --ref-video-audio, --ref-audio, --audio, --prompt-file, --negative-prompt-file and --imat-in took the raw argv, also in the ANSI code page, so they failed on characters it cannot represent. Read those options with argv_to_utf8 (now declared in common.h), define STBI_WINDOWS_UTF8 and STBIW_WINDOWS_UTF8, and build and print paths with fs::u8path and path::u8string. A LoRA name from the prompt that is not valid UTF-8 is reported and skipped instead of throwing.

coderabbitai · 2026-10-15T19:52:40Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI
Review profile: CHILL
Plan: Advanced
Run ID: e9533e7d-d076-4e82-b9c8-ef8f8f301d60

📥 Commits

Reviewing files that changed from the base of the PR and between 8cd731e and efc85fb.

📒 Files selected for processing: 9

examples/cli/image_metadata.cpp
examples/cli/main.cpp
examples/common/common.cpp
examples/common/common.h
examples/common/media_io.cpp
examples/server/routes_index.cpp
examples/server/routes_sdapi.cpp
examples/server/routes_sdcpp.cpp
examples/server/runtime.cpp

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

📝 Walkthrough

Filesystem path handling across the example CLI, shared code, media I/O, and server now uses UTF-8 conversions. The shared code also exposes argv_to_utf8 and applies it to selected command-line arguments.

Changes: UTF-8 filesystem path handling

Layer / File(s)	Summary
Shared argument and resource paths `examples/common/common.h`, `examples/common/common.cpp`	The shared code exposes `argv_to_utf8` and uses UTF-8 conversions for command-line inputs, prompt files, embeddings, model resolution, and LoRA paths. Invalid LoRA path conversion now warns, removes the tag from the prompt, and marks a missing LoRA.
Media file path handling `examples/common/media_io.cpp`, `examples/cli/image_metadata.cpp`	Image, binary, and WAV file operations construct paths with UTF-8 conversions. Image extension parsing returns a UTF-8 string.
CLI input and output paths `examples/cli/main.cpp`	CLI image loading, preview and result output, sidecar handling, and imatrix checks use UTF-8 path conversions.
Server path handling `examples/server/routes_index.cpp`, `examples/server/routes_sdapi.cpp`, `examples/server/routes_sdcpp.cpp`, `examples/server/runtime.cpp`	Server routes and runtime checks construct configured paths with UTF-8 conversions and serialize selected model paths as UTF-8 strings.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Suggested reviewers: leejet

Merge Risk: ⚪ Minimal · up to efc85

This change makes non-ASCII file paths work on Windows and is not expected to change behavior on Linux or macOS. No merge-blocking risk was identified from the supplied context.

Security architecture risk: 🔵 Low · up to efc85

The inspected changes correct Windows path encoding without showing broader filesystem authority or weaker access controls. End-to-end Windows model-file compatibility remains incompletely established.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

inferred — The demonstrated exposure remains filesystem access under the existing process authority and existing HTTP responses. Correctly resolving non-ASCII names changes path identity, but the inspected changes do not add credentials, privileges, or request-controlled file-selection channels.

Trust Boundaries and Controls

observed — The changed CLI callback preserves argument bounds checking and the same destination vector. UTF-8 conversion occurs before the existing consumers rather than bypassing an input-validation or ownership boundary.

Resilience and Maintainability Implications

observed — Malformed LoRA path conversion is caught before map insertion: the tag is removed and extraction reports a missing entry. Valid entries continue to aggregate under map-owned normalized UTF-8 keys.
observed — Configured cache-directory conversion occurs before the local scan exception boundary and before shared-cache replacement. In the HTTP framework's exception-enabled branch, exceptions escaping route execution are caught and produce an error response rather than escaping request dispatch.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 3.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 9 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly identifies the main change: UTF-8 file path handling for the CLI and server on Windows.
Description check		The description explains the problem, implementation, related issue, platform scope, and verification results. It does not include the required checklist section, but the description is otherwise subs…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

laurentvv · 2026-10-16T22:09:39Z

Verified on the official master-970 release binaries (2f016e8) — this PR and companion #2144. Thank you.

Datapoint (Windows 11, ANSI code page 1252, official sd-master-*-bin-win-vulkan-x64.zip builds, SDXL 512×512 20 steps cfg 7 seed 42, same-session A/B):

Before (8cd731e): -o into a directory with accented characters is worse than a clean failure — it logs success and exits 0 while the image lands somewhere else entirely:

[I] save result image 0 to '…\test_modèles_éà\out.png' (success)
[I] 1/1 images saved

exit 0, the real test_modèles_éà directory stays empty, and the PNG is actually written to a freshly created mojibake directory test_modÃ¨les_Ã©Ã (the CP1252 re-decode of the UTF-8 bytes — its name even ends with the U+00A0 that the Ã sequence decodes to). Matches the "before" row of this PR's verification table. The 0 exit code means wrapping scripts have no way to detect the miss.

After (2f016e8): the same command writes into the real directory, valid PNG, and the image is pixel-identical to an ASCII-output run at the same seed.

Companion #2144 validated in the same run (accented model path now loads).

…to_claude Brings in the 20 upstream commits since 2988060 (2026-10-11): per-module automatic flash attention (leejet#2132), automatic direct convolution (leejet#2134, leejet#2141), reclaim before trusting wrapped Vulkan iGPU memory estimates (leejet#2124), UTF-8 file paths on Windows (leejet#2144, leejet#2145), FP8 CPU crash fix (leejet#2140), the ggml sync to dc99877 (leejet#2147), FLUX.3 Klein, MOSS-Image, the MoSR upscaler, MiniMax-H3 LoRAs, Qwen Image 2.1 Turbo preset, ROCm and MUSA Dockerfiles, documentation. Conflicts resolved, keeping both sides: - include/stable-diffusion.h: upstream's flash_attn_mode and conv_direct_mode stay right after conditioning_cache_size, so the layout up to there is still upstream's; memory_guard, disable_backend_fallback and multi_gpu follow. - src/core/ggml_extend_backend.cpp: upstream's new cuda_compiled_arch() keeps asking the compute capability through sd::cuda_driver (the driver is looked up at run time, so the engine starts where there is no NVIDIA driver), not through <cuda.h>; the HIP include from upstream is kept. - src/model_manager.cpp: the CPU memory guard and upstream's requires_reclamation rule are both applied: a Vulkan iGPU estimate that wrapped is reclaimed before it is trusted, and the guard still decides for the CPU backend. - src/model_loader.cpp: includes of both sides; load_worker keeps the file handle pool and the open retry. - examples/server/runtime.cpp: --degradation-cooldown check and upstream's u8path for the --serve-html-path check. - docs/backend.md: the "Several GPUs" section and upstream's "Direct convolution" and "Flash attention" sections. The ggml submodule moves from d25b121 to dc99877 (upstream's pin). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01EhZzDBkuuMp1jBf4WtEySC

therealshulgin mentioned this pull request Oct 15, 2026

fix: open model files with UTF-8 paths on Windows #2144

Merged

leejet merged commit 2f016e8 into leejet:master Oct 16, 2026
2 checks passed

laurentvv mentioned this pull request Oct 16, 2026

Video regressions on 16 GB Vulkan: LTX-2.5 + MiniMax-H3 broken since master-864, Wan 2.2 broken by master-866 (graph segmentation / weight-budget behavior) — --auto-fit off only rescues Wan #1976

Open

leehack mentioned this pull request Oct 17, 2026

Qualify post-v0.2 upstream changes with context ABI and FP8 CPU safety checks leehack/stable-diffusion-native#20

Open

6 tasks

nibor1896 mentioned this pull request Oct 18, 2026

sd-server pin predates Ming-Image support nibor1896/Crow#402

Closed

Navigation Menu

Repository navigation

fix(cli,server): handle UTF-8 file paths on Windows - #2145

fix(cli,server): handle UTF-8 file paths on Windows#2145
leejet merged 1 commit into
leejet:masterfrom
therealshulgin:fix/windows-utf8-cli-paths

therealshulgin commented Oct 15, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

laurentvv commented Oct 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

therealshulgin commented Oct 15, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

laurentvv commented Oct 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

therealshulgin commented Oct 15, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Oct 15, 2026 •

edited

Loading