[Bug] Vulkan/AMD RX 6950 XT: Flux (split text encoders) silently crashes since master-848-9cdb6b6 — regression vs master-841-6b3edaa (weight preparation OOM)

[laurentvv]

Summary

Since release master-848-9cdb6b6, sd-cli (official bin-win-vulkan-x64 asset) crashes silently (process exits, no error message on stdout/stderr, bash reports exit code 127) when generating a Flux image with split text encoders on AMD Vulkan. The exact same command works on the previous release asset (master-841-6b3edaa). SDXL with a monolithic checkpoint works fine on the new build, so this looks related to the multi-model loading / weight-residency refactor (candidates: #1940, #1942, #1905).

Environment

• OS: Windows 11 (10.0.26200)
• GPU: AMD Radeon RX 6950 XT 16 GB (RDNA2 / gfx1030)
• Driver: 32.0.21045.5002 (Adrenalin)
• Build: sd-master-9cdb6b6-bin-win-vulkan-x64.zip (official release asset), commit 9cdb6b6
• Working build for comparison: sd-master-<sha>-bin-win-vulkan-x64.zip with commit 6b3edaa (master-841)

Models (GGUF from the project's recommended sources):

• flux1-schnell-Q6_K.gguf (diffusion model)
• clip_l.safetensors, t5xxl_fp16.safetensors (text encoders)
• ae.safetensors (VAE)

Reproduction

sd-cli.exe --diffusion-model flux1-schnell-Q6_K.gguf \
  --clip_l clip_l.safetensors --t5xxl t5xxl_fp16.safetensors --vae ae.safetensors \
  -p "a red apple on a wooden table, photo" \
  --steps 4 --cfg-scale 1.0 --sampling-method euler -W 512 -H 512 --seed 42 \
  --vae-tiling --backend vulkan -o out.png

Observed on 9cdb6b6 (master-848)

1) Default invocation — silent death. The process terminates without any error message (exit code 127 under Git Bash). Last log lines with -v:

[INFO   ] stable-diffusion.cpp:5322 - get_learned_condition completed, taking 13.73s
[INFO   ] stable-diffusion.cpp:5705 - generating image: 1/1 - seed 42
[INFO   ] ggml_graph_cut.cpp:965  - flux build cached graph cut plan done (taking 6 ms)
[VERBOSE] model_loader.cpp:1243 - loading tensors completed, taking 5.32s (read: 4.06s, ...)
[VERBOSE] model_manager.cpp:428  - model manager prepared params backend buffers (9379.29 MB, 776 tensors, 192 blocks, VRAM) on Vulkan0
<process exits here, no message>

2) --auto-fit off — identical silent crash at the same point.

3) --offload-to-cpu — hard Vulkan OOM error:

ggml_vulkan: vk::Device::createComputePipeline: ErrorOutOfDeviceMemory

4) --max-vram 6 — an actual error line is finally logged, then the process still dies silently:

[ERROR  ] ggml_runner.cpp:848  - t5 segment 17/26 (t5.block.15) failed during weight preparation
<process exits here, exit code 127, no additional message>

Expected / previous behavior

The identical command on the 6b3edaa build completes successfully and produces the image (this GPU is 16 GB; the previous build's streaming/offload behavior fits Flux Q6_K + t5xxl fp16 in VRAM).

Control test: SDXL is unaffected

A monolithic SDXL checkpoint (-m juggernautXL...safetensors, 512×512, euler_a) generates fine on 9cdb6b6 — so the regression is specific to the split-encoders path (diffusion model + separate clip/t5/vae), i.e. exactly what the new model manager / runner lifecycle code (#1940, #1942) and streamed-layer prefetch (#1905) touch.

Additional notes

• Two distinct problems, possibly: (a) the weight-preparation memory management regression itself, and (b) the process exits with no diagnostic at all in cases 1/2/4 — even when an [ERROR] line was just logged (case 4), there is no exception message, no abort message, nothing. A graceful failure with the underlying Vulkan error would have made this much easier to report.
• No other GPU-heavy process was running; no zombie sd-cli processes; VRAM was free before each run (each configuration was tested in a fresh process).
• Happy to test a fix build or additional variants (e.g. quantized t5xxl, flux1-dev, other --max-vram values) on this machine.

[laurentvv]

Data point: still present in master-849-d04e895 (official bin-win-vulkan-x64 asset).

Tested the exact same Flux schnell command (split text encoders, Vulkan, RX 6950 XT): identical silent crash, exit code 127 with no message, at the same point — right after:

[VERBOSE] model_manager.cpp:428  - model manager prepared params backend buffers (9379.29 MB, 776 tensors, 192 blocks, VRAM) on Vulkan0

Unsurprising since the only commit in 849 (#1939, SDXL embeddings loading) doesn't touch this path. 6b3edaa remains unaffected. I'll keep 6b3edaa installed and can test any candidate fix build on this machine (AMD RX 6950 XT 16 GB, Windows 11, Vulkan).

[vmobilis]

@laurentvv, hello, if nobody else will confirm the bug, there is a little possibility that the regression affects only your particular hw+drivers setup and others will not be able to reproduce the error.

If so, since some of 7 commits between 6b3edaa and 9cdb6b6 have binaries in releases, maybe you could try to locate the crashing commit more precisely, to clarify possible reason? (Or are they all crash right after https://github.com/leejet/stable-diffusion.cpp/releases/tag/master-841-6b3edaa)?
Good luck!

[laurentvv]

Follow-up per @vmobilis' suggestion — bisected further, the culprit is now isolated to a single commit.

Binary bisect (official bin-win-vulkan-x64 releases, same Flux schnell command as the original report):

Build	Commits added vs 6b3edaa	Result
master-841 6b3edaa	— (reference)	✅ image generated (re-verified 2026-09-08)
master-843 462d675	#1905 + #1940	❌ silent crash, exit 127
master-845 80bac2d	+ #1941 + #1942	❌ identical crash

Local builds to separate #1905 from #1940 (MSYS2/UCRT64, gcc 16.2, -DGGML_VULKAN=ON, same ggml submodule pin e20c3a14 for all three, same machine/command):

Commit	Result
6b3edaa (control)	✅ image generated
6c57cc3 (#1905 prefetch streamed layers, alone)	✅ image generated — #1905 is not the culprit
462d675 (#1940 unify runner lifecycles and weight residency)	❌ crash

The regression is introduced by 462d675 — PR #1940 "refactor: unify runner lifecycles and weight residency".

Extra detail that may help: with my gcc build the failure is not fully silent — the process aborts with a visible exception:

what():  vk::Queue::submit: ErrorOutOfDeviceMemory

It happens right after the DiT weights finish streaming to VRAM (last log line model_loader.cpp:1243 - loading tensors completed, i.e. the same stage as the model_manager.cpp:428 buffer-preparation line in the original report). So the ~9.4 GB of DiT buffers get allocated successfully, then the first compute submit runs out of device memory — consistent with the --offload-to-cpu / --max-vram behaviors reported earlier. It looks like the weight-residency refactor changed when (or whether) streamed buffers are released before compute on this 16 GB RDNA2 setup.

Happy to test any candidate fix build on this machine (AMD RX 6950 XT 16 GB, Windows 11, Vulkan, AMD proprietary driver).

[mlsterpr0]

master-841-6b3edaa seems to be last working vulkan release. After it, Minimax H3 crashes when loading text encoder.

[INFO ] ggml_graph_cut.cpp:952 - qwen3vl build cached graph cut plan done (taking 4 ms)
\stable-diffusion.cpp\ggml\src\ggml-backend.cpp:930: pre-allocated tensor (text_encoders.llm.model.embed_tokens.weight) in a buffer (Vulkan0) that cannot run the operation (NONE)