Repository navigation
[Feature]Request: Add FP8 (e4m3) and fix Q6_K/Q8_0 Vulkan and Rocm10.0 support for MiniMax-H3 #1927
Description
Feature Summary
Q6_K/Q8_0 crash on Vulkan and Rocm10.0
Detailed Description
I'm running MiniMax-H3 video generation on AMD 7900 XTX (24GB VRAM) with stable-diffusion.cpp using Vulkan backend. Current setup works well with Q4_K_M quantization, but I'd like to use higher-quality or more efficient quantization formats.
Alternatives you considered
or q6.0 q8.0 gguf
Additional context
I'm running MiniMax-H3 video generation on AMD 7900 XTX (24GB VRAM) with stable-diffusion.cpp using Vulkan backend. Current setup works well with Q4_K_M quantization, but I'd like to use higher-quality or more efficient quantization formats.
Problem 1: Q6_K/Q8_0 crash on Vulkan
When selecting Q6_K or Q8_0 quantization, sd.cpp crashes with:
ggml/src/ggml.c:3301: GGML_ASSERT(!ggml_is_transposed(a)) failed
This happens in the Vulkan matmul kernel path. Q4_K_M works fine, suggesting the dequant+matmul pipeline for these quant types isn't fully implemented for Vulkan.
Test command:
sd-cli.exe -M vid_gen --diffusion-model minimax_h3_fl2va_pruned-Q6_K.gguf --llm qwen3vl_32b_minimax_h3-Q4_K_M.gguf --vae minimax_h3_video_vae_fp16.safetensors -p "test" -W 64 -H 64 -o out.png -vExpected: Model loads and runs (same as Q4_K_M)
Actual: Assertion failure during graph execution
Problem 2: No FP8 (e4m3) support
The official MiniMax-H3 models include FP8 e4m3 checkpoints:
minimax_h3_fl2va_pruned_fp8_e4m3fn.safetensorsminimax_h3_ref2va_pruned_fp8_e4m3fn.safetensors
FP8 would be ideal because:
- ~2x memory savings vs BF16
- Near-lossless quality compared to BF16
- Native hardware support on modern GPUs
GPU Details
GPU: AMD Radeon RX 7900 XTX (gfx1100, RDNA3)
Vulkan: 1.3.274
Driver: AMD proprietary (25.30.1)
VRAM: 24GB
Note: RDNA3 has cooperative matrix support but lacks VK_EXT_shader_float8 (that's RDNA4/gfx1200+). The portable e4m3 dequant-to-f16 path from ggml PR #1528 would work here.
Suggested Implementation
- Fix Q6_K/Q8_0 Vulkan matmul: Add proper transposed-tensor handling or route to CPU fallback
- Add FP8 e4m3 support: The ggml PR fix: avoid Vulkan f16 repeat in LTX audio VAE #1528 already implements this - verify sd.cpp properly wires it up for diffusion models
Reference
- ggml PR fix: avoid Vulkan f16 repeat in LTX audio VAE #1528: vulkan: add e4m3 (fp8) matmul for AMD RDNA4
- MiniMax-H3 FP8 discussion: Running MiniMax H3 (FL2VA) on 4x L40S with SGLang: dynamic FP8, ~2x speedup vs 2-GPU BF16, 14GB/card sgl-project/sglang#34079
Would be happy to help test any fixes!
Activity
[-][Feature]Request: Add FP8 (e4m3) and fix Q6_K/Q8_0 Vulkan support for MiniMax-H3[/-][+][Feature]Request: Add FP8 (e4m3) and fix Q6_K/Q8_0 Vulkan and Rocm10.0 support for MiniMax-H3[/+]farawayso commentedon Sep 1, 2026
Update: Q6_K/Q8_0 crash is FIXED in current master (6b3edaa) — remaining ask is FP8
I want to close the loop on Problem 1 of this report. Retesting on the same hardware (AMD 7900 XTX / gfx1100 / 24GB, Vulkan backend) against current master (
6b3edaa), the Q6_K/Q8_0 crash is no longer reproducible:GGML_ASSERT(!ggml_is_transposed(a))is gone. One important practical finding: pruned Q6_K (~16.5GB diffusion) + VAE (~5.5GB) +--eager-loadtogether can exceed the VRAM budget and cause heavy page-swapping, making it look like a hang. Adding--params-backend vae=cpu(keep VAE weights in system RAM, GPU only holds diffusion) restores full speed — ~1m45s for 480x864 / 39 frames / 4 steps, on par with Q4_K_M. So on 24GB cards, Q6_K is a fully viable quality upgrade over Q4_K_M.So the remaining actionable item from this issue is Problem 2: FP8 (e4m3) support — the official MiniMax-H3 ships FP8 e4m3fn checkpoints, and on RDNA3 (which has cooperative matrix support but no
VK_EXT_shader_float8), the portable e4m3→f16 dequant path would give meaningful memory savings without the Q8_0 VRAM pressure.Happy to help test the FP8 path once available.
GreenShadows commentedon Sep 8, 2026
RDNA 3 doesn't support FP8.
INT8 ConvRot is the best option for RDNA 3.