Skip to content

feat(minimax-h3): load ComfyUI NVFP4 AWQ text encoder - #49

Draft
aegioscy wants to merge 1 commit into
2026-08-11from
feature/minimax-h3-nvfp4-awq
Draft

aegioscy wants to merge 1 commit into
2026-08-11from
feature/minimax-h3-nvfp4-awq

Conversation

@aegioscy

@aegioscy aegioscy commented Oct 2, 2026 •

Copy link
Copy Markdown

What changed

Load ComfyUI's MiniMax H3 NVFP4 AWQ Qwen3-VL text encoder directly in stable-diffusion.cpp. The safetensors reader recognizes packed NVFP4 weights and their FP8 block and F32 global scales; the loader repacks them into ggml's existing NVFP4 block layout. Linear layers apply the AWQ pre_quant_scale vectors before matrix multiplication and the global scale afterward.

The exact source diff was previously carried as a patch in QVAC's H3 work. This branch moves that diff into the owning backend repository. QVAC draft PR #4818 separately adds the optional download and pins this backend commit; the core T2V/I2V PR #4789 keeps the earlier merged backend pin. No ggml source change or dependency update is required; the pinned ggml commit already provides GGML_TYPE_NVFP4.

Why

Without the AWQ input scales, H3 generated a scene unrelated to the prompt. Applying them restored the requested objects and composition in matched runs.

Verification

  • The four-file source diff matches the QVAC overlay patch byte for byte.
  • The patched CUDA backend was built in an isolated checkout and generated 124-frame H3 T2V and I2V AVIs with the Comfy-Org NVFP4 encoder, 20 steps, seed 42, and 32 kHz stereo audio.
  • The new backend commit was SSH signed. QVAC follow-up PR #4818 pins its exact commit and source archive SHA-512.
  • No new build or test suite was run after moving the unchanged diff to this branch. The QVAC follow-up #5 vcpkg overlay has not yet been rebuilt end to end.

The checkpoint requests full-precision matrix multiplication in ComfyUI. This implementation uses ggml's native quantized NVFP4 multiplication, so numerical parity with ComfyUI is not claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant