comfyui: pin the application version and bump to v0.33.1 for MiniMax Music3

[AlienWalker1995]

Why

Installing MiniMax Music3 requires ComfyUI >= v0.33.1 (the nodes landed upstream in efd4e951a0); the stack was on v0.30.2. But the bump surfaced a bigger problem worth fixing first.

The pinned COMFYUI_IMAGE digest only ever pinned the runtime — torch, CUDA, the boot entrypoint. ComfyUI itself lives on the comfyui-app named volume (moved there in #156 to escape the 9p pathology), so the image says nothing about which ComfyUI is actually running. The answer was "whatever was last checked out by hand on the volume": unpinned, undeclared, unreproducible. Hand-checking-out a second version would have repeated the workaround instead of fixing it.

What changed

The application version is now declared and enforced

• COMFYUI_APP_REF in services/comfyui/plugin.yaml declares it as a full commit SHA — 72865f4f (v0.33.1). Never a tag or branch; tags are movable refs.
• scripts/comfyui/boot.sh replaces the 20-line inline compose command: blob and runs as the container command (same pattern as scripts/llamacpp/run-llama-server.sh). On every start it reconciles the checkout to the declared ref. If the volume has drifted it fetches and checks out; if it cannot, it refuses to boot rather than serve a version nobody declared. Steady-state boots touch the network zero times.
• The Comfy-Org runtime pins (comfyui-frontend-package, comfy-kitchen, comfy-aimdo, comfyui-workflow-templates, comfyui-embedded-docs) are read out of the pinned ref's own requirements.txt rather than restated in our config — one source of truth, so the app and its runtime deps cannot drift apart. Bumping COMFYUI_APP_REF automatically carries the right deps.
• Those installs now target the user site (/root/.local), which outranks both /usr/local/lib and /usr/local/lib64 on sys.path. That ends the shadowing class that silently broke every fp8/nvfp4 load on 2026-08-07, instead of playing whack-a-mole with rm -rf on whichever copy won. Versions are verified after install, not assumed.

Music3

• scripts/comfyui/models.json gains a minimax-music3 pack with URLs pinned to the Comfy-Org repack revision 6444666e (not /resolve/main/, which is a moving ref). This replaces an untracked throwaway dl_music3.sh that lived on the volume.
• scripts/comfyui/workflows/minimax-music3.json is the committed workflow (API format, so it is both the UI starting point and what automation submits). boot.sh seeds it into a repo-owned user/default/workflows/ordo/ namespace — operator-authored graphs in sibling dirs are never touched.
• services/song-gen/plugin.yaml declares both backends as data, each with the ComfyUI version it needs. Its VRAM floor rises 20 → 24 GB (Music3 is the larger backend).

Vocabulary

tests/test_dashboard_comfyui_packs.py gains audio to the pack-capability set. That set is the vocabulary — the dashboard passes the field through verbatim — and a song-generation pack would otherwise have to lie about itself as other.

Validation (live, on the deployed stack)

Check	Result
Declared == running	image digest sha256:4172d960…, COMFYUI_APP_REF, and git rev-parse HEAD all agree on 72865f4f
Version endpoint	/system_stats → comfyui_version: 0.33.1, torch 2.11.0+cu128, device RTX 5090
Existing pipeline	fp8 Flux-schnell render through the gate → PNG, 37.6s (exercises the comfy-kitchen 0.2.31 quant runtime — the highest-risk dep in the bump)
Music3	30s track through the gate → mp3, 55.3s. Decoded: 29.99s stereo @ 44.1kHz, RMS 0.123, 97% non-silent, natural spectral rolloff
GPU arbitration	gate GRANTED (31.8GB) → llamacpp evicted → failover to llamacpp-cpu → RELEASED after 60s drain → llamacpp + llamacpp-embed restored to the 5090, /health ok
Steady-state boot	restart logs ComfyUI already at declared ref + runtime deps already reconciled — a clean no-op
Post-recreate	7 tailnet sidecars healthy, caddy :443 and :8443-8449 bound, edge returns 302 (SSO)
Tests	625 passed; the 5 failures are pre-existing on main and unrelated (3 from an uncommitted local catalog/models.yaml, 2 hermes-socket)

Validation renders went through comfyui-gate rather than holding an own lease and posting to :8188, which avoids the known bypass-alarm seam entirely.

Rollback

Set COMFYUI_APP_REF back to dec5d9450a5290bcf63430409ea41018e67f41c3 (v0.30.2), re-render, recreate. The reconciler downgrades the checkout and re-pins the runtime deps from the older ref's requirements.txt. No manual cleanup.

Cleanup

Orphaned music3-app / music3-out volumes from the throwaway container are removed. music3-out held 52 real generated songs — checksum-verified into data/comfyui-output/music3-2026-08-14/ before deletion.

🤖 Generated with Claude Code