feat(stems): DiCoSe stem splitter — engine + music-page split (by Hamza Qayyum)

.github/workflows/ci.yml

@@ -39,6 +39,21 @@ jobs:
       - run: npm run acestep:check
       - run: npm run acestep:test
 
+  # Vendored DiCoSe stem-separation runtime (packages/dicose): typecheck +
+  # weight-free vitest suite (the model-contract test self-excludes without
+  # the gitignored public/model package).
+  dicose:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+          cache: npm
+      - run: npm ci --ignore-scripts
+      - run: npm run dicose:check
+      - run: npm run dicose:test
+
   # Prettier — style is enforced, not suggested.
   format:
     runs-on: ubuntu-latest

README.md

@@ -105,6 +105,7 @@ demo page.
 | `eou-parakeet`           | Parakeet EOU 120M                 | **297×** browser-verified (1hr in 12.1s; worker-overlapped wasm decode + linear-cost stream-batch encode) | transcript + end-of-utterance events; TRUE streaming push()/finish() (bit-exact cache-carrying encode) + wasm-SIMD RNNT decode; whole-clip batch runs through the same linear-cost encoder |
 | `asr-voicechat`          | VoiceChat-11B STT (609M encoder)  | 34.6× (1hr file)                                                                                          | the speech-recognition slice of NVIDIA's full-duplex VoiceChat-11B; fully-causal per-frame streaming, parity byte-identical to the torch reference; weights hosted at [`FluidInference/fluidaudio-web`](https://huggingface.co/FluidInference/fluidaudio-web) like the other engines |
 | `musicgen-acestep`       | ACE-Step 1.5 Turbo (3.5B + VAE)   | ~1.9× (180s song in ~95s, M3, warm)                                                                       | full text-to-music on [`/music`](music.html): 8-step DiT + Oobleck VAE in pure WGSL (`packages/acestep`); ~5.7 GB one-time download; requires `shader-f16`; direct mode (optional planner LLM path exists upstream, still being optimized) |
+| `stem-dicose`            | DiCoSe stem separation (BS-RoFormer + 1-step CD) | ~3× fast mode (30s / 48 kHz song in ~9.8s, M5 Pro); refined ~0.45× | 5 stems — drums/bass/other/vocals + derived instrumental — on [`/analyze.html`](analyze.html) and as **Split stems** on [`/music`](music.html); vendored `packages/dicose` (DiCoSe.wgsl by Hamza Qayyum); 623 MB f16 weight package; requires `shader-f16` + fixed 32-wide subgroups; mix-reconstruction NRMSE 1.5% (4-stem sum) / 9e-5 (vocals + instrumental) |
 | `tts-voicechat`          | VoiceChat-11B TTS “Aria” (595M backbone + 159M MoG + 763M codec) | 23.7 GPU-ms per 80 ms frame + ~16 ms host (timestamp-query, M5 Pro dawn; est. ~1.6× in-browser) — node wall is poll-bound at 0.12× (dawn ~100 ms/sync × 5 syncs/frame); WASM 0.19× | the speech-decoder slice of NVIDIA's full-duplex VoiceChat-11B as a standalone TTS voice; GPU-resident decode loop (backbone/MoG-MLP batched submits, GPU KV caches, 5 readbacks/frame for the host-side f64 PRVQ decisions); audio codes bit-exact vs the torch reference ON BOTH BACKENDS (1550/1550), waveform NRMSE 1.1e-6; codec GPU decode ~24 GPU-ms/s of audio; local-only weights (`scripts/extract-voicechat-tts.py`, ~3.5 GB) — hidden from the picker unless exported |
 
 First (cold) run is several× slower — WebGPU compiles pipelines and weights
@@ -134,6 +135,13 @@ exact tuple). The optional 0.6B planner ("thinking") path is excluded from
 the served manifest until its pending optimization experiments
 (OPT-0084/0085/0087) are integrated.
 
+A finished song offers **Split stems**: DiCoSe (also by Hamza Qayyum,
+vendored at [`packages/dicose`](packages/dicose/)) separates the generated
+WAV into drums, bass, other, vocals, and a derived instrumental, right in
+the result panel — playable and downloadable per stem. Fast deterministic
+mode by default (~3× realtime); the 623 MB weight package downloads on
+first use and is released with the result panel.
+
 ## Text processing (WASM)
 
 [`text-processing-rs`](https://github.com/FluidInference/text-processing-rs)
@@ -155,6 +163,7 @@ npm run build      # static site → dist/
 npm run sdk:pack   # publishable SDK tarball (dist-sdk/ + .tgz in repo root)
 
 npm run acestep:check && npm run acestep:test   # ACE-Step runtime (packages/acestep) gates
+npm run dicose:check && npm run dicose:test     # DiCoSe runtime (packages/dicose) gates
 ```
 
 ## Deploy
@@ -182,6 +191,8 @@ src/
 packages/
   acestep/      vendored ace-step-1.5.wgsl music-gen runtime (own kernels,
                 scheduler, tests, and optimization ledger — see its AGENTS.md)
+  dicose/       vendored DiCoSe.wgsl stem-separation runtime (own kernels,
+                tests, and optimization ledger)
 scripts/        node gates: token-identity, kernel parity, per-engine smokes
 rust/           parakeet RNNT decoder + kernel lib sources (wasm32+simd128)
 docs/           architecture, benchmarks, PORTING.md (add-a-model checklist), the ORT removal story
@@ -235,5 +246,12 @@ over for integration here; we took it over, integrated, and are continuing
 the optimization work. The `packages/acestep` runtime and the `/music` page's
 backend seam are his code.
 
+Stem separation is likewise his: DiCoSe.wgsl (vendored at
+`packages/dicose`, MIT) ports DiCoSe — BS-RoFormer plus one-step
+consistency-distilled refinement, [karchkha/DiCoSe](https://huggingface.co/karchkha/DiCoSe)
+checkpoints — to raw WebGPU WGSL with its own correctness-audited kernel
+ledger, and powers both the `stem-dicose` engine and the `/music` page's
+Split stems feature.
+
 See [THIRD-PARTY-LICENSES.md](./THIRD-PARTY-LICENSES.md) for the full
 list of adapted code and licenses.

THIRD-PARTY-LICENSES.md

@@ -65,6 +65,25 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
 ```
 
+## DiCoSe.wgsl (MIT)
+
+Upstream repo currently ships no LICENSE file (its package.json declares
+MIT; the license text is pending from the author and will be added at
+`packages/dicose/LICENSE` when supplied). The complete vendored source is
+in this repository at `packages/dicose`.
+Copyright (c) 2026 Hamza Qayyum (Narcotic Software) — MIT License.
+
+The entire stem-separation runtime (`packages/dicose`) is a vendored
+import of DiCoSe.wgsl by **Hamza Qayyum**: a raw WebGPU WGSL port of
+DiCoSe (BS-RoFormer + one-step consistency-distilled refinement,
+arXiv 2412.06965) with its own correctness-audited optimization ledger
+(`packages/dicose/optimization/`). The `stem-dicose` engine and the
+`/music` page's Split stems feature are thin wrappers over his worker
+client. Model weights are converted from the
+[karchkha/DiCoSe](https://huggingface.co/karchkha/DiCoSe) checkpoints
+(MIT) via `packages/dicose/model/convert.py` and retain their upstream
+license.
+
 ## parakeet.js / ysdede (MIT)
 
 https://github.com/ysdede/parakeet.js — the NeMo log-mel preprocessor in

analyze.html

@@ -47,8 +47,8 @@ <h1 class="site-title">FluidAudio Web <span class="badge">Other Audio Models</sp
       </header>
 
       <p class="site-intro">
-        Voice activity detection and speaker diarization right here — WebGPU + WebAssembly, nothing leaves your machine. First load downloads the model
-        weights (cached after).
+        Voice activity detection, speaker diarization, and music stem separation right here — WebGPU + WebAssembly, nothing leaves your machine. First
+        load downloads the model weights (cached after).
         <span id="gpu" class="badge"></span>
       </p>
 

music.html

@@ -275,9 +275,26 @@ <h2 id="progress-title">Preparing</h2>
               <dd id="summary-time">—</dd>
             </div>
           </dl>
-          <a id="download" class="download-button" download="ace-step-song.wav">
-            Download WAV
-          </a>
+          <div class="result-actions">
+            <button id="split-stems" class="split-stems-button" type="button">
+              Split stems
+            </button>
+            <a id="download" class="download-button" download="ace-step-song.wav">
+              Download WAV
+            </a>
+          </div>
+        </div>
+        <div
+          id="stems-panel"
+          class="stems-panel"
+          aria-label="Separated stems"
+          hidden
+        >
+          <div class="stems-heading">
+            <h2>Stems</h2>
+            <span id="stems-time" aria-hidden="true"></span>
+          </div>
+          <ul id="stems-list" class="stems-list"></ul>
         </div>
       </section>
 

package.json

@@ -39,7 +39,10 @@
     "acestep:build": "npm run build --workspace ace-step-1.5.wgsl",
     "acestep:check": "npm run check --workspace ace-step-1.5.wgsl",
     "acestep:test": "npm run test --workspace ace-step-1.5.wgsl",
-    "build": "npm run acestep:build && tsc --noEmit && vite build && node scripts/postbuild-strip-wasm.mjs",
+    "dicose:build": "npm run build --workspace dicose-wgsl",
+    "dicose:check": "npm run check --workspace dicose-wgsl",
+    "dicose:test": "npm run test --workspace dicose-wgsl",
+    "build": "npm run acestep:build && npm run dicose:build && tsc --noEmit && vite build && node scripts/postbuild-strip-wasm.mjs",
     "preview": "vite preview",
     "typecheck": "tsc --noEmit",
     "gpu:verify": "node scripts/gpu-verify.mjs",
@@ -63,6 +66,7 @@
   },
   "dependencies": {
     "ace-step-1.5.wgsl": "*",
+    "dicose-wgsl": "*",
     "pinyin-pro": "^3.28.2"
   },
   "devDependencies": {

packages/dicose/.gitignore

@@ -0,0 +1,54 @@
+node_modules/
+dist/
+.vite/
+coverage/
+playwright-report/
+test-results/
+*.tsbuildinfo
+
+__pycache__/
+*.py[cod]
+.venv/
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+
+.model-cache/
+.upstream-dicose/
+model/cache/
+public/model/
+model/package/
+model/downloads/
+public/.model.staging-*/
+public/.model.previous-*/
+benchmark/results/
+fixtures/generated/
+*.ckpt
+*.safetensors
+*.bin
+*.onnx
+*.pt
+*.pth
+*.wav
+!Mixture_audio_1.wav
+!public/Mixture_audio_1.wav
+
+.env
+.env.*
+!.env.example
+!.env.*.example
+.dev.vars*
+!.dev.vars.example
+
+.pnpm-store/
+.cache/
+.eslintcache
+*.log
+
+.DS_Store
+._*
+.idea/
+.*.sw[op]
+*~