Environment
- Build:
audio-v0.7.1-bin-windows-x64-cuda13.3.zip (Windows 10 x64, CUDA 13.3 backend)
- Model:
higgs_audio_tts_4b_q8_0
- Commit analyzed:
a088f90
Symptom
When generating longer audio with Higgs Audio TTS, the output has an audible click/glitch roughly every 5 seconds (at ~5 s, ~10 s, ~15 s, ~20 s, ...), as if the waveform breaks at a seam. Short generations (≈10 s or less) sound fine.
Root cause
The official Python implementation decodes the whole sequence in one shot (HiggsAudioTokenizer.decode() in boson_multimodal/audio_processing/higgs_audio_tokenizer.py). src/models/higgs_audio_tts/codec.cpp adds chunked decoding (not present upstream) to bound memory:
constexpr int64_t kCodecDecodeWindowFrames = 128; // 128 frames = 128 × 960 / 24000 = 5.12 s
constexpr int64_t kCodecDecodeOverlapFrames = 8; // only 8 frames of left context, hard-trimmed concat
(codec.cpp:64-65; the chunk loop is around codec.cpp:1584. Frames ≤ 512 take the single-window path, which is why short audio is unaffected — and why the seams land at every 128 frames from sample 0, i.e. every 5.12 s.)
The problem: the overlap is far smaller than the decoder's receptive field. This DAC-style decoder is purely convolutional (5 × ConvTranspose blocks with residual units at dilations 1/3/9, all symmetric padding, i.e. non-causal). Walking the structure layer by layer:
| Item |
Value |
| Decoder total receptive field |
≈ 53 frames (≈ 27 frames per side ≈ 1.1 s of audio) |
| Left context provided by chunking |
8 frames |
| Missing right context at each window tail |
≈ 27 frames |
Since the decoder is non-causal, samples up to ~1 s past each trim point deviate from the full-sequence decode, and each window tail also lacks right context. The hard concat at seams then produces a waveform discontinuity — the periodic click.
Proposed fixes
Option A: widen the context window (recommended, minimal change)
Borrow 32 frames on both sides and trim both after decoding:
constexpr int64_t kCodecDecodeContextFrames = 32; // ≥ 27, covers the RF; aligns with the 32-frame graph capacity bucket
// window: [emit_start - 32, emit_end + 32), emit region still 128 frames
// trim 32 × 960 samples from each side before concatenating
Because the decoder is purely convolutional with no global ops (no attention/LSTM), once every emitted frame's full receptive field lies inside its window the chunked output is bit-identical to the full-sequence decode; the zero padding at the first window's left edge and the last window's right edge also matches the full decode exactly.
- Touches only the
decode_codes() chunk loop, and mirrors the pattern already used by qwen3_tts in this repo (kLeftContextCodes = 25 + drop in tokenizer_speech_decoder.cpp).
- Cost: ~1.4× per-window compute (192/136 frames). The codec decoder is a tiny fraction of the pipeline (AR generation dominates), so end-to-end impact is negligible.
Option B: stateful streaming chunks (more compute-efficient, much bigger change)
Carry the conv input history explicitly across chunks so each chunk computes only new frames:
-
The DAC decoder has 5 time scales (cumulative strides 3/6/24/120/960); each scale needs its own ring buffer of layer history, and the single HiggsCodecDecodeGraph would need to be split into a dozen-plus subgraphs with state passed between them.
-
Benefit: removes the redundant re-decode and naturally supports decode-while-generating streaming.
-
Caveat: the decoder is non-causal, so even with full layer state, streaming the tail ≈1.1 s of audio still has to wait for future code frames — i.e. this does not remove streaming latency; it only saves the re-decode compute (<1% end-to-end).
Suggestion
Ship Option A first (bit-exact, small diff, imperceptible cost). If decode-while-generating streaming is wanted later, it can be driven at the generator/session level per chunk (state = the last 32 code frames) without touching the codec internals; Option B can remain a perf optimization.
Environment
audio-v0.7.1-bin-windows-x64-cuda13.3.zip(Windows 10 x64, CUDA 13.3 backend)higgs_audio_tts_4b_q8_0a088f90Symptom
When generating longer audio with Higgs Audio TTS, the output has an audible click/glitch roughly every 5 seconds (at ~5 s, ~10 s, ~15 s, ~20 s, ...), as if the waveform breaks at a seam. Short generations (≈10 s or less) sound fine.
Root cause
The official Python implementation decodes the whole sequence in one shot (
HiggsAudioTokenizer.decode()inboson_multimodal/audio_processing/higgs_audio_tokenizer.py).src/models/higgs_audio_tts/codec.cppadds chunked decoding (not present upstream) to bound memory:(
codec.cpp:64-65; the chunk loop is aroundcodec.cpp:1584. Frames ≤ 512 take the single-window path, which is why short audio is unaffected — and why the seams land at every 128 frames from sample 0, i.e. every 5.12 s.)The problem: the overlap is far smaller than the decoder's receptive field. This DAC-style decoder is purely convolutional (5 × ConvTranspose blocks with residual units at dilations 1/3/9, all symmetric padding, i.e. non-causal). Walking the structure layer by layer:
Since the decoder is non-causal, samples up to ~1 s past each trim point deviate from the full-sequence decode, and each window tail also lacks right context. The hard concat at seams then produces a waveform discontinuity — the periodic click.
Proposed fixes
Option A: widen the context window (recommended, minimal change)
Borrow 32 frames on both sides and trim both after decoding:
Because the decoder is purely convolutional with no global ops (no attention/LSTM), once every emitted frame's full receptive field lies inside its window the chunked output is bit-identical to the full-sequence decode; the zero padding at the first window's left edge and the last window's right edge also matches the full decode exactly.
decode_codes()chunk loop, and mirrors the pattern already used by qwen3_tts in this repo (kLeftContextCodes = 25+ drop intokenizer_speech_decoder.cpp).Option B: stateful streaming chunks (more compute-efficient, much bigger change)
Carry the conv input history explicitly across chunks so each chunk computes only new frames:
The DAC decoder has 5 time scales (cumulative strides 3/6/24/120/960); each scale needs its own ring buffer of layer history, and the single
HiggsCodecDecodeGraphwould need to be split into a dozen-plus subgraphs with state passed between them.Benefit: removes the redundant re-decode and naturally supports decode-while-generating streaming.
Caveat: the decoder is non-causal, so even with full layer state, streaming the tail ≈1.1 s of audio still has to wait for future code frames — i.e. this does not remove streaming latency; it only saves the re-decode compute (<1% end-to-end).
Suggestion
Ship Option A first (bit-exact, small diff, imperceptible cost). If decode-while-generating streaming is wanted later, it can be driven at the generator/session level per chunk (state = the last 32 code frames) without touching the codec internals; Option B can remain a perf optimization.