Summary
On the default parakeet-tdt-0.6b-v3 batch path (melChunkContext = true), the final sliding window of a long (multi-window) utterance can decode to all-blank even though its audio contains clear speech — so the last several words are silently dropped from the transcript. It's intermittent and correlates with low input gain and a pause shortly before the dropped span. The same audio transcribes correctly when that tail is fed as a single window, or with melChunkContext = false, or when the input is amplified — which points at the 80 ms mel-context prepend (PR #264) degrading a short, low-amplitude final window's encoder features below the joint's blank threshold.
It's a silent failure (no error, high overall confidence), so it's easy to miss.
Environment
- FluidAudio 0.15.4 line,
AsrModelVersion.v3 (parakeet-tdt-0.6b-v3-coreml), Apple Neural Engine, macOS 14.
- Default
ASRConfig (melChunkContext = true, dualDecodeArbitration = false), batch AsrManager.transcribe.
- Observed via a fork that only adds an orthogonal encoder-feature accessor; the
ChunkProcessor / mel-context path is unchanged from upstream, so this should reproduce on stock main.
Symptom
For a ~44 s English dictation (16 kHz mono, peak ≈ 2.6 % FS — i.e. quiet), the transcript ends mid-sentence. The audio clearly continues for ~4.6 s after the last emitted token, with speech-level energy throughout (tail RMS ≈ the utterance's own speech RMS). result.confidence is ~0.98 and every emitted token is high-confidence — nothing signals the loss.
Root cause (per-frame decoder trace)
Instrumenting TdtDecoderV3.decodeWithTimings to log the joint decision at each frame shows the utterance split into 4 windows (stride ≈ 161 frames). The first three decode normally. The final window (isLastChunk = true, globalFrameOffset = 483, 77 valid frames ≈ 6.1 s of real speech) predicts blank (id 8192) at every frame:
ENTER isLast=true encSeq=77 effSeq=76 actualFrames=76 globalOff=483 t0=1
main t=1 tok=8192 p=0.924 dur=1 blank=true
blank t=8 tok=8192 p=1.000 dur=4 blank=true
blank t=12 tok=8192 p=1.000 dur=4 blank=true
blank t=16 tok=8192 p=1.000 dur=4 blank=true
... (every frame to t=76) ...
blank t=72 tok=8192 p=0.912 dur=4 blank=true
MAINLOOP-EXIT t=76 effSeq=76 emitted=0 lastTok=nil
FINALIZE begin t=76 -> recovers nothing
Key point: the decode loop does not terminate early — it walks all 76 frames and the joint calls every one silence at p≈1.0. So the finalization pass (isLastChunk) has nothing to flush; the encoder features for this window are already blank-like. Notably, the audio at the start of this window (its overlap region) was transcribed correctly by the previous window — the same samples read as speech there and as blank here. The only difference is that here they sit at the head of a short, heavily zero-padded 15 s encoder window with the mel-context prepend.
This looks like a new manifestation of the mel-context sensitivity already noted for #594 — but here it blanks the whole short final window rather than just drifting the first frames.
What recovers it (three independent signals)
- Isolated tail — feeding just the final ~6 s as its own clip (single-window path, no prepend) transcribes it perfectly, including a word the full run dropped.
melChunkContext = false — the no-mel/warmup path recovers the tail. (On a set of clean long-form clips it introduced no new tail drops.)- Amplifying the input — a moderate gain boost recovers the full transcript. (Not a robust fix on its own: recovery is non-monotonic in gain for some clips, and utterances with a loud word + quiet tail don't trigger a global-peak normalizer.)
Reproduction
The drop needs the real interaction of low gain + pause structure + final-window alignment, so a synthetic "just attenuate a clip" recipe is unreliable (mild attenuation → word errors; heavy → the whole utterance collapses to empty). It reproduces consistently on genuine quiet, long-form recordings. I can share a minimal anonymized sample WAV on request (the ones I have are user dictation I can't post publicly).
Conditions that trigger it:
melChunkContext = true (default), v3.- Audio long enough for ≥3 windows (roughly >30 s).
- Low input gain (peak on the order of a few % FS).
- A pause and/or the dropped span landing in a short final window.
Suggested direction
A targeted mitigation that's worked well for us: after the decode loop, if the final window emitted no tokens (or stopped well before the end of its own audio) and its untranscribed tail still carries speech-level energy, re-decode that window once with contextSamples = 0 and substitute only if it reaches further — the rescued tokens then flow through the existing overlap merge/dedup unchanged. It's ~90 lines in ChunkProcessor, additive, fires only on the failure signature, and costs one extra decode pass when it does. Happy to open it as a PR if useful — though a root-cause fix in how mel-context is applied to a short final window may well be cleaner than a re-decode, and you'll know that path better than I do.
Two things I'd want your read on either way: the heuristic thresholds (tail-gap and the speech-energy ratio), and behavior on very marginal tails where the re-decode can produce a plausible-but-uncertain clause.
Summary
On the default
parakeet-tdt-0.6b-v3batch path (melChunkContext = true), the final sliding window of a long (multi-window) utterance can decode to all-blank even though its audio contains clear speech — so the last several words are silently dropped from the transcript. It's intermittent and correlates with low input gain and a pause shortly before the dropped span. The same audio transcribes correctly when that tail is fed as a single window, or withmelChunkContext = false, or when the input is amplified — which points at the 80 ms mel-context prepend (PR #264) degrading a short, low-amplitude final window's encoder features below the joint's blank threshold.It's a silent failure (no error, high overall confidence), so it's easy to miss.
Environment
AsrModelVersion.v3(parakeet-tdt-0.6b-v3-coreml), Apple Neural Engine, macOS 14.ASRConfig(melChunkContext = true,dualDecodeArbitration = false), batchAsrManager.transcribe.ChunkProcessor/ mel-context path is unchanged from upstream, so this should reproduce on stockmain.Symptom
For a ~44 s English dictation (16 kHz mono, peak ≈ 2.6 % FS — i.e. quiet), the transcript ends mid-sentence. The audio clearly continues for ~4.6 s after the last emitted token, with speech-level energy throughout (tail RMS ≈ the utterance's own speech RMS).
result.confidenceis ~0.98 and every emitted token is high-confidence — nothing signals the loss.Root cause (per-frame decoder trace)
Instrumenting
TdtDecoderV3.decodeWithTimingsto log the joint decision at each frame shows the utterance split into 4 windows (stride ≈ 161 frames). The first three decode normally. The final window (isLastChunk = true,globalFrameOffset = 483, 77 valid frames ≈ 6.1 s of real speech) predicts blank (id 8192) at every frame:Key point: the decode loop does not terminate early — it walks all 76 frames and the joint calls every one silence at p≈1.0. So the finalization pass (
isLastChunk) has nothing to flush; the encoder features for this window are already blank-like. Notably, the audio at the start of this window (its overlap region) was transcribed correctly by the previous window — the same samples read as speech there and as blank here. The only difference is that here they sit at the head of a short, heavily zero-padded 15 s encoder window with the mel-context prepend.This looks like a new manifestation of the mel-context sensitivity already noted for #594 — but here it blanks the whole short final window rather than just drifting the first frames.
What recovers it (three independent signals)
melChunkContext = false— the no-mel/warmup path recovers the tail. (On a set of clean long-form clips it introduced no new tail drops.)Reproduction
The drop needs the real interaction of low gain + pause structure + final-window alignment, so a synthetic "just attenuate a clip" recipe is unreliable (mild attenuation → word errors; heavy → the whole utterance collapses to empty). It reproduces consistently on genuine quiet, long-form recordings. I can share a minimal anonymized sample WAV on request (the ones I have are user dictation I can't post publicly).
Conditions that trigger it:
melChunkContext = true(default), v3.Suggested direction
A targeted mitigation that's worked well for us: after the decode loop, if the final window emitted no tokens (or stopped well before the end of its own audio) and its untranscribed tail still carries speech-level energy, re-decode that window once with
contextSamples = 0and substitute only if it reaches further — the rescued tokens then flow through the existing overlap merge/dedup unchanged. It's ~90 lines inChunkProcessor, additive, fires only on the failure signature, and costs one extra decode pass when it does. Happy to open it as a PR if useful — though a root-cause fix in how mel-context is applied to a short final window may well be cleaner than a re-decode, and you'll know that path better than I do.Two things I'd want your read on either way: the heuristic thresholds (tail-gap and the speech-energy ratio), and behavior on very marginal tails where the re-decode can produce a plausible-but-uncertain clause.