Row: MODEL-MM-dots3-note-dots3-note-for-causal-lm
W7c-2 is the SAMPLE-RATE half of W7c. It is deliberately separate from W7c-1 (channels,
#2813) because W7c-1 is an exact mirror and this is an unavoidable recorded divergence;
keeping them apart stops one claim borrowing the other's credibility.
Why this cannot be a mirror
Upstream's chain at 9035151d6: decode with soundfile/libsndfile, PyAV/ffmpeg as fallback,
loading at sr=None (vllm/multimodal/media/audio.py:278-284, :216-258); then resample in
the data parser, MultiModalDataParser(target_sr=..., target_channels=1)
(vllm/models/dots3_note/common/processor.py:523-525) via vllm/multimodal/parse.py:695.
AudioResampler's DEFAULT method is "pyav" = libswresample (vllm/multimodal/audio.py:283,
dispatch at :305-316).
libswresample is not bit-identical to itself. Measured locally with ffmpeg 6.1.1, the same
binary and the same input, differing only in CPU dispatch:
ffmpeg -i in.wav -af aresample=16000 -c:a pcm_f32le a.wav
ffmpeg -cpuflags 0 -i in.wav -af aresample=16000 -c:a pcm_f32le b.wav
-> identical: False ndiff 24691/32000 maxabs 9.686e-08
So a bit-exact gate against upstream's default is impossible in principle, not merely
inconvenient. Its option defaults also come from an unpinned linked binary (av appears only
in requirements/test/cpu.txt at the pin, a TEST lockfile, and PyAV bundles its own FFmpeg),
and the auto-resolved cutoff is not readable from outside the source.
This is the opposite of W6c's precedent. PIL's bicubic WAS portable because Resample.c is
~200 lines of documented filter maths over a fixed-point intermediate with exactly one answer,
and W6c gated it at 0 of 255 worst absolute difference. libswresample is float, SIMD-dispatched
and option-defaulted. W6c argues FOR porting a documented filter and AGAINST pretending
swresample is one.
The decision, already taken
Implement resample_audio_scipy (vllm/multimodal/audio.py:232-250 @ 9035151d6) and
refuse the pyav default permanently with the reason above.
This is not inventing a divergence — it is selecting another arm of upstream's own switch,
and vLLM already ships that arm in production for another model:
vllm/model_executor/models/phi4mm.py:580 @ 9035151d6 passes audio_resample_method="scipy".
The algorithm, stated exactly, because a divergence must name what it does:
scipy.signal.resample_poly at defaults over the gcd-reduced ratio — up = target//gcd,
down = orig//gcd, filter firwin(2*half_len+1, 1.0/max(up,down), window=("kaiser", 5.0))
with half_len = 10*max(up,down), scaled by up, applied by upfirdn with scipy's
n_pre_pad/n_pre_remove centring.
Measured distance from the real default, 44100 -> 16000, interior only: scipy 51.36 dB on
band-limited content, soxr 46.59, torchaudio 26.72. Record the 51.36 so nobody re-derives it.
Do NOT reuse Ltx2ResampleWaveform (#2583, ltx2_audio_vae.cpp:1151). It is a genuine
polyphase resampler and it is tempting, but it is ~25 dB FURTHER from this oracle on exactly
the band-limited content a speech encoder sees, because torchaudio's defaults are a short
kernel (lowpass_filter_width=6, hann) against swr's 32-tap kaiser-9. Say so in the spec so
the next reader does not "simplify" it back.
Gating, with no oracle
§6.4 option B holds. Follow #2583's design, which is the template:
commit scipy 1.17.1 goldens for 44100->16000, 48000->16000, 22050->16000, 8000->16000; gate
max|diff| at ~2x the measured float floor; and add a lower bound plus a difference
assertion, because a resampler that returns its input, or returns zeros, passes a
tolerance-only gate. Add a length assertion against ceil(n * new / orig).
scipy stands to the vLLM pin exactly as torchaudio stands to the ltx-2 pin in #2583:
resample_audio_scipy is vLLM's own code, so the precedent transfers.
Scope and seam
Build it as a shared vllm::multimodal resample seam, opted into per model, never blanket.
This matters: five rows already refuse a rate mismatch and they are not all the same policy —
Parakeet's refusal is upstream-faithful (feature_extraction_parakeet.py raises rather than
resampling), while dots3's upstream resamples. dots3 opts in. Leave
audio_processor.cpp's Whisper/Voxtral refusal untouched until each model's own upstream
policy is read; one row, one model's policy.
Reachability
Production entry point ApiServer::handle_chat_completions -> InstallMultiModalChatSeam ->
MakeDots3NoteChatSeam -> RouteDots3NoteAudioWav, plus
Dots3NoteAudioProcessor::ProcessWaveform.
Two existing refusal cases must INVERT, which is the ownership test:
test_api_server_dots3_mm_forward.cpp:1389 ("a 22050 Hz WAV names the resampler refusal and
W7c") and tests/vllm/models/test_dots3_note_audio.cpp:818. A served 44.1 kHz WAV must give
the same logprobs as the same audio pre-resampled offline by scipy.
Not in scope
Containers (#2814, not this row). W8, W9, W10, W11. #2616 and #2709 keep their remaining
blockers.
Row:
MODEL-MM-dots3-note-dots3-note-for-causal-lmW7c-2 is the SAMPLE-RATE half of W7c. It is deliberately separate from W7c-1 (channels,
#2813) because W7c-1 is an exact mirror and this is an unavoidable recorded divergence;
keeping them apart stops one claim borrowing the other's credibility.
Why this cannot be a mirror
Upstream's chain at
9035151d6: decode with soundfile/libsndfile, PyAV/ffmpeg as fallback,loading at
sr=None(vllm/multimodal/media/audio.py:278-284,:216-258); then resample inthe data parser,
MultiModalDataParser(target_sr=..., target_channels=1)(
vllm/models/dots3_note/common/processor.py:523-525) viavllm/multimodal/parse.py:695.AudioResampler's DEFAULT method is"pyav"= libswresample (vllm/multimodal/audio.py:283,dispatch at
:305-316).libswresample is not bit-identical to itself. Measured locally with ffmpeg 6.1.1, the same
binary and the same input, differing only in CPU dispatch:
So a bit-exact gate against upstream's default is impossible in principle, not merely
inconvenient. Its option defaults also come from an unpinned linked binary (
avappears onlyin
requirements/test/cpu.txtat the pin, a TEST lockfile, and PyAV bundles its own FFmpeg),and the auto-resolved
cutoffis not readable from outside the source.This is the opposite of W6c's precedent. PIL's bicubic WAS portable because
Resample.cis~200 lines of documented filter maths over a fixed-point intermediate with exactly one answer,
and W6c gated it at 0 of 255 worst absolute difference. libswresample is float, SIMD-dispatched
and option-defaulted. W6c argues FOR porting a documented filter and AGAINST pretending
swresample is one.
The decision, already taken
Implement
resample_audio_scipy(vllm/multimodal/audio.py:232-250 @ 9035151d6) andrefuse the
pyavdefault permanently with the reason above.This is not inventing a divergence — it is selecting another arm of upstream's own switch,
and vLLM already ships that arm in production for another model:
vllm/model_executor/models/phi4mm.py:580 @ 9035151d6passesaudio_resample_method="scipy".The algorithm, stated exactly, because a divergence must name what it does:
scipy.signal.resample_polyat defaults over the gcd-reduced ratio —up = target//gcd,down = orig//gcd, filterfirwin(2*half_len+1, 1.0/max(up,down), window=("kaiser", 5.0))with
half_len = 10*max(up,down), scaled byup, applied byupfirdnwith scipy'sn_pre_pad/n_pre_removecentring.Measured distance from the real default, 44100 -> 16000, interior only: scipy 51.36 dB on
band-limited content, soxr 46.59, torchaudio 26.72. Record the 51.36 so nobody re-derives it.
Do NOT reuse
Ltx2ResampleWaveform(#2583,ltx2_audio_vae.cpp:1151). It is a genuinepolyphase resampler and it is tempting, but it is ~25 dB FURTHER from this oracle on exactly
the band-limited content a speech encoder sees, because torchaudio's defaults are a short
kernel (
lowpass_filter_width=6, hann) against swr's 32-tap kaiser-9. Say so in the spec sothe next reader does not "simplify" it back.
Gating, with no oracle
§6.4 option B holds. Follow #2583's design, which is the template:
commit scipy 1.17.1 goldens for 44100->16000, 48000->16000, 22050->16000, 8000->16000; gate
max|diff|at ~2x the measured float floor; and add a lower bound plus a differenceassertion, because a resampler that returns its input, or returns zeros, passes a
tolerance-only gate. Add a length assertion against
ceil(n * new / orig).scipy stands to the vLLM pin exactly as torchaudio stands to the
ltx-2pin in #2583:resample_audio_scipyis vLLM's own code, so the precedent transfers.Scope and seam
Build it as a shared
vllm::multimodalresample seam, opted into per model, never blanket.This matters: five rows already refuse a rate mismatch and they are not all the same policy —
Parakeet's refusal is upstream-faithful (
feature_extraction_parakeet.pyraises rather thanresampling), while dots3's upstream resamples. dots3 opts in. Leave
audio_processor.cpp's Whisper/Voxtral refusal untouched until each model's own upstreampolicy is read; one row, one model's policy.
Reachability
Production entry point
ApiServer::handle_chat_completions->InstallMultiModalChatSeam->MakeDots3NoteChatSeam->RouteDots3NoteAudioWav, plusDots3NoteAudioProcessor::ProcessWaveform.Two existing refusal cases must INVERT, which is the ownership test:
test_api_server_dots3_mm_forward.cpp:1389("a 22050 Hz WAV names the resampler refusal andW7c") and
tests/vllm/models/test_dots3_note_audio.cpp:818. A served 44.1 kHz WAV must givethe same logprobs as the same audio pre-resampled offline by scipy.
Not in scope
Containers (#2814, not this row). W8, W9, W10, W11. #2616 and #2709 keep their remaining
blockers.