audio: fix chunk splices, output delay and level policy in the enhancers - #360
Closed
CryptVenture wants to merge 1 commit into
Closed
audio: fix chunk splices, output delay and level policy in the enhancers#360CryptVenture wants to merge 1 commit into
CryptVenture wants to merge 1 commit into
Conversation
Three independent defects in the arithmetic around the enhancement models, not in the models themselves. Each is reachable in its previous form through a new options struct, so existing call sites are unchanged unless they ask otherwise. ZipEnhancer spliced its segmented output with std::copy every 1.5 s and left up to 250 ms of digital silence on the tail for roughly one input length in six -- the lengths where (n - 32000) % 24000 == 0. Replaced with the accumulate-and-normalise overlap-add that deepfilternet2.cpp already uses. Accumulated weight is now exactly 1.0 on every returned sample and the output length matches the input. Measured on Metal with a stride-aligned 59 s input: before trailing silence 4000 samples (250 ms) after trailing silence 0 Envelope-step attenuation against the hard splice, whose step is 1.0: 38.17 dB at a 5 ms fade, 50.13 dB at 20 ms, 78.06 dB at the shipped 500 ms default. A linear, amplitude-complementary fade is correct here and equal-power is not: ZipEnhancer's overlapping chunks see the same input samples through a deterministic model, so the two sides of a join are strongly correlated and equal-power would put a +3 dB bump at every seam. RNNoise left its 20 ms synthesis delay in the output, so every processed file was time-shifted 960 samples against its input -- which matters for any A/B, any mix against the original, and any downstream alignment. deepfilternet2.cpp already crops its own 480-sample delay; RNNoise now pads the tail and crops the front the same way. An impulse in the final 20 ms of input previously vanished from the output entirely; the test asserts it now survives at its input index with zero stray energy. FlashSR peak-normalised every output to 0.999, so one loud sample rewrote the level of the whole file and the input's own level was discarded -- a -40 dBFS whisper and a -3 dBFS shout both came back at -0.009 dBFS. The ceiling is now a limit rather than a target and the input peak is restored: a 37 dB input swing survives as a 37 dB gain swing, where the old policy flattened five test cases onto 0.999. The reference fixtures under tests/unittests/assets were captured from the upstream implementations, which carry the delay and the normalisation, so test_rnnoise_utility and test_flashsr_utility now ask for legacy behaviour explicitly and remain parity tests. Tests: tests/unittests/test_enhancement_alignment.cpp -- 8 cases, no model weights needed. Sweeps 44 input lengths around the tail case and asserts overlap-add weight is exactly 1.0 with no trailing zero run, carries the old give-up coverage as a regression witness, checks the fade envelope is monotonic and complementary to 1e-6, and pins the RNNoise crop arithmetic and the FlashSR gain policy. Build: cmake -S . -B build -DENGINE_BUILD_TESTS=ON && cmake --build build Test: ctest -R enhancement_alignment_test Backend tested: Metal (audiocpp_enhance --backend metal --denoise zipenhancer) and CPU; full suite 39/39. Known limitation: ZipEnhancer's per-block RMS normalisation is still per-block, so the two sides of a join are normalised independently. The crossfade now smears the resulting level step over the overlap instead of concentrating it in one sample, but making the gain continuous across chunks is a redesign of the normalisation and wants a listening test first. FlashSR still measures its peak over the whole file and so cannot stream. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ATa5YkLUPMDPRL7w1gCo9p
0xShug0
marked this pull request as draft
August 31, 2026 00:37
0xShug0
marked this pull request as ready for review
September 1, 2026 16:48
Contributor
Author
|
Closing for now to stay inside the 3-concurrent-PR policy (see the discussion on #422). This one has also gone stale against main; I will rebase it and resubmit once a review slot is free. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Three independent defects in the arithmetic around the enhancement models, not in the models themselves. Each is reachable in its previous form through a new options struct, so existing call sites are unchanged unless they ask otherwise.
ZipEnhancer: hard splices and a 250 ms tail hole
It spliced its segmented output with
std::copyevery 1.5 s, and left up to 250 ms of digital silence on the tail for roughly one input length in six — the lengths where(n - 32000) % 24000 == 0.Replaced with the accumulate-and-normalise overlap-add that
deepfilternet2.cppalready uses. Accumulated weight is now exactly 1.0 on every returned sample and the output length matches the input.Measured on Metal with a stride-aligned 59 s input:
Envelope-step attenuation against the hard splice, whose step is 1.0: 38.17 dB at a 5 ms fade, 50.13 dB at 20 ms, 78.06 dB at the shipped 500 ms default.
A linear, amplitude-complementary fade is correct here and equal-power is not: ZipEnhancer's overlapping chunks see the same input samples through a deterministic model, so the two sides of a join are strongly correlated and equal-power would put a +3 dB bump at every seam.
RNNoise: 20 ms of uncompensated output delay
Every processed file was time-shifted 960 samples against its input — which matters for any A/B, any mix against the original, and any downstream alignment.
deepfilternet2.cppalready crops its own 480-sample delay; RNNoise did not.It now pads the tail and crops the front the same way. An impulse in the final 20 ms of input previously vanished from the output entirely; the test asserts it now survives at its input index with zero stray energy.
FlashSR: peak normalisation destroyed the input level
It normalised every output to 0.999, so one loud sample rewrote the level of the whole file — a −40 dBFS whisper and a −3 dBFS shout both came back at −0.009 dBFS.
The ceiling is now a limit rather than a target and the input peak is restored: a 37 dB input swing survives as a 37 dB gain swing, where the old policy flattened five test cases onto 0.999.
Note on the reference fixtures
The fixtures under
tests/unittests/assetswere captured from the upstream implementations, which carry the delay and the normalisation.test_rnnoise_utilityandtest_flashsr_utilitynow ask for legacy behaviour explicitly, so they remain parity tests. No fixture was regenerated.Validation
Build
Test
tests/unittests/test_enhancement_alignment.cpp— 8 cases, no model weights needed. Sweeps 44 input lengths around the tail case and asserts overlap-add weight is exactly 1.0 with no trailing zero run, carries the old give-up coverage as a regression witness so the bug is proven to have existed, checks the fade envelope is monotonic and complementary to 1e-6, and pins the RNNoise crop arithmetic and the FlashSR gain policy.Run command used for the Metal A/B
Backend tested: Metal and CPU. Full suite 39/39 on macOS/Metal, Apple M4 Max.
Affects
Output changes for ZipEnhancer's segmented path (>6 s input) — that is the fix. The un-segmented path (2–6 s) returns up to 99 more samples; the samples the old path returned are bit-identical and the new ones are the genuine tail. RNNoise output shifts 20 ms earlier. FlashSR output level changes for every file, so anything downstream that assumed near-full-scale FlashSR output will now be quieter by however much headroom the input had.
Known limitations
🤖 Generated with Claude Code
https://claude.ai/code/session_01ATa5YkLUPMDPRL7w1gCo9p