Repository navigation
Fix resample_sig for signals starting with NaN - #604
Merged
Merged
Conversation
pandas interpolate() only fills forward by default, so NaN values at the start of a signal were left in place and the FFT resample spread them over the whole output. Fill in both directions.
tompollard
reviewed
Oct 31, 2026
tompollard
left a comment
Member
There was a problem hiding this comment.
Thanks, this looks good! I checked that the leading-NaN test fails on main, that an all-NaN signal behaves as before, and that record 100 with its first 50 samples set to NaN now resamples to finite values.
One note for later, not for this PR: since the filled values are resampled like real data, the output has no NaN where the input was invalid. It might be worth marking those regions as NaN in the output at some point, but that's a separate change.
tompollard
approved these changes
Oct 31, 2026
tompollard
added a commit
that referenced
this pull request
Oct 31, 2026
Follow-up to @tompollard's note on #604. `resample_sig` fills NaN values before resampling, because the FFT would otherwise spread them over the whole signal. But the filled values were then returned as if they were real data, so the output had no trace of where the input was missing. Now output samples near a NaN input sample are set back to NaN. An output sample at position `t` (in input samples) is NaN if any input sample within half an output spacing of `t` is NaN (and within at least one input sample, so neighbours count when upsampling). The filling and resampling are unchanged, so samples away from the gaps keep the same values. For example, record 100 at 360 Hz with its first 50 samples set to NaN, resampled to 250 Hz, gives NaN for the first 35 output samples (positions below 50) and finite values after. Signals without NaN give the same output as before. This changes the output for signals with NaN, which is why it's a separate PR. The two tests from #604 now expect NaN at the filled positions. Tests: new tests check which output samples are NaN when upsampling, and when downsampling with a single NaN and with a gap between two output samples; they fail without the change. `pytest -m "not network"` passes (109 tests), and the docs build passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
resample_sigreturns only NaN values if the signal starts with NaN:NaN values are filled with
pd.Series.interpolate()before resampling (added for #249), but by default it only fills forward. Trailing NaN values take the last valid value, while leading ones stay NaN, and the FFT resample then spreads them over the whole output. This also affectsresample_singlechanandresample_multichan, for example on a record whose first samples are invalid.The fix passes
limit_direction="both", so leading NaN values take the first valid value. Inner and trailing NaN values are filled as before.Tests: a signal with leading NaN values gives the same result as the signal with them filled (this fails on
main), and a test for inner and trailing NaN values checks that they are still handled as before.pytest -m "not network"passes (98 tests), and the docs build passes.