Skip to content

Keep invalid samples invalid in smooth_frames - #608

Merged
tompollard merged 1 commit into
MIT-LCP:mainfrom
TheNetherWatcher:smooth-frames-invalid
Oct 31, 2026
Merged

tompollard merged 1 commit into
MIT-LCP:mainfrom
TheNetherWatcher:smooth-frames-invalid

Conversation

@TheNetherWatcher

Copy link
Copy Markdown
Contributor

When a signal has more than one sample per frame, smooth_frames averages each frame in digital units. It doesn't check for the invalid sample value, so an invalid sample is averaged with the others and gives a large value that looks valid:

import numpy as np, wfdb

# fmt 16, 2 samples per frame, gain 200. -32768 is the invalid sample value.
np.array([0, 1, -1, -2, -32768, 100], dtype="<i2").tofile("r.dat")
open("r.hea", "w").write("r 1 100 3\nr.dat 16x2 200/mV 16 0 0 0 0 sig\n")

wfdb.rdrecord("r", smooth_frames=False).e_p_signal[0]
# [ 0.     0.005 -0.005 -0.01     nan  0.5  ]
wfdb.rdrecord("r").p_signal[:, 0]
# [ 0.    -0.005 -81.67]   (last frame should be NaN)

The WFDB library returns an invalid value for any frame that contains an invalid sample (meansamp() in lib/signal.c), so this does the same: after averaging a chunk of frames in digital units, frames with an invalid sample are set to the invalid value for the channel's format. That then becomes NaN when converted to physical units. Physical smoothing (sigtype="physical") already gave NaN, since NaN propagates through the sum.

This doesn't change the rounding of valid frames, which is #335.

Tests: a fmt 16 record with two samples per frame and one invalid sample reads back as [3, -32768, 7] (digital) and [0.015, nan, 0.035] (physical). This fails on main. pytest -m "not network" passes (97 tests), and the docs build passes.

@tompollard tompollard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks great!

I compared it against rdsamp from the WFDB C library (frame-averaged output) on the multi-frequency sample records that have invalid samples:

  • wave_4: rdsamp gives 3659 invalid frames. With this PR the Python reader gives the same 3659, at the same positions. On main it gave 7.
  • 041s02: one invalid frame in both, at the same position. On main there were none.

All other multi-frequency records in sample-data read exactly as before.

@TheNetherWatcher
TheNetherWatcher force-pushed the smooth-frames-invalid branch 2 times, most recently from 2f2a81f to dfd0a25 Compare October 31, 2026 18:41
smooth_frames averaged the invalid sample value with the other samples
in a frame, which gave a large valid-looking value instead of an invalid
one (NaN in physical units). As in the WFDB library's meansamp(), a
frame that contains an invalid sample now gives the invalid value.
@tompollard
tompollard merged commit cd242ef into MIT-LCP:main Oct 31, 2026
18 checks passed
@TheNetherWatcher
TheNetherWatcher deleted the smooth-frames-invalid branch November 1, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants