Skip to content

Use NumPy for the digital range check in check_sig_cohesion - #610

Merged
tompollard merged 1 commit into
MIT-LCP:mainfrom
TheNetherWatcher:faster-sig-cohesion
Oct 31, 2026
Merged

tompollard merged 1 commit into
MIT-LCP:mainfrom
TheNetherWatcher:faster-sig-cohesion

Conversation

@TheNetherWatcher

@TheNetherWatcher TheNetherWatcher commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

check_sig_cohesion finds the minimum and maximum of each channel to check that the digital samples fit the format. It used Python's built-in min and max, which go through a NumPy array one element at a time, so this check took about 85% of the time of wrsamp for large records. This uses np.min and np.max instead, in both the single-frequency and the expanded (multi-frequency) path.

Time for a 4-hour, 8-channel record at 500 Hz (57.6 million samples), written from digital samples:

Format check_sig_cohesion before → after wrsamp before → after
16 2.41 → 0.07 s 2.85 → 0.45 s
212 2.41 → 0.08 s 3.07 → 0.78 s

The written files are unchanged. There were no tests for this check, so I added TestSigCohesion: a value just outside the range in the last sample is rejected (in both paths), and the smallest and largest values of each format are allowed. These pass on main as well, and fail if the check skips the last sample.

Related: #607 speeds up the byte packing in the same wrsamp call. The two don't touch the same lines, apart from both adding a changelog entry and a test class at the end of the file.

pytest -m "not network" passes (101 tests), and the docs build passes.

@tompollard tompollard left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good, and it's nice to have tests for this check now! For a 1-hour, 8-channel record the check went from 0.61 s to 0.02 s on my machine.

@TheNetherWatcher
TheNetherWatcher force-pushed the faster-sig-cohesion branch 2 times, most recently from fc3e22d to 1b20ac2 Compare October 31, 2026 18:42
The minimum and maximum of each channel were found with Python's
built-in min and max, which iterate over the array one element at a
time. This took about 85% of the time of wrsamp for large records. Use
np.min and np.max instead.
@tompollard
tompollard merged commit a141be1 into MIT-LCP:main Oct 31, 2026
18 checks passed
@TheNetherWatcher
TheNetherWatcher deleted the faster-sig-cohesion branch November 1, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants