[jesshaley]

Summary

This PR adds ndr.reader.video, an NDR reader for .mp4, .avi and .mov files. The video reader type is defined by the file format, not by a library. It reads the frames of a video container (MPEG-4, AVI, QuickTime) and the time stored for each frame, so any language can implement it. This MATLAB implementation uses MATLAB's built-in VideoReader and needs no toolbox. A Python implementation can use any library that reads the same containers and their frame timestamps (e.g. PyAV).

Until now the only NDR reader that could open video was ndr.reader.imagestack, which depends on NANSEN. With this reader, NDI can read video through ndi.daq.reader.image.ndr('video') without extra toolboxes. The NDI V2 Haley import records 'video' as its cameras' reader.

• New reader +ndr/+reader/video.m (subclass of ndr.reader.base). It has the same frame API as imagestack/tiffstack: numframes, framesize, dimensionorder, datatype, frametimes, readframes (including SelectC/SelectZ), epochclock, t0_t1, getchannelsepoch and metadata.
  • Dimensions: frames are [Y X C Z T] in 'YXCZT' order, with Z=1. RGB video has C=3 and greyscale video has C=1. Both C and the datatype are read from the first decoded frame.
  • Epochs: an epoch is one video file. Directories are expanded and non-video companion files (such as epochprobemap.ndi) are ignored. Anything other than exactly one video file is an error.
  • Frame times are each frame's presentation time: the timestamp the container stores for the frame, in seconds from the start of the file, on the dev_local_time clock. They are read, not computed from the frame rate. In MATLAB, frametimes decodes the file once and records VideoReader's CurrentTime before each readFrame, then caches the result per file for the session. This matters on real data: a one-hour lab video (10,796 frames) drifts up to 1.33 s from (k-1)/FrameRate.
  • t0_t1 reads only a few frames (ndr.reader.video.endtimes). It reads the first frame's time before decoding frame 1. For the last frame it seeks a few frame periods before Duration and decodes to the end, widening the margin if the seek overshoots. It falls back to the full scan only when the file reports no usable FrameRate or Duration. NDI asks every epoch for t0_t1 when it lists a system's epochs, so a full decode there would decode every video of the system. On the one-hour video this takes 0.25 s, against 10.2 s for the full scan, with identical start and end times.
  • numframes uses NumFrames, and counts frames by decoding only when the container reports no count. readframes reads with read(v, idx), one call per run of consecutive indices.
  • metadata: israster=false; only frame_period = 1/FrameRate is filled in.
• Registration: added to resource/ndr_reader_types.json as video, mp4, avi and mov. These are format names only, so the stored reader type means the same thing in MATLAB and Python.
• Merged with main, including smartspim: prefer SmartSPIM's stitched All_Channels volume #142. The merge was clean.

Tests

tools/tests/+ndr/+unittest/+reader/TestVideo.m writes synthetic videos: 12-frame 24x32 at 4 fps, plus a 60-frame one. RGB video is written as MPEG-4 where the platform supports it and as Motion JPEG AVI otherwise; greyscale is written as lossless Grayscale AVI. The tests cover:

• the known_readers listing, the size, dimension order and datatype;
• frame times and their spacing, and t0_t1;
• random access with indices in mixed order, SelectC, and greyscale content round-tripping exactly;
• companion files being ignored, and the errors for zero or several video files and bad indices;
• metadata;
• t0_t1 on a file neither cache has seen, which must agree with the full scan, for both RGB and greyscale;
• endtimes on a 60-frame video, which reaches the end by seeking and must agree with the full scan.

On real data, on macOS with the Haley lab's StreamPix .mp4 files:

• a one-hour behaviour video and a 74-frame lawn clip both give identical start and end times from the seek and from the full scan;
• NumFrames equals the number of frames decoded;
• read through NDI, the videos give 117 and 118 frames of 1024x1024x3 uint8.

In NDI, TestHaleyReadVideo (VH-Lab/NDI-matlab#1021) reads a frame through daqsystem_load → readframes using this branch.

Test plan

• Test code (run_tests.yml) green on 6dfeb51: 409 passed, 0 failed. The 3 incompletes are the CED sonpipe tests, which skip without that tool.
• Test NDR Symmetry and Test .smrx green on 6dfeb51.
• Re-run on 488417d (help text and the videoreader alias only; no code change).

After merge

NDI's test-import-v2.yml checks out this branch in a step marked temporary. That step can be removed once this PR is merged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FxKLA8Tbf9cpvDe7fuVeKf

---

Generated by Claude Code

[github-actions]

Test Results

412 tests  +14   409 ✅ +14   43s ⏱️ -4s
 52 suites + 1     3 💤 ± 0 
  1 files   ± 0     0 ❌ ± 0 

Results for commit 794a0e8. ± Comparison against base commit 966fe3c.

♻️ This comment has been updated with latest results.

[codecov]

Codecov Report

❌ Patch coverage is 86.51685% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 54.88%. Comparing base (966fe3c) to head (794a0e8).

Files with missing lines	Patch %	Lines
+ndr/+reader/video.m	86.51%	24 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #140      +/-   ##
==========================================
+ Coverage   54.26%   54.88%   +0.61%     
==========================================
  Files         163      164       +1     
  Lines        9420     9598     +178     
==========================================
+ Hits         5112     5268     +156     
- Misses       4308     4330      +22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[stevevanhooser]

This raises an interesting issue. The videos would be unplayable on python if we make this mean "the MATLAB VideoReader". Maybe we want to make it an 'mp4', 'avi', and 'mov' reader generically, so that that both Matlab and python can play it? On the Matlab side, VideoReader can be the implementation, and on the python side, it can use whatever libraries are appropriate. How does that sound?

[jesshaley]

I think I’m a bit confused. Are you saying it’s the matlab only video reader because of the PR title? Or the code itself? Your description sounds exactly how I was imagining it. The PR title ended up that way because V1 was using a Nansen-style imagestack reader and I wanted to use the native matlab video reader instead for V2 videos. *Jess Haley, PhD* ***@***.*** Scientific Programmer and Data Consultant Waltham Data Science LLC Makers of NDI Cloud <https://ndi-cloud.com/>

…

On Tue, Oct 20, 2026 at 4:52 AM Steve Van Hooser ***@***.***> wrote: *stevevanhooser* left a comment (VH-Lab/NDR-matlab#140) <#140 (comment)> This raises an interesting issue. The videos would be unplayable on python if we make this mean "the MATLAB VideoReader". Maybe we want to make it an 'mp4', 'avi', and 'mov' reader generically, so that that both Matlab and python can play it? On the Matlab side, VideoReader can be the implementation, and on the python side, it can use whatever libraries are appropriate. How does that sound? — Reply to this email directly, view it on GitHub <#140?email_source=notifications&email_token=AJDALDLFIEXB7HWS5LLFW5L5U5GYNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMRZGAZDCNRYGQZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KYZTPN52GK4S7MNWGSY3L#issuecomment-6290216842>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJDALDIBMUOY6NXUBTUBQKT5U5GYNAVCNFSNUABFKJSXA33TNF2G64TZHMZTAMZYGEYDGOBRHNEXG43VMU5TKNRWHAYDOOJZHE42C5QC> . Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS <https://github.com/notifications/mobile/ios/AJDALDJABZM7HPN6HLXB4PD5U5GYNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMRZGAZDCNRYGQZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2KUZTPN52GK4S7NFXXG> and Android <https://github.com/notifications/mobile/android/AJDALDN4IXQR4EJCZOE74HT5U5GYNA5CNFSNUABFM5UWIORPF5TWS5BNNB2WEL2JONZXKZKDN5WW2ZLOOQXTMMRZGAZDCNRYGQZKM4TFMFZW63VGMF2XI2DPOKSWK5TFNZ2K4ZTPN52GK4S7MFXGI4TPNFSA>. Download it today! You are receiving this because you authored the thread.Message ID: ***@***.***>

[stevevanhooser]

OK I didn't look carefully, I just read the header. So great, sounds like it is fine, thanks! I'll merge

[jesshaley]

Agreed, and the code already works that way: the reader type stored in the data is video (class ndr.reader.video), and nothing in a document depends on VideoReader. What tied it to MATLAB was the wording, plus one alias. Fixed in 488417d:

• Title and help text now say the video type is defined by the file format (mp4/avi/mov). VideoReader is how MATLAB implements it; Python can use whatever reads the same containers (e.g. PyAV).
• The videoreader alias is removed from ndr_reader_types.json. The type is now registered as video, mp4, avi and mov. Nothing in NDI used the alias.
• Frame times are defined independently of MATLAB: each frame's presentation time, the timestamp the container stores for it, in seconds from the start of the file. In MATLAB that is VideoReader's CurrentTime read just before each frame is decoded. A Python reader that returns the same timestamps gives the same times.

No change to how frames are read; the tests are unchanged.

---

Generated by Claude Code

[jesshaley]

Correction to my comment above: this PR was merged at 794a0e8 a minute before 488417d was pushed, so the wording change and the removal of the videoreader alias are not in main. They are in #143.

---

Generated by Claude Code