Skip to content

feat(source_sampling): sample keyframes from remote video through a byte-range reader - #636

Merged
kstonekuan merged 4 commits into
mainfrom
remote-keyframe-sampling
Sep 27, 2026
Merged

kstonekuan merged 4 commits into
mainfrom
remote-keyframe-sampling

Conversation

@kstonekuan

@kstonekuan kstonekuan commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Lets probe_video and sample_source_frames read a video in object storage without downloading it.

  • KEYFRAMES seeks per bin. It used to decode every keyframe in the window in one pass, which reads the whole window even though it keeps at most maximum_frames. It now probes each bin start for its first keyframe and extracts that keyframe on its own. Selection and JPEG bytes are unchanged, so SOURCE_FRAME_SAMPLING_VERSION stays v1: tests/test_byte_range_source.py compares both against the old one-pass command across B-frames, scene-cut keyframes, a nonzero container start, mid-video windows, 1 to 16 frames, and both canvas modes.
  • hflow.serve_byte_ranges(reader) serves a caller's ByteRangeReader (size_bytes, read_range(start, stop)) to FFmpeg on 127.0.0.1 and yields a source both functions accept in place of a path. It fetches 256 KiB blocks as FFmpeg reads them and caches them for the with block. Credentials stay in the reader. A reader exception is re-raised as itself, never as UnreadableVideo or SourceSamplingError.
  • hflow.sources.PinnedSourceRangeReader is that reader for an obstore object: it takes a SourceExpectation with the revision and size (refusing a digest it cannot verify), sends the version with every range, and re-checks revision, path, and size, so a replaced object raises SourceReadError.

On four 90 MB, 2-minute test files (MP4 with the index at either end, MOV with B-frames, MKV), probing plus 3-frame KEYFRAMES_FIRST sampling fetched 4.5 to 5.5 MB. UNIFORM, NEAREST_KEYFRAMES, and the keyframes-first fallback still read the whole window.

…yte-range reader

KEYFRAMES sampling now seeks to each bin start and extracts each selected
keyframe on its own, instead of decoding every keyframe in the window in one
pass. It selects the same frames and writes byte-identical JPEGs, and reads
only the bytes near the selected keyframes.

hflow.serve_byte_ranges serves a caller's ByteRangeReader to FFmpeg on
loopback, so probe_video and sample_source_frames can read an object in
storage without downloading it. Reader failures are re-raised as the
reader's own exception rather than reported as unreadable media.
@greptile-apps

greptile-apps Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[High risk] Adds byte-range HTTP server for remote video sampling.

The PR appears safe to merge; no new actionable issue or outstanding previous finding remains.

Summary

The PR adds a loopback byte-range source for probing and sampling remote video, changes keyframe sampling to seek per bin, and documents how to use a pinned object reader. The change since the previous review clarifies that digest-bearing expectations are refused.

Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Pinned object reader] --> B[Loopback byte-range server]
  B --> C[FFprobe and FFmpeg]
  C --> D[Probe video]
  C --> E[Sample keyframes by bin]
  E --> F[JPEG frames]
Loading

Reviews (3) · Last reviewed commit: "docs(sources): say the range reader refu..."

Comment thread src/hflow/source_sampling.py
Comment thread src/hflow/media.py
Comment thread src/hflow/sources.py
Comment thread src/hflow/byte_range_source.py
…found; join loopback handlers; refuse unverifiable digests
Comment thread src/hflow/sources.py
@greptile-apps

greptile-apps Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Want your agent to iterate on Greptile's feedback? Try greploops.

@kstonekuan
kstonekuan merged commit ea18b18 into main Sep 27, 2026
3 checks passed
@kstonekuan
kstonekuan deleted the remote-keyframe-sampling branch September 27, 2026 19:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant