Skip to content

feat(videos): take gallery thumbnails from a representative frame (~1s in), not frame 0 - #199

Draft
lstein wants to merge 2 commits into
mainfrom
fix/video-thumbnail-representative-frame
Draft

feat(videos): take gallery thumbnails from a representative frame (~1s in), not frame 0#199
lstein wants to merge 2 commits into
mainfrom
fix/video-thumbnail-representative-frame

Conversation

@lstein

@lstein lstein commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #195 (feat/video-ingest-transcode) — merge that first; this follows cleanly.

Why

Frame 0 is a poor thumbnail source: generated videos routinely fade in from black, and #195's audio-only uploads render their synthesized waveform track's first frame from a near-empty audio window — every audio upload lands in the gallery as an undistinguished black tile.

What

  • representative_thumbnail_frame_index(duration, fps): ~1 s into the clip, capped at its midpoint so short clips still resolve to a real frame; unknown/degenerate metadata degrades to frame 0; unknown fps assumes 24 (the models' native rate and the waveform track's rate).
  • extract_representative_video_frame(...): decodes that frame, falling back to frame 0 when the index has no decodable frame (container metadata is untrusted — overstated duration/fps must not cost the thumbnail). A timeout is never retried in either mode — the helper distinguishes timeout from decode failure internally — so an adversarial upload cannot double its hold on a request worker.
  • Wired into both thumbnail producers: the upload probe (which already decodes a frame to prove decodability — it now proves it on the representative frame; acceptance is thereby slightly wider: a file with a corrupt frame 0 but a decodable ~1 s frame is accepted instead of 415'd) and the disk store's save-time fallback used by generated/derived videos, with duration/fps passed down from the video service. The copy path still reuses the source's existing thumbnail; existing thumbnails are not regenerated.

Testing

  • 19 new tests: index selection across metadata edge cases; through-the-worker extraction verified against a brightness-ramp clip (the pixel value identifies the frame); metadata-overstatement fallback; timeout no-retry pinned in both modes.
  • Full affected suites pass (121: thumbnails, upload limits, video_files, videos service) plus the 60 multiuser router tests.
  • Adversarial fresh-context review run; its two findings (a timeout-retry hole in non-raise mode, and two docstring overclaims) are fixed in the second commit. It also verified: imageio's seek path bounds the cost of a ~1 s index well under the decode timeout; the webv2 <video> poster is the only frame-semantic consumer and self-corrects on loadedmetadata; the waveform wrap's probed duration/fps are real, so the fix genuinely engages for audio uploads.

🤖 Generated with Claude Code

https://claude.ai/code/session_014xFbHnmFTLsHE9e1PAdvme

lstein and others added 2 commits September 1, 2026 22:29
…s in), not frame 0

Frame 0 is a poor thumbnail source: generated videos commonly fade in
from black, and an audio-only upload wrapped in the synthesized waveform
track renders its first frame from a near-empty audio window - an
undistinguished black tile in the gallery.

Add representative_thumbnail_frame_index (~1s into the clip, capped at
its midpoint; untrusted/absent metadata degrades to frame 0) and
extract_representative_video_frame (falls back to frame 0 when the
computed index has no decodable frame, so overstated container metadata
still yields a thumbnail; a decode timeout propagates without a second
attempt so an adversarial upload cannot double its hold on a request
worker).

Wire it through both thumbnail producers: the upload probe (which
already decodes a frame to prove decodability - it now proves it on the
representative frame, same acceptance semantics) and the disk store's
save-time fallback used by generated/derived videos, with duration/fps
now passed down from the video service. The copy path still reuses the
source's existing thumbnail.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014xFbHnmFTLsHE9e1PAdvme
- A timeout is now never retried in either mode: the helper raises
  internally to distinguish timeout from decode failure, so the disk
  store's non-raise mode no longer runs a second full-budget frame-0
  decode after a hung representative decode (which would have doubled an
  adversarial upload's hold on a request worker).
- Correct two overclaims: the probe docstring no longer says acceptance
  semantics are unchanged (they are strictly wider - a file with a
  corrupt frame 0 but a decodable ~1s frame is now accepted), and it
  proves 'a decodable frame', not 'a decodable first frame'.
- Two new tests pin the timeout behavior in both modes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014xFbHnmFTLsHE9e1PAdvme
Base automatically changed from feat/video-ingest-transcode to main September 3, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant