fix(webv2): audio uploads as Ref2VA references - #206
Open
lstein wants to merge 4 commits into
Open
Conversation
An uploaded audio file is stored as a rendered-waveform video, so a reference made from one arrived on "Video + audio" -- conditioning the generation on a picture of the sound as well as the sound. The ingest pipeline already stamps `media_origin: audio_upload` on those clips. The reference add path now reads that stamp (fetched alongside the gallery resolve, so it costs no extra round trip) and starts such a reference on "Audio only"; the selector still offers all three modes. Reading the server's stamp rather than the picked file's type also covers a clip uploaded through the gallery first and then dragged onto the reference list. An unreadable metadata record is not an error -- it just means the ordinary video default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S4B5exWsbC2z2Uu2tA167A
lstein
requested review from
JPPhoto,
Pfannkuchensack and
blessedcoolant
as code owners
September 3, 2026 22:22
`check:architecture` runs lint, the unit/browser/fixture suites, a production build, and the performance and project-file gates -- ~9-10 minutes on a standard runner against a 10-minute cap, so a slightly slow runner cancelled a healthy run. Recent runs: 571s, 585s, 594s, 597s, and two cancellations at 617s/624s (one on an unrelated PR). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S4B5exWsbC2z2Uu2tA167A
An uploaded audio file becomes a waveform video, so it fills a VIDEO reference slot and shares that cap -- a separate "Add audio" button would carry the same counter and grey out with it, which reads as a bug. The one button is relabeled instead, and its picker now lists the accepted extensions explicitly alongside the video/* and audio/* wildcards: a file whose type the OS cannot map arrives as application/octet-stream, and then only the extension matches. Also accepts Windows Media (.wmv/.asf/.wma), which the bundled ffmpeg demuxes and decodes -- verified end to end: .wmv transcodes to H.264/AAC and .wma wraps into a waveform clip. Added to the upload route's extension lists and the gallery's client-side classifier so the two agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S4B5exWsbC2z2Uu2tA167A
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-ups to the audio-upload ingest from #195, in the Ref2VA reference list:
.wmv/.asf/.wma) uploads are accepted.frontend-webv2-testsjob'stimeout-minutesraised from 10 to 20.How
Audio-only default. The ingest route already stamps
media_origin: audio_uploadon wrapped audio uploads.getDefaultReferenceConditioning(metadata)reads that stamp, andVideoReferenceListFieldfetches the clip's metadata in parallel with the gallery resolve, so the add costs no extra round trip; a failed fetch falls back to the ordinary video default. Reading the server's stamp rather than the picked file's MIME type also covers a clip uploaded through the gallery first and then dragged onto the reference list. Recall (which restores the recorded conditioning) and the reference-extend anchor are untouched.One button, not two. An uploaded audio file becomes a waveform video, so it fills a video reference slot and shares that cap. Two buttons would show the same counter and grey out together — "Add audio (3/3)" disabled because you added three videos — so the single button is relabeled instead. The picker keeps the
video/*andaudio/*wildcards and adds the extension list: a file whose type the OS cannot map arrives asapplication/octet-stream, and then only the extension matches.Windows Media. Verified against the bundled ffmpeg (v7.0.2, which demuxes ASF and decodes wmv1/2/3, VC-1, wmav1/2/pro): a
.wmvfixture transcodes to H.264/AAC and a.wmafixture wraps into a waveform clip..wmv/.asf/.wmaadded to the upload route's extension lists and to the gallery's client-side classifier so the two agree.CI timeout.
check:architectureis lint + the unit/browser/fixture suites + a production build + the performance and project-file gates — ~9-10 minutes against a 10-minute cap (recent runs: 571s, 585s, 594s, 597s), so slightly slow runners cancelled healthy runs; two were cancelled on this branch and one on an unrelated PR. Happy to split this commit into its own PR if you'd rather.Testing
getDefaultReferenceConditioningunit tests (stamped / unstamped / absent metadata)._classify_uploadparametrized tests for the octet-stream extension fallback (.wmv,.asf,.wma,.MOV,.M4A, rejection of.txt);classifyGalleryUploadrows for.wmv/.WMA.pytest tests/app/routers/test_videos_multiuser.py— 66 passed.pnpm run check:architecturegreen locally (509 unit + 158 browser files, perf and project-file gates included).🤖 Generated with Claude Code
https://claude.ai/code/session_01S4B5exWsbC2z2Uu2tA167A