feat: add imported media tracks and editor waveforms by richiemcilroy · Pull Request #2308 · CapSoftware/Cap

richiemcilroy · 2026-09-17T18:11:41Z

Summary

Import local images and videos directly in the Tauri and GPUI editors, add persistent video lanes and reorderable track rows, and support projects made entirely from imported media.
Render imported video with its source audio in preview and export. Show video and music waveforms with source trim, gain, and mute applied.
Stream full-band audio into compact per-project waveform summaries. Bound decoding to two workers, cache by source identity, and draw only the visible timeline range.
Open screenshot projects in the regular timeline editor in Tauri and GPUI. Image selection replaces the standard sidebar tabs with image controls; Back restores them.
Style the full image canvas like the editor, with the original annotation and appearance tools, and apply drawings to the selected image track while preserving originals.
Reset drawing source and annotations on image replacement so the next canvas opens the new asset.
Prewarm the regular editor for screenshot windows, removing the old screenshot editor instance startup path.

Verification

macOS isolated native GPUI playback reached 1:04 of a 65 second imported video while video and music waveforms remained aligned. The isolated Tauri app showed both waveforms and inline image drawing.
Tauri native export produced a 65.000 second H.264 1280 × 720, 30 fps MP4 with AAC audio.
macOS native Tauri and GPUI screenshot fixtures opened the regular editor, showed the replacement image sidebar, entered Draw and Arrow canvases, applied annotations, restored standard tabs on Back, and preserved source-image hashes and one image timeline segment.
The final isolated Tauri build opened a screenshot card directly at tauri://localhost/editor after the prewarm change, then showed the image sidebar and canvas and returned with Back.
Twenty distinct two-hour, 48 kHz stereo music sources generated cold waveforms in 35.939 seconds wall time with two decoder workers and 37,437,440 bytes peak process RSS. Each two-hour cache is 72,036 bytes; ten warm reads rounded to 0 ms.
Rust formatting and strict Clippy passed for audio, editor, and Tauri on Rust 1.88. GPUI formatting, check, and Clippy passed for changed code on Rust 1.95. Desktop TypeScript, scoped Biome, desktop build, and focused tests passed.
Prior waveform-head CI passed: GPUI test suites and Tauri app builds on macOS, Windows, and Linux; strict Clippy on macOS and Windows, plus formatting, typecheck, and Biome. A/V sync passed on all three platforms.
Final-head CI has passed formatting, typecheck, Biome, macOS and Windows Clippy, and code analysis as of 2026-09-18 14:13 UTC; its macOS, Windows, and Linux desktop builds are still running. Final-head A/V sync passed on all three platforms.

Limits

Windows and Linux interactive editor sessions have not been run; automated desktop suites passed on both.
The existing playback/export mixer preloads the used duration of music sources into 48 kHz float buffers. Waveform generation and drawing are bounded, but twenty simultaneous two-hour playback segments can still use substantial mixer memory. That mixer path needs a separate streaming change before claiming end-to-end memory bounds for such playback.
Legacy screenshot annotations remain editable in the image canvas. On first open, the regular timeline preview uses the original bitmap until Apply renders those annotations into the image track.

Confidence Score: 5/5

The PR appears safe to merge, with no outstanding blocking or non-blocking findings.

Summary

This PR adds imported image and video tracks, source-audio playback and export, waveform generation and rendering, and unified screenshot editing across the Tauri and GPUI editors.

Adds persistent, reorderable media lanes and imported-media controls.
Adds bounded waveform decoding, caching, cancellation, and visible-range rendering.
Extends preview, playback, rendering, and export pipelines for imported video and audio.
Routes screenshot projects through the regular timeline editor while retaining drawing and annotation workflows.

_{Reviews (12) · Last reviewed commit: "fix: prewarm screenshots in unified edit..."}

superagent-security

Superagent found 1 security concern(s).

richiemcilroy · 2026-09-17T18:36:01Z