Reduce CPU memory usage for long videos

[heiwang1997]

Summary

• Store cached RGB frames as uint8 on CPU and restore float RGB when moving cached frames back to CUDA.
• Make vipe infer stream inputs by default, with --cache-input as an opt-in fallback for malformed videos.
• Write artifacts in a single stream pass and avoid output-stream caches unless visualization or returned streams require replay.
• Compact the default VDA frame staging path and add regression tests for single-pass artifact writing and compact cache storage.

Related issues: #44, #61, #66, #68.

Long-video RAM benchmark

Benchmark video: /tmp/vipe-ram-bench/dog-488f.mp4, generated by looping assets/examples/dog-example.mp4 to 488 frames at 1280x720, duration 16.28s. Measurements used /usr/bin/time -v.

Mode	Frames	Cached RGB	Max RSS	Wall time
Stream, no cache	488	0	0.87 GiB	3.46s
Old float cache simulation	488	5146.9 MiB	5.78 GiB	4.91s
New compact cache	488	1286.7 MiB	2.48 GiB	6.99s

End-to-end checks on the same synthesized clip:

Run	Config	Frames	Max RSS	Wall time	Output validation
Pose/artifacts	no instance, no post depth, no viz	488	4.52 GiB	55.55s	RGB, pose, intrinsics all 488 frames
Depth-enabled	default adaptive_unidepth-l_svda, instance off, no viz	244	8.62 GiB	1m13.56s	RGB, pose, intrinsics, depth all 244 frames

The default VDA path is still the heavier mode because it stages whole-video depth alignment data, but cached RGB retention is reduced 4x and the artifact path no longer forces a full post-depth output cache.

Validation

• uv run --group dev ruff check vipe/streams/base.py vipe/pipeline/processors.py vipe/priors/depth/videodepthanything/video_depth.py vipe/cli/main.py vipe/utils/io.py vipe/pipeline/default.py vipe/pipeline/panorama.py tests/test_memory_usage.py
• uv run --group dev ruff format --check vipe/streams/base.py vipe/pipeline/processors.py vipe/priors/depth/videodepthanything/video_depth.py vipe/cli/main.py vipe/utils/io.py vipe/pipeline/default.py vipe/pipeline/panorama.py tests/test_memory_usage.py
• uv run --group dev mypy vipe/streams/base.py vipe/pipeline/default.py vipe/pipeline/panorama.py vipe/cli/main.py vipe/utils/io.py vipe/pipeline/processors.py
• uv run --group dev pytest -> 31 passed

[sruggier]

I've been experimenting with this yesterday and today, and it makes a big difference with the dataset I'm using. I don't have a precise comparison to share (I wasn't using time -v during the earlier run), but I was seeing peak RSS usage of roughly 90 GiB, and these changes bring it down to less than 49.