[ruccho]

Closes #163.

Problem

The MP4 files produced by every backend carried no color information at all — no colr box and no colour description in the SPS VUI. Players have to guess at the color space, and some warn about it.

Two backends were also converting with the wrong matrix, so this was not purely a tagging gap:

• VideoFrameBgra32::to_yuv420_planes (the readback path on Android and Windows) used BT.601 limited-range coefficients.
• The FFmpeg backend let FFmpeg pick the matrix, which is BT.601 at the resolutions this library records. Measured with a pure-red frame at the default 1280x720: Y=81 (BT.601) before, Y=63 (BT.709) after.

Change

Tag the output as BT.709 limited range (primaries 1, transfer 1, matrix 1, limited range) on all backends, and make the conversions actually produce BT.709 so the tags describe the pixels.

• Apple — color properties on the VTCompressionSession, plus matching format-description extensions in the muxer. AVAssetWriter writes the colr box from the format description it is handed, and since encoded frames travel back through the caller as opaque buffers, the muxer never sees the encoder's own.
• Android — color-standard / color-transfer / color-range on the encoder format, restated on the muxer track format because codecs are not guaranteed to echo them back.
• Windows — color attributes on both input and output media types, and a colr box added by the muxer after finalizing. Neither the MPEG-4 file sink nor the Microsoft software H.264 encoder writes any color description, whatever the media types say: the sink writes no colr box, and the encoder writes no video signal type into the VUI and documents no attribute or codec property for it. The sink writes moov as the last top-level box, so mux/colr.rs rewrites moov in place with an nclx box appended to the video sample entry; no media data or chunk offset moves. A file laid out any other way is left as written, with a warning. The media type attributes are kept for hardware encoders that may honor them.
• FFmpeg — -vf setparams=.... The -color_primaries / -color_trc output options are not reliably forwarded to the encoder: on 8.0.1 only the matrix and range reach the VUI. setparams also selects the conversion matrix, fixing the BT.601 conversion above.
• WebCodecs — convert to I420 in Rust with the same BT.709 conversion as the readback paths (VideoFrameBgra32::to_i420) and hand the browser an I420 frame tagged BT.709 limited range. Handing it a BGRA frame leaves the YUV conversion to the browser, and browsers disagree: Chrome 154 converts into whatever matrix and range the frame's color space declares, while the Chrome on the CI runner converted the same frame to BT.601 and tagged it as such. With no color space declared, Chrome tags the output as sRGB transfer and full range. muxide writes no colr box, so on the web the SPS VUI is the only declaration.

to_i420 shares its plane layout with to_yuv420_planes. This also fixes the chroma stride for odd widths, which was rounded down while the plane size was rounded up, so each chroma row after the first overwrote the end of the previous one.

Behavior change

This changes the color of existing output. The CPU conversion coefficients change from BT.601 to BT.709, and the FFmpeg backend's conversion changes likewise. Footage recorded after this change will not be pixel-identical to footage recorded before it. The new output is correct: previously the pixels were BT.601 but untagged, so players defaulted to BT.709 for HD content and rendered them with a color shift.

On the web, frames are now converted on the CPU in WebAssembly before encoding instead of by the browser. This adds a per-frame cost on the main thread.

Tests

• End-to-end (unienc_testkit) — the harness now reads the color description from both the video sample entry's colr box and the VUI of the SPS in avcC, and requires each one present to state BT.709 limited range. They are checked separately because they come from different places (the muxer and the encoder) and players disagree on which one wins. A colr box is required on every target except the web, where the SPS VUI is required instead. This runs on every CI target: Linux (FFmpeg), macOS, iOS simulator, Android emulator, Windows and the browser.
• Unit (unienc_common) — to_yuv420_planes / to_i420: limited-range end points, neutral chroma for every gray level, the BT.709 values of the primaries, agreement with the floating-point BT.709 definition across the RGB cube, padding, the odd-width chroma stride, and that to_i420 packs the same planes. Restoring the BT.601 coefficients or the old chroma stride fails them.
• Unit (unienc_testkit::h264, unienc_windows_mf::mux::colr) — the SPS VUI reader against SPS units written by libx264, and the colr insertion including box sizes and the untouched-file cases.

Verification

All jobs of CI unienc pass on d6a149b (run 36963200242). The color descriptions the e2e harness found:

Target	colr	SPS VUI
Linux / FFmpeg	BT.709 limited	BT.709 limited
macOS / VideoToolbox	BT.709 limited	BT.709 limited
Android emulator	BT.709 limited	BT.709 limited
Windows / Media Foundation	BT.709 limited	none
WebAssembly / Chrome	none (muxide)	BT.709 limited

The checks fail where the declaration is missing or wrong. With the Apple backend reverted to main, the harness reported colr: none; SPS: none locally on macOS. Before the Windows and WebCodecs fixes, CI (run 36957393259) reported colr: none; SPS: none on Windows and SPS: primaries 6, transfer 6, matrix 6 on the runner's Chrome.

Not covered by CI:

• The GPU blit paths on Apple and Android rely on the platform encoder honoring the requested matrix when converting BGRA itself. The harness feeds CPU frames only.
• A Windows hardware encoder could write its own VUI. If it wrote anything other than BT.709 there, it would contradict the colr box. CI runs the software encoder only.
• The e2e checks verify the declarations, not the decoded pixels. Pixel values were checked for WebCodecs by decoding in Chrome (white at Y=235, red at Y=63), and for the CPU conversion by the unit tests.

Transfer characteristics

The pixels reaching the encoder are sRGB-encoded in both Unity gamma and linear workflows. On the readback path, FramePreprocessor renders into an R8G8B8A8_UNorm target in gamma and an R8G8B8A8_SRGB target in linear. On the blit paths, is_gamma_workflow selects the sRGB render pipeline and destination format on Apple and the source view format on Android to the same end. The same tags therefore apply to both workflows.

Strictly, sRGB-encoded content is transfer characteristics 13 (IEC 61966-2-1) in ITU-T H.273, not 1. This change tags 1 (BT.709) anyway, following the common practice for SDR screen capture. Players generally render the two identically, while support for 13 is less consistent across players and hardware decoders.

Note on merge order

This touches CompressionSession::new in unienc_apple_vt/src/video/mod.rs and the MediaFormat setup in unienc_android_mc/src/video/mod.rs — the same places as #172 (configurable IDR interval). Depending on merge order one of the two will need conflict resolution.

Before merging

The native binaries under Packages/jp.co.cyberagent.instant-replay/UniEnc/Plugins/ must be rebuilt via the build-unienc.yml workflow (manual dispatch).