demux: grow MJPEG MP4 sample buffer - #55
binghan-meng-spacemit merged 3 commits into
Conversation
## Purpose Prevent valid high-resolution MJPEG recordings from ending early when a compressed sample exceeds the old 512 KiB MP4 read buffer. The reader returned the same status as EOF, hiding the remaining frames from replay consumers. ## Changes - Grow and reuse a demuxer-owned heap buffer, with a 64 MiB allocation guard. - Preserve the buffer and sample index on allocation failure; free it on close. - Keep the existing Annex-B output limit and explicitly reject oversized non-JPEG samples rather than returning partial output. - Add a hardware-independent sample-reader test and reproducible validation notes in docs/mp4_mjpeg_read_buffer.md. - Base this focused fix on upstream 2a2dacd, without PR spacemit-com#51, global stream payload changes, camera-sdk pins, or UAV changes. ## Validation - Host: cc -std=c11 -D_GNU_SOURCE -Wall -Wextra -Wno-unused-parameter -Werror -g -fsanitize=address,undefined -Iinclude test/test_mp4_read_buffer.c -o /tmp/test_mp4_read_buffer; ASAN_OPTIONS=detect_leaks=1 /tmp/test_mp4_read_buffer: PASS. - X86 cross build, SpacemiT v1.2.4 / GCC 15.2, Release: cmake --build .cross/build/mpp-mp4-mjpeg-read-buffer --parallel 4 --target mpp test_mp4_read_buffer test_sys_stream_ref test_vdec_stream_ref: PASS. - K3 10.0.91.119, taskset -c 0-7: five sample-reader test runs plus existing SYS/VDEC reference-handoff unit tests: PASS. - Existing 4000x1200 MJPEG MP4 (~45.49 seconds): unpatched upstream reader stopped at 794 packets; patched libmpp returned all 1363. First formerly rejected sample: 525556 bytes; maximum: 649848 bytes. Common packet sizes and PTS match; final PTS 45459474 us. - ldd confirmed the isolated patched libmpp.so.1; binary SHA-256 and exact commands are documented. Initial wrong-SONAME harness run was rejected. - git diff --check: PASS. New test formatted with clang-format. ## Configuration and compatibility No public API, SYS/VB layout, decoder input pool, or timestamp changes. Buffer growth is ordinary heap allocation, not CMA. The 64 MiB low-level reader guard does not change bind-mode DEMUX's independent 1 MiB stream limit. No large H.264/H.265 support is claimed. Full camera/VDEC/VIO/LAS2/mapping/DRM integration on the newly merged SYS ABI was not rerun; formal runtime and dependency pins remain unchanged. This branch is for review, not an automatic SDK upgrade.
Robot CI AI Review
结论:暂无明确问题。 备注:worker-cache review 失败,已回退 baseline review:worker-cache review failed: snode5: fetch base failed: From github.com:spacemit-com/mpp |
## Purpose Unblock the existing MPP PR spacemit-com#55 Board CI style gate and document the source and runtime evidence for its automated review findings. ## Changes - Add the comparison-specific CHECK_GE test macro and use it for the minimum generated sample size, resolving the reported readability/check warning. - Clarify that standalone commands run from the repository root, and that quoted private headers resolve relative to the included implementation. - Document Close versus Destroy ownership; Close intentionally leaves the reader object valid for the regression's member checks. - Keep the production MP4 reader and the actual 2026-09-22 validation date unchanged. The review itself withdrew its sample-index and object-lifetime findings; the original documented compiler command also works unchanged. ## Validation - Reproduced the original readability/check failure with the available ament cpplint 1.5.5: exactly one CHECK(a >= b) diagnostic before this change; zero afterward, using --filter=-,+readability/check. - clang-format --style='{BasedOnStyle: LLVM, IndentWidth: 4, ColumnLimit: 110}' --dry-run --Werror test/test_mp4_read_buffer.c: PASS. - Original standalone command from the repository root, unchanged -Iinclude: cc -std=c11 -D_GNU_SOURCE -Wall -Wextra -Wno-unused-parameter -Werror -g -fsanitize=address,undefined -Iinclude test/test_mp4_read_buffer.c -o <evidence>/host/test_documented_command_before: PASS; ASAN_OPTIONS=detect_leaks=1 <binary>: PASS. - bash .cross/validation/mpp-mp4-mjpeg-read-buffer-20260922/build.sh: PASS. X86 cross-build, SpacemiT 1.2.4 / GCC 15.2, K3 Release; mpp and the reader, SYS stream reference and VDEC stream reference test targets all built. - bash .cross/validation/mpp-mp4-mjpeg-read-buffer-20260922/run-tests.sh: PASS. Rebuilt host ASan/UBSan test and five K3 reader runs, SYS/VDEC mocked handoff tests on root@10.0.91.119 cores 0-7, plus actual MP4 demux: baseline 794 packets versus fixed 1363, matching common packet sizes and PTS. Low-level demux only; this is not a hardware decode/inference/display test. - Evidence: ci-review-build.log and ci-review-test.log beside these scripts. - git diff --check: PASS. Remote Board CI must rerun on this new commit; local checks are not a claim that the remote gate has already passed. ## Configuration and compatibility Test and documentation only. No decoder behavior, public API, timestamps, buffer sizing, production board runtime or camera-sdk/UAV pins change. Reuse fix/mp4-mjpeg-read-buffer and the existing upstream PR spacemit-com#55; do not push to main or open a duplicate PR.
CI follow-up for
|
Purpose
Prevent valid high-resolution MJPEG recordings from ending early when a
compressed sample exceeds the old 512 KiB MP4 read buffer. The reader returned
the same status as EOF, hiding the remaining frames from replay consumers.
Changes
non-JPEG samples rather than returning partial output.
notes in docs/mp4_mjpeg_read_buffer.md.
payload changes, camera-sdk pins, or UAV changes.
Validation
-g -fsanitize=address,undefined -Iinclude test/test_mp4_read_buffer.c
-o /tmp/test_mp4_read_buffer; ASAN_OPTIONS=detect_leaks=1
/tmp/test_mp4_read_buffer: PASS.
cmake --build .cross/build/mpp-mp4-mjpeg-read-buffer --parallel 4 --target
mpp test_mp4_read_buffer test_sys_stream_ref test_vdec_stream_ref: PASS.
SYS/VDEC reference-handoff unit tests: PASS.
stopped at 794 packets; patched libmpp returned all 1363. First formerly
rejected sample: 525556 bytes; maximum: 649848 bytes. Common packet sizes
and PTS match; final PTS 45459474 us.
commands are documented. Initial wrong-SONAME harness run was rejected.
Configuration and compatibility
No public API, SYS/VB layout, decoder input pool, or timestamp changes. Buffer
growth is ordinary heap allocation, not CMA. The 64 MiB low-level reader guard
does not change bind-mode DEMUX's independent 1 MiB stream limit. No large
H.264/H.265 support is claimed. Full camera/VDEC/VIO/LAS2/mapping/DRM integration
on the newly merged SYS ABI was not rerun; formal runtime and dependency pins
remain unchanged. This branch is for review, not an automatic SDK upgrade.