Skip to content

fix(recording): keep a Studio recording when a resumed segment captures no frames - #4297

Open
richiemcilroy wants to merge 10 commits into
mainfrom
fix/empty-segment-pause
Open

richiemcilroy wants to merge 10 commits into
mainfrom
fix/empty-segment-pause

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

A user reported a Studio recording that hung on Pause and would not open afterwards. Their log shows Failed to finish MP4 encoder: NotWriting right after pausing.

Root cause

Two ways a short resumed segment loses the whole recording on main:

  • No screen frame before Pause (non-fragmented writer: camera on, or crash recovery off). The AVAssetWriter never started, so MP4Encoder::finish_start returned NotWriting. AVFoundationMp4Muxer::finish made that a display media error, Actor::stop_pipeline made it terminal, and Pause failed the recording.
  • One screen frame before Pause (fragmented writer). Recording succeeds, but the app's finalize step (RecoveryManager remux) fails the whole recording. macOS fragments carry no sample durations. StreamRemuxer copied the lone frame with duration 0, so the MP4 track had zero length and readers discarded its only sample: display video validation failed ... No decodable frames found after trying 1 packets.

Only the first segment gets the start gate's held frame, so any resumed segment can hit this when Pause lands before ScreenCaptureKit delivers frames. On the Mac mini, pausing straight after Resume reproduces the second case on main with no fault injection.

flowchart LR
  R[Resume: new SCK stream] --> P[Pause before frames arrive]
  P -->|0 frames| A[MP4Encoder::finish]
  A -->|before: NotWriting| F1[Pause fails, recording Failed]
  A -->|now: NoFrames -> NoVideoFramesCaptured| K1[segment left out, recording kept]
  P -->|1 frame, fragmented| B[finalize remux]
  B -->|before: zero-length track| F2[finalize fails, recording lost]
  B -->|now: one-frame duration| K2[segment kept and playable]
Loading

Change

  • crates/enc-avfoundation/src/mp4.rs: a writer that never started reports FinishError::NoFrames. NotWriting stays for a second finish.
  • crates/recording/src/output_pipeline/macos.rs: the screen and camera muxers map NoFrames to a typed NoVideoFramesCaptured (output_pipeline/core.rs). Every other finish error is unchanged. OutputPipeline::stop_with_outcome is untouched, so the capture-exited rules stay as they are.
  • crates/recording/src/studio_recording.rs:
    • Pipeline::stop marks the segment screen_empty when the screen reported NoVideoFramesCaptured, or finished cleanly with zero frames (fragmented, Windows and Linux writers). That is not a media error. A camera with no frame is left out of that segment, like a segment recorded with the camera off.
    • The watcher treats NoVideoFramesCaptured as a clean end only while Pause or Stop is stopping the pipeline. A track that ends on its own is still a failure.
    • stop_pipeline leaves an empty segment out. It moves the segment folder to content/empty-segments/segment-N-K and the next segment reuses index N, so content/segments stays numbered like the saved segments, which recovery and finalize rely on. The segment's other files are kept.
    • RecordingMarkers::segment_dropped moves markers pressed in that segment to the end of the segment before.
    • A recording whose only segments were empty still fails clearly, as on main.
  • crates/enc-ffmpeg/src/remux.rs: StreamRemuxer gives a video stream's first packet one frame's duration when it has none. The MP4 muxer takes every other sample's duration from the next sample, so only a single-frame output changes.

Callers checked

  • MP4Encoder::finish / finish_nowait: Studio screen (non-fragmented), Studio camera, Instant camera-only (still an error, now typed), Drop.
  • Pipeline::stop: Studio Pause, Stop, Cancel. stop_pipeline: Pause and Stop.
  • StreamRemuxer: remux_file, concatenate_m4s_segments_with_init(_validated), the concat-demuxer paths. Audio streams are untouched.
  • stop_recording and SegmentStartTimes are untouched; an empty segment never reaches them.

Tests

  • enc-avfoundation: finish_without_any_frame_reports_no_frames (fails on main with NotWriting), finish_after_a_single_frame_writes_a_playable_file.
  • enc-ffmpeg: remux_of_a_single_durationless_frame_stays_playable (0 decodable frames on main).
  • recording, macOS: finishing_without_a_frame_reports_no_video_frames for the screen and camera muxers.
  • recording, all platforms: stop_reports_a_screen_and_camera_that_captured_nothing_as_empty, a_dropped_segment_hands_its_markers_to_the_segment_before.
  • recording, macOS and Windows: a_resumed_segment_without_screen_frames_is_left_out drives the actor through Resume, Pause and Stop (and Stop while recording) for both writer behaviours, and checks the folder is moved aside.
  • hardware_studio_empty_segment (real ScreenCaptureKit, debug build): 8 quick pauses after Resume, and a resumed segment with cap_fail skipping every screen frame. Fragmented and not, each followed by the app's finalize step and a decode of every saved display.

Measured on a Mac mini (M4, macOS 27), main vs this branch

Scenario main this branch
Resumed segment with no screen frame, non-fragmented Pause fails, recording Failed Pause in 32 ms; 2 segments kept, Complete, exports 4.10 s (4.08 s recorded)
Same, fragmented Pause fails Pause in 36 ms; 2 segments kept, finalized, exports 4.07 s (4.05 s recorded)
8 quick pauses after Resume, fragmented finalize fails: segment-2/display.mp4 not decodable 9/9 segments finalized, exports 2.90 s (2.89 s recorded)
8 quick pauses after Resume, non-fragmented 9/9 kept 9/9 kept, exports 2.73 s (2.73 s recorded)
Remux of that recording's 9 real segments 5 single-frame segments undecodable all decodable; the 4 multi-frame outputs byte-identical to main
framemd5 of 120 frames with a pause gap through the AVFoundation writer 14ca1258… 14ca1258…

Normal recordings (3 segments, system audio, 2 s each with 0.5 s pauses):

main this branch
Saved meta (segment count, status, fps, tracks) identical identical
Pause median, non-frag / frag 101 / 761 ms 104 / 561 ms
Resume median, non-frag / frag 92 / 90 ms 99 / 95 ms
Stop median, non-frag / frag 735 / 754 ms 553 / 657 ms
Display start relative to audio, median −74.6 / −74.4 ms −77.2 / −71.9 ms
Segment display duration, median 2.053 s 2.050 s

The mini was shared with other builds, so fragmented Pause times swing by seconds on both sides. One run on each side hit an unrelated Output storage stalled while finishing on the build volume.

Not proven on hardware: Windows and Linux capture. Their writers finish cleanly with zero frames, a case the cross-platform unit tests cover.

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no new actionable issue was found.

Summary

Keeps Studio recordings when a resumed segment captures no screen frames, and gives a lone durationless video packet a usable duration during remux.

  • Studio keeps saved segments when a resumed segment captures no screen frames.
  • Remux keeps a single-frame video track playable.

The three previous unnumbered findings are addressed: segment folders stay aligned with saved indexes, unexpected capture endings still reach the caller, and the redundant test comment was removed. No new actionable issue was found. Tests were inspected, not run.

Reviews (3) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

@sandchest-com

sandchest-com Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Sandchest

Sandchest reviewed 566222c: nothing exploitable found in this change.

Sandchest reviews every change for vulnerabilities an attacker could use · 3 credits · Review details

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

View 1 other finding in Capy.

Open in Capy

Comment thread crates/enc-avfoundation/src/mp4.rs Resolved
@vercel

vercel Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Nov 17, 2026 12:58am UTC

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

View 2 other findings in Capy.

Open in Capy

&& packet.duration() <= 0
&& let Some(rate) = self.video_frame_rate[output_index]
{
packet.set_duration(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-packet duration fallback may shift video timing on multi-frame tracks

Medium Severity · Investigate

The new branch in StreamRemuxer::write_packet runs for the first packet of every video stream whose packet duration is <= 0, not only for a stream that holds one frame. recovery.rs (lines 2586 and 2622) sends every fragmented Studio video track through concatenate_m4s_segments_with_init[_validated], and the code comment and the new test say these fragments carry no sample durations. So for a normal multi-frame display or camera track, packet 0 receives a duration of 1/avg_frame_rate, while the real gap from packet 0 to packet 1 is whatever the capture produced. Packets after the first still get their durations from dts deltas, so if the mov muxer honours an explicit first-packet duration (as the single-frame test's packets[0].duration assertion suggests), every later video sample is shifted earlier by (first_gap - 1 frame) relative to the audio tracks. Screen capture delivers frames on content change, so a static screen right after a resume can produce a first gap far larger than one frame. Fix: apply the fallback only when the stream has no later packet, or derive the first duration from the next packet's dts, which is what the muxer already does. Verification target: libavformat movenc.c duration handling for packets with a non-zero explicit duration, and a new test with a multi-frame durationless fixture whose first dts gap is not one frame, checking the output video sample durations against the input dts. The existing multi-frame tests compare the validated and legacy outputs, and both now take this branch, so they cannot detect the change. Not verified by running code.

Open in Capy

# Conflicts:
#	crates/recording/src/studio_recording.rs

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

This branch was successfully deployed

1 active (outdated) and 1 inactive (outdated) deployments
Preview – cap-web — 6a39dcd8 Deployed Nov 17, 2026 by vercel[bot]
Preview – cap-status — 6a39dcd8 Deployed Nov 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant