You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat(desktop-gpui): add captions at the playhead and on the empty caption lane - #4246
Add a caption by hand: the + button in the Transcript tab adds one at the playhead, and clicking the empty caption lane adds one where you click, like the other lanes do.
Screenshots
Linux under Xvfb at 1600x1000, debug build of #4245, #4246 and #4247 combined, on a 52 s recording with a word-timed transcript. The real editor is driven with xdotool and recorded with ffmpeg (probe run). Screen recordings: light, dark. Each add logged a reason="add-caption" timeline edit.
Light
Dark
+ in the Transcript header adds Caption 4 at the playhead, 9.84 to 11.84 s, selected for typing
A click on the empty caption lane adds Caption 5 centred on the click, 12.35 to 13.35 s, playhead moved there
Both hand-added captions come out in the saved SRT as cues 4 and 5.
macOS still (the + button (add caption at playhead) in the Transcript header):
How it works
flowchart LR
A[+ in Transcript tab] -->|playhead, 2 s| C
B[click empty caption lane] -->|click time, lane default length| C
C[find_placement on the caption track] -->|free gap| D[add_caption_segment]
C -->|no room| N[notice: No room for another caption here]
D --> E[captions.segments, recording time, no word timings]
E --> F[synchronize_captions] --> G[caption track, selected for typing]
Loading
The 0.6.0 editor had "Add" in its captions panel (TranscriptPage.tsxaddCaptionAtPlayhead). The GPUI editor had insert functions for the other segment kinds but not captions, and a click on the empty caption lane did nothing (editor_window.rs, the Hit::Empty arm).
add_caption_at (src/editor_window.rs) places the caption with the same find_placement the text, mask and shape lanes use, so it never overlaps an existing caption and takes the nearest free gap. It then selects it, which opens the caption panel so its text can be typed straight away.
The + button starts it at the playhead and makes it 2 s long.
A lane click centres it on the click, with the lane's default length, and moves the playhead there.
add_caption_segment (src/editor_edits.rs) maps the placement back to recording time through the clip under it, like every other caption:
it follows clip speed and fullscreen text holds, and stays on its moment as clips are trimmed, cut or moved
it ends where its clip ends; past that point the recording jumps, and the caption would cover footage it was never placed over
starting inside a fullscreen text hold there is no recording time to attach to, so it is refused with a notice instead of landing after the title
captions are switched on and the caption lane is shown. Projects with no transcript and no captions get a fresh source-time captions block.
legacy output-time captions (0.6.0 projects that were never opened since) are moved to recording time first, so the new caption cannot be mis-migrated later.
Decision: hand-added captions carry no word timings
A caption typed in by hand is text only (words: []), with a caption- id, the prefix the 0.6.0 editor also used for hand-added captions (is_manual_caption, src/transcription.rs).
That keeps it out of the transcript: it never shows up as spoken words, and filler removal and pause shortening never treat it as speech.
Editing such a caption's text in the caption panel keeps it text only (write_caption_edit_to_source). It used to invent evenly spaced word timings for any wordless caption.
Hand-added captions now survive a new transcription (apply_caption_result), including one added while transcription was still running. Generated captions are still replaced as before, and a refinement already keeps wordless captions (keep_caption_identities).
Splitting a hand-added caption gives its second half a caption- id too, so both halves survive.
Where a kept caption overlaps the new speech captions, it stays on screen there: it was placed by hand, and this is the same state as dragging a caption over another one today. The download in feat(desktop-gpui): download captions as SRT or WebVTT #4247 follows the same rule.
Callers checked
fresh_caption_source_id gained a prefix argument; its only other caller, split_caption_segment, still passes cap-split.
write_caption_edit_to_source: edit_caption_segment (caption panel) and the caption drag release. Captions with word timings behave exactly as before.
apply_caption_result: caption generation (editor_tabs.rs) and tests. Only caption- captions with no words are carried over.
The Hit::Empty arm: only the caption lane is new; every other lane keeps its branch.
What could regress
A wordless caption edited in the panel no longer gains word timings, so it shows without word-by-word highlighting. Only hand-added and old wordless captions are affected.
A regenerated transcript keeps caption- captions with no words. Generated captions never use that prefix, so they cannot be duplicated.
Projects saved by 0.6.0 and 0.6.1 to 0.6.3 load unchanged; nothing new is stored.
I found another case where a manual caption edit can be lost:
Start a word-timing refinement with an existing manual caption.
Edit that caption's text while refinement is running.
When refinement completes, the old text is restored.
words_signature doesn't include the text or timing of wordless captions, so the edit doesn't change the signature. apply_refined_captions then replaces the current segments with the refinement snapshot, which contains the old manual caption. A regression test against 94dcffd reproduced the replacement ("edited" became "original"). Could refinement preserve the current manual caption edits when it lands?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a caption by hand: the + button in the Transcript tab adds one at the playhead, and clicking the empty caption lane adds one where you click, like the other lanes do.
Screenshots
Linux under Xvfb at 1600x1000, debug build of #4245, #4246 and #4247 combined, on a 52 s recording with a word-timed transcript. The real editor is driven with xdotool and recorded with ffmpeg (probe run). Screen recordings: light, dark. Each add logged a
reason="add-caption"timeline edit.Both hand-added captions come out in the saved SRT as cues 4 and 5.
macOS still (the + button (add caption at playhead) in the Transcript header):
How it works
TranscriptPage.tsxaddCaptionAtPlayhead). The GPUI editor had insert functions for the other segment kinds but not captions, and a click on the empty caption lane did nothing (editor_window.rs, theHit::Emptyarm).add_caption_at(src/editor_window.rs) places the caption with the samefind_placementthe text, mask and shape lanes use, so it never overlaps an existing caption and takes the nearest free gap. It then selects it, which opens the caption panel so its text can be typed straight away.add_caption_segment(src/editor_edits.rs) maps the placement back to recording time through the clip under it, like every other caption:Decision: hand-added captions carry no word timings
words: []), with acaption-id, the prefix the 0.6.0 editor also used for hand-added captions (is_manual_caption,src/transcription.rs).write_caption_edit_to_source). It used to invent evenly spaced word timings for any wordless caption.apply_caption_result), including one added while transcription was still running. Generated captions are still replaced as before, and a refinement already keeps wordless captions (keep_caption_identities).caption-id too, so both halves survive.Callers checked
fresh_caption_source_idgained a prefix argument; its only other caller,split_caption_segment, still passescap-split.write_caption_edit_to_source:edit_caption_segment(caption panel) and the caption drag release. Captions with word timings behave exactly as before.apply_caption_result: caption generation (editor_tabs.rs) and tests. Onlycaption-captions with no words are carried over.Hit::Emptyarm: only the caption lane is new; every other lane keeps its branch.What could regress
caption-captions with no words. Generated captions never use that prefix, so they cannot be duplicated.Tests added
src/editor_edits.rsadd_caption_testsa_caption_with_no_transcript_turns_captions_on_in_recording_timea_caption_follows_clip_speed_and_stops_at_its_clip_edgea_caption_under_a_fullscreen_hold_is_refuseda_caption_inside_a_later_cut_leaves_with_it_and_comes_back_on_restorelegacy_output_time_captions_move_to_recording_time_firstsrc/editor_window.rsadd_caption_ui_tests(Linux, like the other editor window tests)the_playhead_caption_takes_the_next_free_gapa_full_caption_track_refuses_with_a_noticesrc/transcription.rsa_new_transcript_keeps_captions_typed_in_by_handediting_a_caption_typed_in_by_hand_keeps_it_out_of_the_transcriptsrc/editor_window.rs:both_halves_of_a_split_hand_added_caption_survive_a_new_transcript

Confidence Score: 5/5
The PR appears safe to merge; no new actionable issues remain.Summary
Adds captions from the Transcript header and empty caption lane.
Reviews (14) · Last reviewed commit: "style(desktop-gpui): format the retyped ..." · Reviewed by Greptile