feat: show AssemblyAI speaker diarization in transcripts

[pavzagor]

Cap already stores AssemblyAI word speakers, but transcription did not request them and captions/UI discarded them. This enables diarization for full recordings and editable-transcript backfills, shows Speaker A/B labels in the transcript, editor and player captions, and preserves labels through transcript edits, video cuts, copying, VTT/text downloads and agent API round-trips. Existing transcripts without labels continue to render normally.

Live chunks remain provisional and use no speaker labels: AssemblyAI identities are scoped to a transcription request. On recording completion, Cap queues a full-recording transcription instead of promoting independent chunks into a misleading final transcript. This adds a full transcription pass for recordings previously eligible for live promotion; the final labels appear when that pass completes. Queue failures propagate for workflow retry.

Validation:

• 2,749 web tests passed; 27 existing opt-in tests skipped. Added speaker-boundary, storage/edit/export round-trip, unknown-speaker, escaping, agent API, live handoff and translation validation tests.
• pnpm typecheck and pnpm exec biome ci . --linter-enabled=false passed; scoped Biome checks passed.
• Real AssemblyAI run through the local app, MySQL, MinIO and media server: 33.5-second synthetic two-speaker recording, four alternating turns, correctly identified A/B; transcript ID b7ffdd4a-2d21-4425-b8bd-bffa3f28159f.
• Browser verified transcript seeking, speaker captions, caption edit → save → reload, editor labels, word deletion → render → reload with remapped timestamps, and VTT download preserving voice tags. Mobile checked at 360px and 393px without horizontal overflow.
• Independent review completed; agent API metadata/entity findings resolved. Greptile translation finding fixed: both provider responses and cached translations must preserve cue IDs, timings and voice tags.

Also corrected the existing Slack-manifest test's stale expected brand color to match the current manifest, so the full web suite passes. No database migration is required; uses the existing ASSEMBLY_API_KEY (see the October 7 update for the new optional CAP_LIVE_TRANSCRIPTION setting).

Upstream validation on 2766dc0: CI and Recording Reliability passed. Greptile re-reviewed 24 files and added no new comments; security checks passed. Vercel preview remains blocked on Cap Software team authorization.

October 5 review fixes

Legacy cues containing literal text such as 2 < 3 no longer lose text. The parser strips recognized WebVTT tags and timestamps, then decodes entities; escaped VTT writes and React text rendering remain intact. Regression coverage includes caption display, copying, downloads, and the agent API.

Validation on ce473ccc: 49 focused transcript tests and 27 scheduling/live-handoff tests passed; scoped Biome passed. The actual-component transcript fixture used by #2417 also verifies complete literal text.

Cost-control review: recording-wide speaker identities require one canonical full-recording pass after provisional live chunks. This deliberately adds a pass to the previous successful-live-promotion path. The pre-feature workflow already used chunk transcription plus a full pass when promotion failed (321ae61b9, queueFullPassFallback). The canonical database claim prevents competing triggers from scheduling multiple canonical passes. Account-wide usage budgets and trusted media-duration enforcement remain broader pre-existing cost-control gaps; this change does not establish a new quota policy.

October 7 update

Rebased on current main (d327051).

• Live transcription is now opt-in. New optional server setting CAP_LIVE_TRANSCRIPTION (default false), defined with the existing boolString helper next to ASSEMBLY_API_KEY. With it off, instant recordings get exactly one full-recording transcription with speaker labels, so no recording is transcribed twice. Setting it to true restores provisional live transcripts, which then adds the full speaker-labelled pass after each live recording. Documented in self-hosting.mdx and docker-compose.coolify.env.example.
• Agent/MCP transcript text. parseAgentVtt strips complete markup tags from the raw cue before decoding entities, so escaped speech such as <value> survives and unfinished literal text such as Before <script is kept, matching the transcript view and exports. The existing agent-api.test.ts expectation for the unterminated case was updated accordingly; complete tags such as <script>…</script> are still removed.

Validation on 50ae1be: bun run typecheck passes; full web suite 3,461 passed / 28 skipped; scoped Biome clean. Greptile 5/5 on 50ae1be.

Confidence Score: 5/5

The PR appears safe to merge based on the reviewed changes.

Summary

This PR requests speaker diarization for full-recording transcription and carries speaker labels through captions, transcript editing, exports, and agent API round-trips. It makes provisional live transcription opt-in and validates that translations retain cue structure and speaker tags.

• Both previous Greptile findings are resolved and fixed in the current code.

_{Reviews (5) · Last reviewed commit: "fix: keep unfinished literal text in age..." · Reviewed by Greptile}

[richiemcilroy]

Thanks for the thorough testing here. We require Greptile 5/5 on the latest commit and green CI before review, so please rebase on current main (the branch is far behind) and trigger a Greptile re-review on the angle-bracket fix.

[superagent-security]

Superagent found 1 security concern(s).

[pavzagor]

@greptileai review

[pavzagor]

@greptileai review

[greptile-apps]

Want your agent to iterate on Greptile's feedback? Try greploops.

[pavzagor]

@greptileai review

[pavzagor]

@greptileai review

---

Generated by Claude Code

[pavzagor]

@richiemcilroy