perf: prepare recording sources during background uploads - #2255
Merged
Conversation
Member
Author
|
hey @greptileai, please re-review the PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Desktop Instant recordings previously waited until Stop to preserve every uploaded source fragment. Both desktop uploaders now prepare closed, successfully uploaded fragments in bounded background batches so finalization can reuse those validated copies.
Preparation is optional, owner-scoped, and cancelled without delaying completion. It retries each fragment at most twice, backs off outages, stops scheduling after finalization starts, and falls back to the existing source commitment path when preparation is missing or invalid. Source identity, audio, completion proof, and final media verification requirements remain unchanged. Initialization files are still captured after Stop because they can change during recording.
Copies stay within S3 or Google Drive. Vercel receives metadata only: at most 32 references per request, one request every 30 seconds, four concurrent copies, and bounded execution. No schema migration is required; older clients and servers remain compatible. Final muxing and full verification still happen on the media worker, so this does not promise constant-time finalization.
Live rollout checks exposed S3 connection exhaustion during source copying. Storage access previously constructed fresh clients and keep-alive pools for every checkpoint. The change reuses default clients and a scoped connection pool for default and custom buckets, bounded to 50 connections per host and 128 per protocol. Idle connections expire without adding an active-copy timeout, and custom credentials are still resolved on every access. A real HTTP regression test reproduced 320 connections across 40 checkpoints before the fix and at most 16 afterward; concurrent recording and credential-rotation checks also pass.
Validation:
Updated desktop binaries still require packaged native recording/upload verification before release. Production replay timings exclude provider/network latency; post-release readiness and cost measurements remain necessary.
The PR appears safe to merge; the latest connection-pool changes address the reported resource exhaustion without introducing an actionable regression.
Summary