Skip to content

perf(infra): encode long 1080p-and-below Infra videos' adaptive stream in slices - #4238

Merged
richiemcilroy merged 18 commits into
mainfrom
perf/infra-sliced-hls-v2-audio
Nov 14, 2026
Merged

richiemcilroy merged 18 commits into
mainfrom
perf/infra-sliced-hls-v2-audio

Conversation

@richiemcilroy

@richiemcilroy richiemcilroy commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Cap Infra videos of 20 minutes or more whose ladder tops out at 1080p or lower now have their adaptive stream encoded in time slices across the farm. Such a video is playable from its first slice, and the whole ladder finishes in about half the single-job time. Every other video keeps the single ladder job exactly as before: shorter ones, 1440p and 4K ladders, and everything while INFRA_SLICED_HLS=off.

sequenceDiagram
    participant W as web (workflow)
    participant LB as load balancer
    participant A as worker A
    participant B as worker B
    participant C as worker C
    W->>LB: POST /video/process (MP4, infra)
    LB->>A: MP4 runs on its own
    W->>LB: POST /video/ladder range 0 (audio + poster), up to 4 tries
    LB-->>A: 503, A is making an Infra MP4
    LB->>B: slice 0
    W->>LB: slices 1, 2 (2 tries each)
    LB->>C: slice 1
    Note over W: each finished slice webhook sends the next one,<br/>the last one writes the whole ladder
Loading

How it works

  • Plan (apps/web/lib/developer-video-slices.ts): a 20 s first slice, a ramp slice, then even slices sized so three workers finish together in four rounds (at least 300 s each, at most 14 slices). Only videos of 20 minutes or more are sliced (SLICED_HLS.minDurationSeconds), and only when the ladder tops out at 1080p or lower (SLICED_LADDER_TIERS). Both limits come from the measurements below.
  • Dispatch (apps/web/lib/developer-video-processing.ts):
    • the MP4 goes out first, then slice 0 on its own with 4 tries, then the rest with 2 tries;
    • up to 3 slices are in flight;
    • a finished slice's webhook sends the next one immediately.
  • Admission (apps/media-server/src/lib/job-manager.ts):
    • a slice keeps half the worker's slots free;
    • a slice counts against the one-heavy-ladder-per-worker limit;
    • a slice is refused on a worker running an Infra MP4, so the MP4 runs alone as it does beside a whole-video ladder.
    • Whole-video ladders, MP4s and Cap's own jobs are admitted exactly as before.
  • Audio: slice 0 carries the whole video's audio. Stereo AAC-LC at 44.1/48 kHz up to 192 kbps is copied (sourceAudioCopiesToHls) instead of encoded, which takes seconds instead of about 90 s for an hour. Anything else is encoded as before.
  • Playback (apps/web/lib/developer-video-playback.ts):
    • while slices come in, playlists are built from the contiguous finished slices, cached for 2 s;
    • once every slice is in, the whole ladder's playlists are written next to the media, and the video is served like any single-job ladder.
  • Ready:
    • the video is ready at whichever comes first, the MP4 or the first slice, so short videos stay ready at the MP4 time;
    • video.ready still fires once the whole ladder is in.
  • Failure handling:
    • a failing slice retries up to the part attempt limit;
    • a slice whose job keeps vanishing gives up the slices;
    • an encoder without the ladder route abandons the slices and falls back to one whole-video job.
  • Metering (apps/web/lib/infra-delivery-log.ts): slice media files (<rendition>-<slice>-<attempt>.m4s) resolve to exact HLS seconds through the joined playlist, or through the slice's own playlist before that.
  • Kill switch: INFRA_SLICED_HLS=off keeps every video on the single ladder job. Videos already slicing finish their slices.

Benchmarks

Three L4 workers (g6.4xlarge, the farm's image, tuning and admission), with production's dispatch order and retry timing. Each video was run on main's path and on this branch back to back. Only the rows marked sliced are sliced by this PR as merged. The others were measured with slicing forced on, and are why the limits exist.

Source Sliced as merged Playable: main Playable: sliced Adaptive stream done: main Adaptive stream done: sliced MP4: main MP4: sliced
720p, 5 min no (under 20 min) 2.0 s 2.0 s 10.6 s 8.2 s 2.0 s 2.0 s
1080p, 5 min no (under 20 min) 5.0 s 5.0 s 18.4 s 13.4 s 5.0 s 5.0 s
4K iPhone HEVC, 5 min no 49.2 s 5.2 s 51.6 s 40.2 s 49.2 s 49.2 s
720p, 21 min sliced 5.9 s 6.1 s 36.4 s 28.6 s 5.9 s 6.1 s
4K H.264, 21 min no (4K) 10.2 s 5.1 s 169 s 75.3 s 10.2 s 10.3 s
1080p, 1 h sliced 52.8 s 9.5 s 219 s 103 s 52.8 s 52.9 s
4K H.264, 1 h no (4K) 30.3 s 6.7 s 506 s 223 s 30.3 s 30.3 s
4K iPhone HEVC, 1 h (13 GB) no (4K) 580 s 14.8 s 599 s 397 s 580 s 579 s
  • Playable is the earlier of the MP4 and, when sliced, the first slice.
  • The 720p 21 min source's audio is not stereo AAC-LC, so its first slice encodes the whole track (28.6 s). That video is playable from its MP4 on both paths.
  • In a later repeat, the 720p 21 min MP4 was 5.91 / 5.86 s on main and 5.87 / 5.83 s sliced.

Cap's own jobs beside Infra work

A regular Cap job (a 1 min 4K iPhone recording) arrives every 8 s, spread over the 3 workers, while one Infra video processes. On an idle farm each regular job takes 5.32 to 5.38 s. Each pair was run back to back; repeats are separated by /, and the 1 h 4K pairs were repeated in reverse order (sliced first). No regular job failed in any run.

Infra video Sliced as merged Path Regular jobs Slowest Extra seconds, summed over all regular jobs Infra adaptive stream done
720p, 21 min sliced main 5 / 5 5.40 / 5.38 s 0.0 / 0.0 s 37.3 / 37.3 s
720p, 21 min sliced sliced 4 / 4 5.40 / 5.38 s 0.1 / 0.0 s 27.8 / 27.5 s
1080p, 1 h sliced main 26 5.77 s 1.1 s 205 s
1080p, 1 h sliced sliced 14 5.72 s 0.7 s 106 s
4K iPhone HEVC, 5 min no main 7 / 7 11.96 / 11.94 s 16.3 / 16.1 s 52.6 / 52.5 s
4K iPhone HEVC, 5 min no sliced 7 / 7 10.61 / 10.54 s 19.1 / 18.6 s 35.8 / 36.2 s
4K H.264, 21 min no main 23 / 23 11.50 / 11.27 s 40.6 / 39.6 s 179 / 179 s
4K H.264, 21 min no sliced 10 / 10 11.57 / 11.75 s 40.7 / 40.6 s 79.4 / 79.0 s
4K H.264, 1 h no main 67 / 67 11.36 / 11.37 s 124.7 / 125.8 s 534 / 534 s
4K H.264, 1 h no sliced 31 / 29 11.65 / 11.57 s 108.7 / 128.7 s 245 / 231 s
4K iPhone HEVC, 1 h no main 80 / 80 12.17 / 12.05 s 281 / 281 s 640 / 640 s
4K iPhone HEVC, 1 h no sliced 74 / 74 12.80 / 12.50 s 265 / 272 s 409 / 419 s
  • 720p and 1080p ladders: sliced, they cost Cap's jobs nothing measurable. The slowest job and the total are the same as main's or lower.
  • 4K ladders: sliced, the slowest Cap job was 0.07 to 0.63 s higher than main's in every pair, including both reverse-order repeats. The median of the slowed jobs matched (11.20 to 11.23 s vs 11.19 to 11.31 s on 4K H.264).
    • It is not the slices' source downloads, their start-ups, or sharing a worker with the MP4: Cap jobs overlapping those were faster.
    • It comes from Cap jobs running beside a slice while it encodes.
    • 4K and 1440p ladders therefore stay on the single job until that is found and fixed.
  • Under 20 minutes: a 5 min 4K video in slices cost Cap's jobs about 3 s more in total than one whole job, since each slice pays a job's fixed start cost. Videos under 20 minutes stay on the single job.
  • MP4 times are within 0.2 s of main's on every row, since slices are kept off a worker that is making an Infra MP4.

Retained slices after a fallback

When viewers were already playing from slices and the video then falls back to one whole-video job, the slices' media and their own playlists are kept. This way:

  • players that loaded the slice playlists keep working, through supersededSlicedHls in playback;
  • delivery metering still counts those requests' seconds exactly.

The kept files are left out of the video's storedBytes on purpose. They exist only because our encode fell back, so the customer is not charged storage for them. bytes.hls is the whole job's ladder, as for any other single-job video.

RetriggerView in GreptileConfidence Score: 3/5

Fix the slice worker limit and workflow dispatch ordering before merging.

Findings

  1. P1 Slices bypass the worker limit ▶
  2. P1 Workflow bypasses MP4-first ordering ▶
Fix with agent prompt
### Issue 1
apps/media-server/src/routes/video.ts:1066-1069
The route sets `infraHeavyLadder` only for 1440p and 2160p, but the new slicing path selects ladders at 1080p or below. On a four-slot worker, two slices can therefore run together, and neither blocks a heavy whole-video ladder. This breaks the promised worker limit and lets several ladder encodes compete on one worker. Count Infra slices against that limit regardless of their resolution.

### Issue 2
apps/web/lib/developer-video-processing.ts:900-902
`dispatchDeveloperVideoPart` starts slices without waiting for the MP4 to be accepted. The workflow calls this function for both pending parts concurrently, bypassing the MP4-first ordering in `dispatchDeveloperVideoParts`. If a slice reaches a worker first, that worker can then accept the MP4 too: admission blocks slices beside an existing MP4, but not the reverse. The jobs then compete on the same worker instead of keeping the MP4 separate. Preserve the MP4-first ordering in the workflow path as well.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds sliced adaptive-stream encoding for Infra videos of at least 20 minutes whose ladder tops out at 1080p.

  • Long, lower-resolution Infra videos encode their adaptive stream in slices.
  • Viewers can play a growing adaptive stream before every slice is finished.
  • The first slice can copy compatible audio instead of encoding it.
  • Slices leave worker capacity free and avoid workers encoding an Infra MP4.
  • Delivery metering counts seconds from slice media files.

Reviews (9) · Last reviewed commit: "Merge branch 'main' into perf/infra-slic..." · Reviewed by Greptile

Richie McIlroy added 8 commits November 13, 2026 21:27
Ports perf/infra-sliced-hls (ee2d6ef..8400435) onto current main.
Videos of two minutes or more encode their HLS ladder as time slices on
several workers: a 20 s first slice so the video plays in seconds, then
slices growing six times, then the tail split in three. A growing
playlist is served from the contiguous finished slices, the whole ladder
is written once every slice is in, and a slice that fails twice falls
back to one whole-file job.

Reconciled with main:
- Slice jobs carry infra: true, so they get the 1920 Infra poster and
  the Infra admission rule. A slice reserves max(reserved slots, half
  the worker) inside canAcceptNewInfraVideoProcess and still counts as a
  heavy ladder at 1440p and 2160p. canAcceptNewVideoProcess is main's.
- The first dispatch after prepare starts the MP4 and the slices and
  never a whole HLS job beside them.
- A slice turned away busy stamps busyAt, and is not reported busy while
  other slices of the video encode, so the busy-wait deadline reset only
  applies when nothing of the video is encoding.
- A slice job whose start lands after a delete is cancelled, and video
  and app deletes cancel running slice jobs.
- Growing playlists go through main's subtitles, short segment and
  text/plain media playlist handling and carry the cache tags.
- Delivery metering counts slice media files under their rendition with
  exact seconds, from the whole ladder's playlist or, while it grows,
  the slice's own playlist.
…parity test

A static import reached the media server's Bun-only input policy and brought
Bun's global types into the web app's typecheck, which changed fetch for
every file and added 44 errors.
The first slice of a sliced Infra video carries the whole video's audio,
so it finished only when one AAC encode of the full length did: about
90 s for an hour, which held the first playable slice back by that much.
Stereo AAC-LC at 44.1 or 48 kHz and up to 192 kbps is now copied, which
reads the file once and takes a few seconds. Every other source is
encoded as before. Sliced master playlists allow 192 kbps of audio in
BANDWIDTH.
…h together

Thirds of a long video are long slices, and while one is still encoding
the other workers sit idle. Past 450 s a slice, the rest of the video goes
out in four rounds of even slices per worker, at least 300 s each, after
the first slice and the ramp. Videos up to about 25 minutes keep the plan
they had. On three L4 workers an hour of 4K finished in 206 s instead of
242 s with even slices.
… MP4

A slice that lands beside an Infra MP4 slows the MP4 and pulls the same
source at the same time, so the MP4 finished later than it does beside a
whole-video ladder and the first slice could wait half a minute for its
download. Infra MP4 jobs are now marked, and a slice turns a worker making
one away. Whole-video ladders and every other job are admitted as before.
The MP4 now reaches a worker first, so the slices sent after it find that
worker taken and land on the others.
The first slice makes the video playable, and sent together with the
others it could reach the MP4's worker and then a worker already taking
another slice, and wait for the next pass. It now goes out on its own
first and asks up to four workers before it waits.
@sandchest-com

sandchest-com Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Sandchest

Sandchest reviewed e0c6965: nothing exploitable found in this change.

Sandchest reviews every change for vulnerabilities an attacker could use · 12 credits · Review details

@vercel

vercel Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
cap-status Skipped Skipped Nov 14, 2026 6:06am UTC

Comment thread apps/web/lib/developer-video-state.ts Resolved

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found 1 potential issue (1 medium).

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 02:51 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 02:53 Inactive
@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 02:56 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 4 other findings in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 04:23 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 1 other finding in Capy.

Open in Capy

@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 05:59 Inactive
@richiemcilroy

Copy link
Copy Markdown
Member Author

hey @greptileai, please re-review the PR

@richiemcilroy richiemcilroy changed the title perf(infra): encode long videos' adaptive stream in slices across the farm perf(infra): encode long 1080p-and-below Infra videos' adaptive stream in slices Nov 14, 2026

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

View 2 other findings in Capy.

Open in Capy

@vercel
vercel Bot temporarily deployed to Preview – cap-status November 14, 2026 06:06 Inactive

@capy-ai capy-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capy found no issues.

Open in Capy

@richiemcilroy
richiemcilroy merged commit 42c5d0b into main Nov 14, 2026
46 checks passed
Comment on lines +1066 to +1069
!canAcceptNewInfraVideoProcess(
infraHeavyLadder,
result.data.range !== undefined,
))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Slices bypass the worker limit

The route sets infraHeavyLadder only for 1440p and 2160p, but the new slicing path selects ladders at 1080p or below. On a four-slot worker, two slices can therefore run together, and neither blocks a heavy whole-video ladder. This breaks the promised worker limit and lets several ladder encodes compete on one worker. Count Infra slices against that limit regardless of their resolution.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/media-server/src/routes/video.ts
Line: 1066-1069

Comment:
**Slices bypass the worker limit**

The route sets `infraHeavyLadder` only for 1440p and 2160p, but the new slicing path selects ladders at 1080p or below. On a four-slot worker, two slices can therefore run together, and neither blocks a heavy whole-video ladder. This breaks the promised worker limit and lets several ladder encodes compete on one worker. Count Infra slices against that limit regardless of their resolution.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines 900 to +902
part: DeveloperVideoPart,
): Promise<DeveloperVideoDispatch> {
if (part === "hls") {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Workflow bypasses MP4-first ordering

dispatchDeveloperVideoPart starts slices without waiting for the MP4 to be accepted. The workflow calls this function for both pending parts concurrently, bypassing the MP4-first ordering in dispatchDeveloperVideoParts. If a slice reaches a worker first, that worker can then accept the MP4 too: admission blocks slices beside an existing MP4, but not the reverse. The jobs then compete on the same worker instead of keeping the MP4 separate. Preserve the MP4-first ordering in the workflow path as well.

Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/web/lib/developer-video-processing.ts
Line: 900-902

Comment:
**Workflow bypasses MP4-first ordering**

`dispatchDeveloperVideoPart` starts slices without waiting for the MP4 to be accepted. The workflow calls this function for both pending parts concurrently, bypassing the MP4-first ordering in `dispatchDeveloperVideoParts`. If a slice reaches a worker first, that worker can then accept the MP4 too: admission blocks slices beside an existing MP4, but not the reverse. The jobs then compete on the same worker instead of keeping the MP4 separate. Preserve the MP4-first ordering in the workflow path as well.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

This branch was successfully deployed

1 active and 1 inactive deployments
Preview – cap-web — e0c69650 Deployed Nov 14, 2026 by vercel[bot]
Preview – cap-status — e0c69650 Deployed Nov 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant