Skip to content

Commit 573557a

Browse files
Brooooooklynclaude
andauthored
fix(deploy): match Cloudflare's Worker size limit on Void Cloud (#439)
## Problem Void Cloud refused Workers that Cloudflare accepts. The napi.rs deploy failed with: ``` Deploy failed (HTTP 413): request body exceeds 67108864 bytes ``` `@cloudflare/vite-plugin` 1.64 writes Worker source maps by default. The CLI sent every Worker module, as base64, inside the deploy request, so the maps counted against the API's 64 MiB body cap. ``` Cloudflare Void Cloud before this PR code limit 64 MiB uncompressed JSON body ≤ 64 MiB, as base64 → ≈ 48 MiB raw, maps included source maps not counted (15 MB gz each) counted per module no limit 32 MiB napi.rs: 26.8 MB code ✓ accepted ✗ 413 (≈ 67.3 MB body) + 23.7 MB maps ``` Raising the cap does not fix it. The API Worker has 128 MB per isolate and held about two full copies of the modules: one while it parsed the body, and one in the `FormData` it sent to Cloudflare. ## What this PR changes Worker modules now take the same path as static assets. The API streams them to Cloudflare and never holds the whole bundle. ``` CLI API (request) R2 │ POST deploy/worker-upload-urls │ │ │ {modules:[name,size,sha256]} ──▶│ code without maps ≤ 64 MiB? │ │ │ no → 413 worker_too_large │ │◀── {id, urls[]} ──────────────────│ presigned PUT, size signed │ │ PUT module i ─────────────────────┼──────────────────────────────────▶│ deployment-inputs/{project}/{id}/{i} │ POST deploy {workerInput} ───────▶│ one LIST: every module present, │ │ │ exact size │ │ │ queue {deploymentId} │ ▼ queue: one module open at a time R2 ──▶ size + SHA-256 check ──▶ multipart body ──▶ Cloudflare (FixedLengthStream; a mismatch stops the body before the closing boundary, so Cloudflare gets no complete form) ``` Old clients keep working: ``` CLI ──GET /.well-known/void-platform──▶ directWorkerUpload: true ? yes → direct path (above) no → inline workerFiles, as before (temporary, see Rollout) no usable answer → ask once more, then decide ``` A finalize retry with the same idempotency key finds the admitted operation before the upload is listed. It gets the receipt even after the operation's cleanup deleted the modules. ## Decisions | Question | Decision | Why | | --- | --- | --- | | Where the 64 MiB rule lives | API only, when it issues upload URLs | One place to update when Cloudflare changes it. A Worker that is too large fails before any byte is uploaded. | | Source maps | No Void limit | Cloudflare counts them separately and enforces its own 15 MB gzipped limit. | | Integrity | SHA-256 per module, checked while streaming | One R2 read per module, no buffering. A changed module never reaches Cloudflare. | | Size | `Content-Length` signed into each PUT URL, checked again by one LIST at finalize | R2 enforces the declared size. One LIST costs less than a HEAD per module (napi.rs has 668 modules). | | Inline path for old CLIs | Kept, with its old caps | Its memory limits still apply. Its 413 text now names Worker modules and says to upgrade the CLI, instead of blaming assets. | | Custom Durable Object snapshots | Copied and restored as streams | They re-upload the user Worker, so they need the same memory bound. | ## Rollout ``` merge ──▶ staging deploys (main) ──▶ prod (fast-forward prod) ──▶ npm release X │ old CLIs ─────────── inline path keeps working ─────────────────────┤ new CLI + old platform (1.1.11) ── inline fallback ─────────────────┤ new CLI + new platform ─────────── direct path ─────────────────────┘ ▼ later: MIN_PLATFORM_VERSION = X, delete the inline fallback ``` ## Known limits - Connected apps that publish with OAuth approval still send inline `workerFiles`. The approval digest covers inline bytes only, so they keep the 64 MiB body limit. Changing that needs module digests in the approved artifact. That is a product decision, so this PR leaves it alone. - Until `MIN_PLATFORM_VERSION` reaches the release that ships this, the new CLI keeps the inline fallback for platforms without `directWorkerUpload`. - R2 does not check SHA-256 at PUT time. Wrong bytes with the right size fail the deploy at the Cloudflare upload step, not at finalize. ## Testing | Check | Result | | --- | --- | | `vp run check` | 33/33 | | `vp test run` | 712 files, 13,583 tests pass | | Live Cloudflare upload from workerd (Miniflare, local R2), napi.rs Worker: 668 modules, 26.8 MB code + 23.7 MB maps | Cloudflare returned 200 and created the script in 9.6 s. Test script deleted. | | Same upload with one module changed by one byte, same size | The stream stopped with an integrity error. Cloudflare created no script. | | New tests | upload-URL route (limit, maps not counted, deploy-token access), finalize with `workerInput` (queued and inline execution, missing or wrong-size module, both fields sent, idempotent retry, reused upload id), streamed multipart (exact bytes, escaped names, mismatch, retry re-opens streams), Durable Object snapshot streaming, CLI path selection and request shapes | 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
1 parent 5802901 commit 573557a

35 files changed

Lines changed: 2683 additions & 319 deletions

‎packages/deploy-core/src/artifact.ts‎

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,43 @@ export type DeploySourceMetadata = {
2828
gitRef: string | null;
2929
};
3030

31-
/** JSON payload accepted by the managed deployment API. */
31+
/** One Worker module uploaded to R2 before the deployment is finalized. */
32+
export type DeployWorkerModule = {
33+
/** Module path relative to the Worker output, the same key `workerFiles` uses. */
34+
name: string;
35+
/** Byte length of the module. */
36+
size: number;
37+
/** SHA-256 of the module bytes as 64 lowercase hex characters. */
38+
sha256: string;
39+
};
40+
41+
/** Request body of `POST /projects/:id/deploy/worker-upload-urls`. */
42+
export type DeployWorkerUploadRequest = {
43+
modules: Array<DeployWorkerModule>;
44+
};
45+
46+
/** Upload URLs for each requested module, in request order. */
47+
export type DeployWorkerUploadResponse = {
48+
id: string;
49+
urls: Array<{ url: string; headers: Record<string, string> }>;
50+
};
51+
52+
/** Worker modules already uploaded through `worker-upload-urls`, named by its upload `id`. */
53+
export type DeployWorkerInput = {
54+
id: string;
55+
modules: Array<DeployWorkerModule>;
56+
};
57+
58+
/**
59+
* JSON payload accepted by the managed deployment API. Worker modules travel
60+
* either inline as `workerFiles` or uploaded beforehand as `workerInput`,
61+
* never both.
62+
*/
3263
export type DeployUploadPayload = {
3364
manifest: DeployManifest;
3465
assetManifest: Record<string, DeployAssetManifestEntry>;
3566
workerFiles?: Record<string, string>;
67+
workerInput?: DeployWorkerInput;
3668
} & DeploySourceMetadata & {
3769
preflight: 'true' | null;
3870
};
@@ -61,15 +93,31 @@ export function createDeployArtifact(input: {
6193
};
6294
}
6395

64-
/** Convert the canonical artifact into the stable hosted-platform wire shape. */
96+
/**
97+
* Convert the canonical artifact into the stable hosted-platform wire shape.
98+
* Pass `workerInput` when the Worker modules were uploaded beforehand; the
99+
* artifact must then carry no inline `workerFiles`.
100+
*/
65101
export function createDeployUploadPayload(
66102
artifact: DeployArtifact,
67103
metadata: DeploySourceMetadata & { preflight: 'true' | null },
104+
workerInput?: DeployWorkerInput,
68105
): DeployUploadPayload {
106+
if (workerInput && artifact.workerFiles) {
107+
throw new Error('A deployment sends its Worker modules inline or as uploads, not both.');
108+
}
69109
return {
70110
manifest: { ...artifact.manifest, bindings: { ...artifact.manifest.bindings } },
71111
assetManifest: { ...artifact.assetManifest },
72112
...(artifact.workerFiles ? { workerFiles: { ...artifact.workerFiles } } : {}),
113+
...(workerInput
114+
? {
115+
workerInput: {
116+
id: workerInput.id,
117+
modules: workerInput.modules.map((module) => ({ ...module })),
118+
},
119+
}
120+
: {}),
73121
...metadata,
74122
};
75123
}

‎packages/deploy-core/src/index.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -338,6 +338,10 @@ export type {
338338
DeployAssetUploadRequest,
339339
DeploySourceMetadata,
340340
DeployUploadPayload,
341+
DeployWorkerInput,
342+
DeployWorkerModule,
343+
DeployWorkerUploadRequest,
344+
DeployWorkerUploadResponse,
341345
} from './artifact.ts';
342346
export {
343347
DEPLOY_ARTIFACT_VERSION,

‎packages/deploy-core/test/unit/artifact.test.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,4 +57,38 @@ describe('deployment artifacts', () => {
5757
});
5858
expect(payload).not.toHaveProperty('artifactVersion');
5959
});
60+
61+
it('names uploaded Worker modules by workerInput instead of carrying their bytes', () => {
62+
const metadata = {
63+
source: 'cli',
64+
commit: null,
65+
repositoryUrl: null,
66+
commitUrl: null,
67+
gitRef: null,
68+
preflight: null,
69+
};
70+
const workerInput = {
71+
id: 'upload-1',
72+
modules: [{ name: 'index.js', size: 18, sha256: 'a'.repeat(64) }],
73+
};
74+
const payload = createDeployUploadPayload(
75+
createDeployArtifact({ manifest, assetManifest: {} }),
76+
metadata,
77+
workerInput,
78+
);
79+
80+
expect(payload.workerInput).toEqual(workerInput);
81+
expect(payload).not.toHaveProperty('workerFiles');
82+
expect(() =>
83+
createDeployUploadPayload(
84+
createDeployArtifact({
85+
manifest,
86+
assetManifest: {},
87+
workerFiles: { 'index.js': 'ZXhwb3J0IGRlZmF1bHQge30=' },
88+
}),
89+
metadata,
90+
workerInput,
91+
),
92+
).toThrow('not both');
93+
});
6094
});

‎packages/platform/src/discovery.ts‎

Lines changed: 28 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ export type PlatformDiscoveryDocument = Readonly<{
2929
capabilities: PlatformCapabilities;
3030
displayName: string;
3131
dashboardUrl?: string;
32+
/**
33+
* The API accepts Worker modules uploaded to R2 through
34+
* `POST /projects/:id/deploy/worker-upload-urls`. Older platforms omit it.
35+
*/
36+
directWorkerUpload?: boolean;
3237
installationId: string | null;
3338
platformVersion: string;
3439
proxyUrl: string;
@@ -54,11 +59,24 @@ function normalizeOrigin(value: string, name: 'apiUrl' | 'proxyUrl'): string {
5459
return url.origin;
5560
}
5661

62+
type PlatformDiscoveryInput = Omit<
63+
PlatformDiscoveryDocument,
64+
'directWorkerUpload' | 'schemaVersion' | 'usageProtocolVersion'
65+
> & {
66+
usageProtocolVersion?: 0 | 1 | 2;
67+
};
68+
69+
/**
70+
* Builds the document a platform serves at `/.well-known/void-platform`.
71+
* Every platform built from this source accepts direct Worker uploads.
72+
*/
5773
export function createPlatformDiscoveryDocument(
58-
input: Omit<PlatformDiscoveryDocument, 'schemaVersion' | 'usageProtocolVersion'> & {
59-
usageProtocolVersion?: 0 | 1 | 2;
60-
},
74+
input: PlatformDiscoveryInput,
6175
): PlatformDiscoveryDocument {
76+
return { ...normalizeDiscoveryDocument(input), directWorkerUpload: true };
77+
}
78+
79+
function normalizeDiscoveryDocument(input: PlatformDiscoveryInput): PlatformDiscoveryDocument {
6280
if (!input.displayName.trim()) {
6381
throw new Error('displayName must not be empty.');
6482
}
@@ -195,7 +213,11 @@ export function parsePlatformDiscoveryDocument(value: unknown): PlatformDiscover
195213
};
196214
const apiUrl = string('apiUrl');
197215
const proxyUrl = string('proxyUrl');
198-
return createPlatformDiscoveryDocument({
216+
const directWorkerUpload = document.directWorkerUpload;
217+
if (directWorkerUpload !== undefined && typeof directWorkerUpload !== 'boolean') {
218+
throw new Error('directWorkerUpload must be a boolean.');
219+
}
220+
const parsed = normalizeDiscoveryDocument({
199221
apiUrl,
200222
applicationDomain: string('applicationDomain'),
201223
...(document.applicationRouting !== undefined
@@ -213,6 +235,8 @@ export function parsePlatformDiscoveryDocument(value: unknown): PlatformDiscover
213235
proxyUrl,
214236
usageProtocolVersion: parseUsageProtocolVersion(document.usageProtocolVersion),
215237
});
238+
// Report only what the platform said: an older platform omits the field.
239+
return directWorkerUpload === undefined ? parsed : { ...parsed, directWorkerUpload };
216240
}
217241

218242
function parseAuthentication(value: unknown): PlatformAuthenticationDiscovery {

‎packages/platform/test/unit/auth-discovery.test.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,3 +110,24 @@ describe('authentication discovery compatibility', () => {
110110
expect(JSON.stringify(parsed.authentication)).not.toContain('internal.example.com');
111111
});
112112
});
113+
114+
describe('direct Worker upload discovery', () => {
115+
it('advertises direct Worker uploads on every document this platform serves', () => {
116+
for (const input of [base, modern]) {
117+
const discovery = createPlatformDiscoveryDocument(input);
118+
expect(discovery.directWorkerUpload).toBe(true);
119+
expect(parsePlatformDiscoveryDocument(discovery)).toEqual(discovery);
120+
}
121+
});
122+
123+
it('reports only what an older platform said', () => {
124+
const { directWorkerUpload: _, ...older } = createPlatformDiscoveryDocument(base);
125+
expect(parsePlatformDiscoveryDocument(older)).not.toHaveProperty('directWorkerUpload');
126+
expect(
127+
parsePlatformDiscoveryDocument({ ...older, directWorkerUpload: false }).directWorkerUpload,
128+
).toBe(false);
129+
expect(() => parsePlatformDiscoveryDocument({ ...older, directWorkerUpload: 'yes' })).toThrow(
130+
'directWorkerUpload must be a boolean.',
131+
);
132+
});
133+
});

‎packages/void/src/cli/client.ts‎

Lines changed: 91 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,14 @@ import {
1313
type ProjectPermission,
1414
} from './platform-errors.ts';
1515
import { normalizePlatformApiUrl } from './platform-registry.ts';
16-
import type { DeployAssetUploadRequest, DeployEvent, DeployUploadPayload } from './types.ts';
16+
import type {
17+
DeployAssetUploadRequest,
18+
DeployEvent,
19+
DeployUploadPayload,
20+
DeployWorkerModule,
21+
DeployWorkerUploadRequest,
22+
DeployWorkerUploadResponse,
23+
} from './types.ts';
1724

1825
// Null omits `X-Void-CLI-Version`: the platform answers an unparseable version with 426.
1926
function readCliVersion(): string | null {
@@ -34,21 +41,22 @@ export const MIN_PLATFORM_VERSION = '1.1.11';
3441
const RELEASE_VERSION = /^(\d+)\.(\d+)\.(\d+)(-[0-9A-Za-z.-]+)?(\+[0-9A-Za-z.-]+)?$/;
3542

3643
/**
37-
* Version checks keyed by platform base URL. Every outcome (pass, inconclusive,
38-
* or too old) is kept for the rest of the process, so polling commands probe
39-
* discovery once.
44+
* Discovery probes keyed by platform base URL. Every outcome (pass,
45+
* inconclusive, or too old) is kept for the rest of the process, so polling
46+
* commands probe discovery once.
4047
*/
41-
const platformVersionChecks = new Map<string, Promise<void>>();
48+
const platformVersionProbes = new Map<string, Promise<PlatformVersionProbe>>();
4249

4350
/**
4451
* What public discovery says about the platform version:
45-
* - `version`: the reported `platformVersion`
52+
* - `version`: the reported `platformVersion`, and whether the platform
53+
* accepts Worker modules uploaded straight to R2
4654
* - `missing`: discovery answered 404, so the platform predates discovery
4755
* - `unknown`: no usable answer (network error, timeout, disabled platform,
4856
* Access login page, or an invalid document)
4957
*/
5058
type PlatformVersionProbe =
51-
| { kind: 'version'; version: string }
59+
| { kind: 'version'; version: string; directWorkerUpload: boolean }
5260
| { kind: 'missing' }
5361
| { kind: 'unknown' };
5462

@@ -661,19 +669,34 @@ export class PlatformClient {
661669

662670
/**
663671
* Checks the platform version once per process and base URL. Concurrent
664-
* first requests share one check, and its outcome is kept for later requests.
672+
* first requests share one probe, and its outcome is kept for later requests.
665673
*/
666-
private ensureSupportedPlatform(): Promise<void> {
667-
const apiUrl = this.baseUrl;
668-
const cached = platformVersionChecks.get(apiUrl);
674+
private async ensureSupportedPlatform(): Promise<void> {
675+
assertSupportedPlatform(this.baseUrl, await this.platformProbe());
676+
}
677+
678+
private platformProbe(): Promise<PlatformVersionProbe> {
679+
const cached = platformVersionProbes.get(this.baseUrl);
669680
if (cached) {
670681
return cached;
671682
}
672-
const check = this.probePlatformVersion().then((probe) =>
673-
assertSupportedPlatform(apiUrl, probe),
674-
);
675-
platformVersionChecks.set(apiUrl, check);
676-
return check;
683+
const probe = this.probePlatformVersion();
684+
platformVersionProbes.set(this.baseUrl, probe);
685+
return probe;
686+
}
687+
688+
/**
689+
* Whether discovery says the platform accepts Worker modules uploaded to R2.
690+
* False when discovery omits the field or gave no usable answer.
691+
*/
692+
async supportsDirectWorkerUpload(): Promise<boolean> {
693+
let probe = await this.platformProbe();
694+
if (probe.kind === 'unknown') {
695+
// One failed probe must not send a large Worker inline. Ask again,
696+
// without replacing the outcome the version check keeps.
697+
probe = await this.probePlatformVersion();
698+
}
699+
return probe.kind === 'version' && probe.directWorkerUpload;
677700
}
678701

679702
/** Reads the platform version from public discovery, without credentials. */
@@ -695,7 +718,11 @@ export class PlatformClient {
695718
}
696719
try {
697720
const discovery = await readPlatformDiscoveryResponse(this.baseUrl, response);
698-
return { kind: 'version', version: discovery.platformVersion };
721+
return {
722+
kind: 'version',
723+
version: discovery.platformVersion,
724+
directWorkerUpload: discovery.directWorkerUpload === true,
725+
};
699726
} catch {
700727
return { kind: 'unknown' };
701728
}
@@ -1117,6 +1144,53 @@ export class PlatformClient {
11171144
return json.urls;
11181145
}
11191146

1147+
/**
1148+
* Requests one presigned R2 PUT per Worker module, in `modules` order. The
1149+
* platform refuses a Worker over Cloudflare's size limit here (413
1150+
* `worker_too_large`), before any bytes are uploaded.
1151+
*/
1152+
async requestWorkerUploadUrls(
1153+
projectId: string,
1154+
modules: Array<DeployWorkerModule>,
1155+
): Promise<DeployWorkerUploadResponse> {
1156+
const url = `${this.baseUrl}/projects/${projectId}/deploy/worker-upload-urls`;
1157+
this.cliLog?.info('worker_upload_urls_post', { url, moduleCount: modules.length });
1158+
1159+
const request: DeployWorkerUploadRequest = { modules };
1160+
const res = await this.request(url, {
1161+
method: 'POST',
1162+
headers: {
1163+
'Content-Type': 'application/json',
1164+
...this.authHeaders,
1165+
},
1166+
body: JSON.stringify(request),
1167+
});
1168+
this.cliLog?.info('worker_upload_urls_response', {
1169+
status: res.status,
1170+
cfRay: res.headers.get('cf-ray'),
1171+
});
1172+
1173+
if (!res.ok) {
1174+
await throwPlatformApiError('Failed to request worker upload URLs', res, 'deploy');
1175+
}
1176+
1177+
// Module i uploads to urls[i], so a short or malformed list must not start.
1178+
const json = (await res.json()) as Partial<DeployWorkerUploadResponse> | null;
1179+
if (
1180+
typeof json?.id !== 'string' ||
1181+
!json.id ||
1182+
!Array.isArray(json.urls) ||
1183+
json.urls.length !== modules.length ||
1184+
!json.urls.every(
1185+
(entry) =>
1186+
typeof entry?.url === 'string' && typeof entry.headers === 'object' && !!entry.headers,
1187+
)
1188+
) {
1189+
throw new Error('Invalid worker upload URLs from platform.');
1190+
}
1191+
return { id: json.id, urls: json.urls };
1192+
}
1193+
11201194
async *finalizeDeploy(
11211195
projectId: string,
11221196
payload: ManifestOnlyDeployPayload,

0 commit comments

Comments
 (0)