feat(api): direct-to-R2 asset upload (bump framework) (#137)

* feat(api/r2-presign): presigned R2 PUT URLs with SHA-256 enforcement

Adds packages/api/src/r2-presign.ts: a small module that generates
presigned PUT URLs for the direct-to-R2 asset upload flow described in
meta/design-docs/0068-direct-r2-asset-upload.md (Task 1).

The signed URL covers x-amz-checksum-sha256 in X-Amz-SignedHeaders, so
R2 verifies the uploaded body hashes to the declared digest at write
time. Tampering the header invalidates the signature, and a content
mismatch is rejected by R2 with HTTP 400 — no server-side rehash.

- Uses aws4fetch (signQuery + allHeaders) so Content-Type and the
  checksum header are both included in the canonical request.
- Pure function: takes credentials/options as parameters, reads no env.
  The endpoint that calls it (Task 2) loads R2_ACCESS_KEY_ID and
  R2_SECRET_ACCESS_KEY from env and threads them through.
- Documents the two new secrets in env.ts and wrangler.jsonc.
- Tests cover URL shape, hex-to-base64 conversion, X-Amz-Expires TTL,
  determinism via a now override, and that x-amz-checksum-sha256 is in
  X-Amz-SignedHeaders.

* docs(operations): document R2 presigning secrets

Add R2_ACCESS_KEY_ID and R2_SECRET_ACCESS_KEY to the API worker
secrets table and the fresh-environment provisioning checklist; they
are required by the new R2 presigning utility used for direct-to-R2
asset uploads in the deploy flow.

* feat(api/r2-presign): switch from SHA-256 to Content-MD5 for R2 upload integrity

Cloudflare R2's S3 compatibility layer documents Content-MD5 (RFC 1864)
as the supported PutObject checksum, while x-amz-checksum-sha256 is not
enforced and would silently no-op. The previous SHA-256 design would
have shipped a signed header R2 ignored, breaking the "R2 rejects
mismatched bytes at write time" guarantee on the wire.

Switching to Content-MD5 keeps the same architectural property with a
documented, supported wire contract:

- presignAssetPut takes md5 (32 hex) instead of sha256 (64 hex)
- header signed becomes Content-Md5, value is base64 of the raw 16-byte
  digest with standard padding (not URL-safe)
- aws4fetch's allHeaders: true keeps the header genuinely covered by
  the V4 signature, so R2's edge enforces it

MD5's weaker collision resistance is acceptable: the hash is purely an
upload-integrity primitive, content addressing stays on BLAKE3, and
project-scoped namespacing means any collision is exploitable only
against the attacker's own project. The no-overwrite-guard concern
from adversarial review is also closed, because R2 rejects any PUT
whose body MD5 doesn't match the declared header — replays and buggy
CLIs cannot clobber live keys with mismatched bytes.

Design doc 0068 updated end-to-end (assetManifest v2 schema, wire
contracts, Task 1-4 specs, Risks section). All 6 unit tests pass with
the empty-string MD5 fixture (d41d8cd98f00b204e9800998ecf8427e ->
1B2M2Y8AsgTpgAmY7PhCfg==).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(api/deploy): add direct-to-R2 upload endpoints + JSON-mode deploy

Task 2 of design doc 0068. Adds two endpoints to packages/api/src/routes/deploy.ts:

- POST /:id/deploy/upload-urls — issues batched presigned R2 PUT URLs
  (Content-Md5 signed, 10-min TTL) so the CLI can upload asset bytes
  directly to STATIC_ASSETS_R2 without going through the API Worker
  request body. Auth/ownership mirrors /deploy/preflight.
- POST /:id/deploy now branches on Content-Type at handler entry. JSON
  bodies route to handleManifestOnlyDeploy; multipart stays bit-for-bit
  unchanged for old CLIs. The JSON-mode handler validates the v2 asset
  manifest, head-checks each declared blake3 in R2 in bounded
  concurrency, then threads through the same post-validate pipeline as
  multipart (precheck-and-claim, dedupe short-circuit, status
  transitions, NDJSON event stream, routing switch, retention cleanup,
  placeholder reaper). For WfP deploys it hydrates assetsByHash by
  fetching bytes back from R2 so the existing CF asset upload session
  flow keeps working unchanged.

The post-parse pipeline is duplicated (carefully, with cross-reference
comments) rather than extracted into a helper. Extracting would have
required threading 15+ pieces of parse-loop-local state through a
shared function, with high risk of regressing the multipart path. The
duplication is mostly mechanical mirroring of the existing pipeline.

Adds R2_STATIC_ASSETS_BUCKET to wrangler vars (production + staging) so
the presigner can read the bucket NAME at runtime — STATIC_ASSETS_R2
binding only surfaces the R2Bucket object.

17 new integration tests in test/integration/deploy-direct-r2.test.ts
cover: presign-URL endpoint shape + validation + auth, JSON-mode static
deploy success path, missing-hash rejection, WfP deploy with
workerFiles, dedupe short-circuit, mid-pipeline cancellation,
v2-version enforcement, manifest validation, plus a multipart
backward-compat smoke test. All 506 existing API integration tests
still pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/deploy): three JSON-mode bugs from direct-to-R2 adversarial review

Three small but distinct bugs in the JSON-mode deploy handler introduced
in 4cc3524e (Task 2 of design-doc 0068). All three live in the same
handler so they ship as a single commit.

1. Carry-forward R2-head verification (static branch)

   `handleManifestOnlyDeploy` previously R2-head'd every declared asset
   hash and failed 400 if any were missing. But existing static deploys
   may have small assets in KV (`STATIC_ASSETS`) — not R2 — and the
   multipart path's preflight is happy to skip those without re-upload.
   JSON mode now distinguishes carry-forward hashes (present in the
   prior live deploy's `asset_hashes`) from new hashes (uploaded
   direct-to-R2 this deploy). Only new hashes are R2-head'd. The
   static-branch serving manifest also preserves the prior KV-vs-R2
   placement for carry-forward entries; only newly uploaded hashes get
   the `!{hash}` (R2) prefix. Mirrors the multipart static branch's
   existing prev-prefix-preservation logic.

2. workerFiles rejection for spa/static manifests

   The multipart path explicitly rejects `worker:*` parts when
   `manifest.type` is `spa` or `static`. JSON mode silently accepted
   them, then routed to the WfP branch — breaking the project type
   invariants. Now reject 400 BEFORE the precheck-and-claim transaction
   so a rejected request leaves no phantom placeholder row.

3. Canceling-placeholder reaper

   After the placeholder row is inserted by the precheck-and-claim
   transaction, several validation paths can early-return without going
   through the catch handler (asset-verify failures, quota exceeded,
   etc.). If `cancelDeployment` lands in that window, the row's status
   flips to `canceling` and the existing finally-reaper — which only
   handles `pending + manifest=null` — leaves the row stranded. Extend
   both reapers (multipart + JSON) to also delete `canceling +
   manifest=null` rows. Multipart had the same gap as a pre-existing
   bug; fixed for consistency.

Tests added in `deploy-direct-r2.test.ts`:
  - Carry-forward KV-stored asset survives a JSON-mode redeploy and the
    new serving manifest preserves KV placement.
  - workerFiles rejected 400 for `spa` and `static` types, and no
    placeholder row remains.
  - cancelDeployment racing the asset-verify early-return is reaped
    cleanly (no row left in `canceling` or `pending`).

* fix(api/deploy): strip v2 manifest fields before Cloudflare WfP asset session

The framework CLI emits v2 assetManifest entries shaped as
{ blake3, md5, size, hash } where hash is a deprecated alias of blake3.
The platform accepts these via structural typing and propagates the
manifest mostly verbatim through the deploy pipeline. Cloudflare
documents the assets-upload-session body shape as { hash, size }
per entry only and may strictly validate.

Add normalizeForCfAssetSession in packages/api/src/assets.ts that
returns a fresh { hash, size } object per entry, and apply it at the
two CF-bound call sites in packages/api/src/routes/deploy.ts (the
multipart and JSON-mode WfP paths). The persisted manifest in the DB
asset_hashes column is unchanged so future preflight reasoning still
sees the full v2 shape.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/deploy): widen JSON body cap and lengthen presign TTL

Two regressions surfaced by the Codex adversarial review of the Task 4
direct-to-R2 deploy work:

1. MAX_JSON_DEPLOY_BODY_BYTES (32 MiB) was tighter than the per-module
   raw cap (32 MiB) accounting for ~33% base64 expansion: a single
   full-size worker module weighs ~43 MiB on the wire. Bump the cap to
   64 MiB so a full-size module + manifest + assetManifest fits with
   safety margin, still well under the 128 MiB Worker memory limit.

2. Presigned PUT TTL of 600s expired on the tail of slow deploys
   (570 assets × concurrency 10 × residential uplink) before the
   upload pool reached the asset, and R2's SignatureExpired 403 was
   surfaced as terminal. Bump TTL to 1800s. Pathological multi-hour
   deploys are deferred to the re-presign-on-expiry follow-up noted
   inline.

Tests: assert X-Amz-Expires=1800 on issued upload URLs, and exercise
a JSON-mode deploy body comfortably above the legacy 32 MiB cap to
prove the regression is fixed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: bump framework to rebased HEAD + propagate git fields on JSON-mode

Picks up the framework branch's rebased HEAD f70264b (post origin/main
rebase). Also extends the JSON-mode `handleManifestOnlyDeploy` to read
`repositoryUrl`, `commitUrl`, `gitRef` from the body and pass them
through to `updateDeploymentManifest` so the new deploy metadata fields
introduced on platform main land for direct-to-R2 deploys too — not
just multipart.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: bump framework + drop dead body.vars reader on JSON-mode handler

Bumps framework submodule to 5c22706 — picks up:
- refactor(cli/package): buildVoidManifest pure, warning moved to callers
- refactor(cli/client): drop unused vars field on ManifestOnlyDeployPayload

Pairs the framework wire-shape cleanup with the platform side: removes
the defensive `body.vars` reader from `handleManifestOnlyDeploy`. No
released CLI ever sent top-level `body.vars` (env vars always travel
inside `manifest.vars`), so this is dead code, not a wire-compat
change. Replaced with a one-line comment documenting the contract.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(api): remove legacy multipart deploy upload path (bump framework)

The direct-to-R2 deploy path (design doc 0068) is now the only supported
flow. Drop the multipart streaming handler in `routes/deploy.ts` and its
supporting infrastructure:

- `routes/deploy.ts`: replace the multipart `app.post('/:id/deploy', ...)`
  body with a direct dispatch to `handleManifestOnlyDeploy`. Drop the
  multipart-only constants (`MAX_MANIFEST_BYTES`, `MAX_ASSET_MANIFEST_BYTES`,
  `MAX_PREFLIGHT_BYTES`, `MAX_SMALL_TEXT_BYTES`, `STREAMED_ASSET_CLAIM_*`)
  and helpers (`readPartBytes`, `readPartText`, `discardPartBody`,
  `readPartAsBlob`, `streamedAssetClaimKey`).
- `assets.ts`: drop the `./multipart.ts` import. Inline the
  `MultipartParseError` and `PartTooLargeError` shells locally — the
  multipart streaming asset helpers (`verifyStreamedAssetPart`,
  `streamStaticAssetToR2`, `readStaticAssetPart`, `uploadStaticAssetPart`,
  `cleanupStreamedStaticAssets`) and `collectBufferedAsset` are now dead
  code, but kept for now so this PR doesn't grow further; they'll be
  swept in a follow-up since the JSON-mode handler no longer references
  them.
- `multipart.ts` + `multipart.test.ts`: deleted outright.
- Test conversion: rewrite `api-deploy.test.ts` to use a JSON-mode
  `buildDeployRequest` helper that pre-uploads asset bytes to
  `STATIC_ASSETS_R2` and POSTs the manifest-only body. Drop the entire
  "(streaming static)", "(streaming static direct-to-R2)", and "(edge
  cases)" describe blocks — they tested multipart parser semantics that
  no longer exist; manifest validation coverage stays in the
  `manifest-rule-limits` unit test. Convert `api-sandbox.test.ts`,
  `deploy-collision-warning.test.ts`, and `deploy-direct-r2.test.ts`
  similarly. Drop the heartbeat test (the JSON-mode deploy is too fast
  for the heartbeat-during-pipeline assertion to fire deterministically).
- `meta/design-docs/0068-direct-r2-asset-upload.md`: append a "Migration
  complete" section noting the cleanup.
- Bump the framework submodule to the matching CLI commit.

Old CLI versions on npm will fail against the new platform with a clear
JSON 400 until users upgrade — accepted per the rollout plan.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore: bump framework submodule to origin/main HEAD

Framework PR #192 (direct-to-R2 asset upload + legacy-multipart removal)
landed on framework main as 5d5f609. Bump the submodule pointer so the
platform PR references the merged commit instead of the now-deleted
feature branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/upload-urls): streaming size cap before JSON parse

The `POST /:id/deploy/upload-urls` handler called `c.req.json()` directly,
which buffers arbitrary body bytes before parsing. A malicious or buggy
client could pin Worker memory by sending a huge JSON blob. Replace with
the same TransformStream + size-cap pattern used by
`handleManifestOnlyDeploy`, capped at 4 MiB — generous for 5000 asset
declaration tuples (~750 KB) with a 4× safety margin. Asset bytes never
travel this endpoint; they go direct-to-R2 via the presigned PUTs it issues.

Codex P2 review finding.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/deploy): stream JSON body via TransformStream to avoid OOM

The JSON body in `handleManifestOnlyDeploy` was previously buffered as a
chunks Array, copied into a Uint8Array, decoded into a UTF-16 string, and
parsed — peaking at ~4× the body size in residency. For a legitimate
64 MiB body this means 240+ MiB of peak memory, dangerously close to the
Worker isolate's 128 MiB hard limit.

Replace with a counting TransformStream + native `Response.json()`. The
TransformStream aborts the stream the moment the cap is exceeded; the
native parser consumes the stream incrementally with reused internal
buffers. The only retained allocation at the end is the parsed object.

Codex P1 review finding (high impact — every deploy goes through this).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(api/assets): remove dead multipart-streaming helpers

The streaming-static and worker-backed multipart deploy paths were retired
when JSON-mode direct-to-R2 became the only deploy entry point. The
helpers and their backing error classes were left behind as dead code:

  - `MultipartParseError`, `PartTooLargeError` (stub classes)
  - `streamStaticAssetToR2`, `verifyStreamedAssetPart`
  - `readStaticAssetPart`, `uploadStaticAssetPart`
  - `cleanupStreamedStaticAssets`, `collectBufferedAsset`
  - `readAssetBody`, `updateBase64Hash` (private helpers)
  - `PreparedStaticAsset` interface
  - unused `createHasher` import

Drop them along with the now-orphan `cleanupStreamedStaticAssets` test
block. `applyDeclaredAssetManifest`, `normalizeForCfAssetSession`,
`buildComparableAssetManifestForDedupe`, `uploadStaticAssets`,
`uploadAssetBuckets`, `createAssetUploadSession`, and the worker-bundle
helpers all remain — those are still on the live path.

Verified production code carries no remaining references; integration
test setup files mention the names only inside historical comments.

Cursor LOW review finding (sweep follow-up).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(api/deploy): remove unreachable preflight fill-in loop

In `handleManifestOnlyDeploy` the post-`applyDeclaredAssetManifest`
preflight fill-in loop was unreachable. The two arguments are always the
same `incomingAssetManifest` object, so the helper produced an output
manifest containing every key from the input — and the loop's
`if (!(path in assetManifest))` predicate could never be true. The
`getLastLiveAssetManifest` fallback (`?? await ...`) was equally
unreachable because `incomingAssetManifest` is always non-null in this
handler. Replace `applyDeclaredAssetManifest(...)` with `{ ...incomingAssetManifest }`,
the only effect that the call ever had at this site.

Update the surrounding comment block to reflect that JSON-mode CLIs
already send the full assetManifest (preflight-skipped paths included).
Add a defensive contract test pinning that behaviour.

`getLastLiveAssetManifest` and `applyDeclaredAssetManifest` remain
imported / exported — they're still used elsewhere
(`buildComparableAssetManifestForDedupe`, rollbacks, persistence
recovery, `applyDeclaredAssetManifest` unit tests).

Cursor MEDIUM review finding (dead code).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/upload-urls): reject deleting projects with 410

The `POST /:id/deploy/upload-urls` handler issued writable presigned R2
URLs after only ownership/existence checks, even for projects already
marked `status='deleting'`. Such projects can never finalize a deploy —
`POST /:id/deploy` returns 410 — so any bytes uploaded against those
URLs become orphaned `assets/{projectId}/*` blobs that survive teardown
and add storage churn.

Mirror the guard `handleManifestOnlyDeploy` already enforces, returning
410 immediately after the ownership check and before any presigning.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* perf(api/deploy): single-pass base64 decode for workerFiles

The JSON-mode deploy handler decoded each base64-encoded user worker
module with `atob` + a hand-rolled per-byte loop, materializing four
full-size copies of the bytes (UTF-16 source string, latin1 `bin`
string, `Uint8Array`, and the eventual `Blob` part). Iterating via
`for…of Object.entries(...)` also captured the entire `workerFiles`
map for the full loop, holding every base64 string live until the
loop exited. For a near-cap 32 MiB module that pushed peak isolate
residency to ~214 MiB — past the 128 MiB worker cap — and terminated
the deploy before the pipeline ever ran. This was a regression from
the multipart path, whose blob-first handling never paid the
intermediate-string cost.

Two changes drop peak residency to ~118 MiB for one near-cap module
(under the cap with margin):

1. Prefer the (TC39 stage-4) `Uint8Array.fromBase64` helper — single
   allocation, no intermediate string. Production CF Workers
   (compatibility_date=2026-02-12) ships the helper natively; the
   previous `atob` loop is kept as a fallback for runtimes that don't
   expose it (Node-based integration tests).

2. Iterate over `Object.keys(...)` and `delete` each source string
   from the parsed body after decode, so V8 can reclaim the ~86 MiB
   UTF-16 backing string before the next module's decode begins.

Adds a regression test exercising a multi-entry `workerFiles` map
that round-trips every decoded module byte-for-byte through the
persisted R2 worker bundle — would surface either iteration skips
caused by `delete`-as-we-go or any byte-level corruption from the
`fromBase64` helper.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/teardown): sweep assets/{projectId}/* during project deletion

Direct-to-R2 deploy lets the CLI upload bytes to `assets/{projectId}/{blake3}`
before finalizing. Bytes uploaded via presigned PUTs that never reach a
finalized deploy (TOCTOU race against teardown, abandoned upload, etc.) live
durably in R2 — retention cleanup only walks hashes referenced by retained
manifests, so without a teardown sweep these would leak permanently.

Add `deleteProjectAssets(r2, projectId)` mirroring `deleteWorkerBundle`
(paginated list + bounded batch delete) and call it from
`cleanupBestEffortResources`. Failures are warnings, matching the
surrounding best-effort semantics.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/deploy): extend retention cleanup with R2 keyspace sweep

The retention loop only deleted hashes named in EVICTED deployments'
asset_hashes. Bytes uploaded via direct-to-R2 presigned PUTs that never
finalized into a live deploy are not named in any manifest, so they
outlived eviction indefinitely.

Add `sweepProjectAssetOrphans(r2, projectId, retainedHashes)` and call
it at the end of both retention paths (WfP + static). Also build the
retained-hashes union unconditionally — previously skipped when
`toEvict.length === 0`, which is precisely when the orphan sweep is
the only cleanup running. Best-effort try/catch so a sweep failure
never fails the deploy.

Plus a TODO in upload-urls noting the active-project case (abandoned
upload that never deploys) is still only opportunistically GC'd by
the next retention run; periodic per-project sweep is follow-up work.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/upload-urls): serialize presign with teardown via FOR SHARE lock

The unlocked status check in `POST /:id/deploy/upload-urls` was a TOCTOU
gap: a teardown calling `lockAndMarkDeleting` (FOR UPDATE) could land
between the unlocked read and the presigner, after which presigned R2
PUT URLs would let the CLI write durable orphan bytes against a
project being deleted.

Wrap the presign loop in a transaction that takes FOR SHARE on the
project row and re-reads the project status inside the lock. FOR SHARE
is compatible with concurrent upload-urls / deploy-precheck flows
(also reading project rows) but blocks teardown's FOR UPDATE until the
presign batch completes. A 5000-entry batch holds the lock for
~250 ms (presigning is pure HMAC, no I/O), which is comfortably
shorter than the surrounding teardown timeline.

The unlocked early-out is kept as a fast 410 path for the common case
where the project was already marked deleting before the request
arrived; the in-tx re-read is the authoritative check.

Adds a regression test that simulates the race by intercepting the
second `getProject` call (in-tx, post FOR SHARE) so it sees a
status='deleting' that committed between the two reads. To support
the test, `setup.ts` exposes a `setGetProjectOverride` hook similar
to the existing `setTestDb`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/assets): protect pending uploads in retention sweep with age window

Direct-to-R2 presigned PUTs land bytes in the asset keyspace before any
deploy manifest references them. The retention sweep walked every
`assets/${projectId}/*` key whose hash wasn't in the union of retained
manifests, which raced concurrent `upload-urls`-then-finalize flows on
the same project: a sweep firing between an upload-urls PUT and its
deploy's manifest persist would delete the bytes and break finalize with
"asset not found".

Add a 3600s pending-upload protection window (2× the 1800s presign TTL)
in `sweepProjectAssetOrphans`. R2 objects whose `uploaded` timestamp is
inside the window are preserved even when not in `retainedHashes`. The
tradeoff is documented in the function docstring and on the existing
`TODO(orphan-gc)` follow-up: orphans uploaded within the window only
clear on the next eligible deploy ≥ 1h later.

Also extends the test mock (`MemoryR2Bucket`) to expose
`uploaded` on `list()` responses and a `setUploadedAt` helper, so the
existing orphan-sweep regression can age its orphan past the window and
a new companion test covers the within-window preservation case.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(api/assets): extract shared paginated-list-and-delete helper

`deleteProjectAssets`, `sweepProjectAssetOrphans`, and `deleteWorkerBundle`
all repeated the same R2 walk-and-batch-delete loop. Pull it out into a
single `paginatedR2Delete(r2, prefix, { keepIf })` helper and let each
call site become a 1–4 line wrapper. The orphan sweep's hash filter and
pending-upload protection window are expressed as the `keepIf` predicate
so the helper stays generic.

Behaviour-preserving — every existing test (unit `assets.test.ts`,
integration `deploy-direct-r2.test.ts`, `api-delete.test.ts`,
`api-deploy.test.ts`) passes against the refactored implementations.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(api/r2-presign): split length vs character validation errors

`hexToBase64` reported \`got \${hex.length}\` regardless of which
constraint failed, so a 32-char string containing non-hex characters
produced the misleading \"got 32\" message — opaque during CLI
debugging. Split the check: length first, character-set second, with a
distinct message for each.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

Author: Brooooooklyn