Skip to content

fix(cli): retry deploy POST on transport failure - #327

Merged
Brooooooklyn merged 9 commits into
mainfrom
fix/deploy-post-transport-retry
Sep 18, 2026
Merged

Brooooooklyn merged 9 commits into
mainfrom
fix/deploy-post-transport-retry

Conversation

@Brooooooklyn

@Brooooooklyn Brooooooklyn commented Sep 17, 2026 •

Copy link
Copy Markdown
Member

Summary

  • The finalize-deploy POST /:id/deploy had no retry: a single socket reset (ECONNRESET, observed on oxc-project/website CI after ~3 min of successful R2 asset upload) failed the whole deploy with deploymentId=null.
  • streamDeployRequest now retries fetch rejections (TypeError — no HTTP response received) with 1s → 5s → 10s backoff, max 3 retries.
  • Server-side arbitration: a rejected fetch does not prove the request never arrived, but a client-side lookup can never prove it either (the row can transition between GET and replay). So the replay always reaches the server's precheck — the only atomic, project-locked verdict. A 409 ({error, deployment} — 'deploy in progress' / 'deploy cancellation requested') is surfaced as a structured error event naming the deployment id; if nothing is in flight the deploy proceeds normally. Residual accepted: a lost attempt completing within seconds replays as an identical re-deploy — unavoidable without a server-persisted request id.
  • Other HTTP responses still surface verbatim (non-TypeError rejections, e.g. AbortError, fail fast).
  • JSONL log gains deploy_fetch_retry warnings, an attempts count on deploy_fetch_rejected, and deploy_conflict on 409s.

Test plan

  • Unit tests (16 in direct-r2-client.test.ts): retry-then-succeed, exhaustion (exact backoff schedule + original error), 409-with-deployment → error event (first attempt and after-retry), 409-without-id → PlatformApiError, AbortError no-retry, HTTP-503 no-retry
  • Adjacent deploy tests pass (direct-r2-upload, stream-deploy-errors)
  • tsc --noEmit clean, vp lint clean

Generated with Devin

A transient socket reset (ECONNRESET on a stale pooled keep-alive
connection, or any edge/network blip) during the finalize-deploy POST
failed the entire deploy even though no response ever arrived — so the
platform could not have started processing. Retry the fetch with 1s/5s/10s
backoff (max 3 retries). HTTP responses of any status still surface
verbatim since a server answer means the request may have been accepted.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-18T03:51:55.813324Z 04d3929 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ba0aa8695

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts
…ed a deployment

A rejected fetch does not prove the request never arrived — the
connection may have dropped after delivery. Before replaying the
non-idempotent POST, check for a deployment created inside a 120s window;
finding one means the earlier attempt reached the platform, so the retry
is suppressed (reported via a terminal error event naming the deployment)
instead of bouncing off the server's 409 concurrency guard or registering
a duplicate deploy. The check is best-effort: on lookup failure the retry
still runs and the server's guard remains the backstop.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d651891c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
… retry

A recency-only match could wrongly attribute an unrelated deploy (a
manual or CI deploy racing ours) to the lost attempt and abort the retry.
The deployments table stores the payload's git commit, so require the
candidate row's commit to equal ours — null===null counts since ownership
cannot be disproved without metadata. A commit mismatch means the row is
not ours and the POST is still replayed.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: df5c1c6e42

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
createDeployment inserts the deployment row as a concurrency claim with
commit=NULL; updateDeploymentManifest fills it only after R2 inventory.
For a deploy carrying a commit, our own lost attempt therefore reads
commit:null during its first seconds and the equality check missed it,
letting the replay hit the 409 guard. Now also treat a still-in-flight
null-commit row as a possible placeholder when the payload has a commit
(an unrelated in-flight deploy would reject the replay with 409 anyway).
Failed/canceled rows are dead attempts and no longer suppress.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9a6c42859

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
Commit equality cannot prove ownership of a deployment row — same-SHA
redeploys and commit-less manual deploys collide — so matching a recent
row on commit could suppress a POST that never reached the server,
silently dropping the deploy the user asked for.

Suppress only when a row created inside the window has a non-terminal
status: getActiveDeployment treats any such row as an active deploy and
the replay would be rejected with 409 regardless of ownership, so
suppression is equivalent to retrying and produces the clearer error.
Terminal rows never suppress — a completed attempt instead replays as an
identical re-deploy, the benign side of the ambiguity. This also drops
the payload-commit plumbing, which no longer discriminates.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d526b9924d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
Comment thread packages/void/src/cli/client.ts Outdated
Two suppression gaps found on review:

- The deployment lookup ran before the backoff sleep, so a placeholder
  inserted while the lost POST's precheck was still running could be
  missed entirely and the replay hit the 409 the dedupe exists to avoid.
  The lookup now runs after the sleep, immediately before the replay.
- `cutover`/`compensating` rows are not guaranteed to 409: the precheck
  can reconcile them (completed compensation → failed, marker-verified
  cutover → live) and accept the new deploy, so suppressing on them
  could discard the requested deployment. Suppression now uses an
  explicit status allowlist (pending/uploading/migrating/prerendering/
  canceling) — only rows whose replay is guaranteed redundant.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 21271b7a6f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
Postgres `timestamp` mode:'string' values (e.g. `2026-09-18 12:00:00`)
carry no zone marker, so `Date.parse` read them as local time: in
UTC+N a fresh row looked N hours old and the dedupe missed it; in
UTC−N a stale row could appear to be in the future and wrongly
suppress. Normalize offset-less values as UTC, mirroring the platform's
`databaseTimestampMs` helper. Tests now emit the real wire format.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 25884e55cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/void/src/cli/client.ts Outdated
Brooooooklyn and others added 2 commits September 18, 2026 11:49
A client-side GET-then-decide dedupe can never be authoritative — the
deployment row can transition between the lookup and the replay, so any
suppression rule carries an unavoidable race that can drop the deploy
the user asked for. Remove the /deployments lookup, status allowlist,
and timestamp-window machinery entirely.

The retried POST now always reaches the server's precheck — the only
atomic, project-locked verdict. A 409 conflict ({error, deployment})
is surfaced as a structured error event naming the deployment id
instead of a bare HTTP failure. Residual: a lost attempt completing
within seconds replays as an identical re-deploy — the benign side of
not being able to prove ownership client-side.

Addresses Codex review on PR #327.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`parsed = JSON.parse(body) as typeof parsed` let the declared type query
resolve to the narrowed `null` under stricter checking, collapsing the
variable to `never` and failing CI typecheck (TS2339 at the conflict
handling site). Parse into `unknown` and narrow explicitly instead.

Generated with [Devin](https://devin.ai)

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@Brooooooklyn
Brooooooklyn merged commit 24f2910 into main Sep 18, 2026
19 checks passed
@Brooooooklyn
Brooooooklyn deleted the fix/deploy-post-transport-retry branch September 18, 2026 07:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant