Skip to content

Deploy production on push to prod. - #320

Merged
Brooooooklyn merged 10 commits into
mainfrom
platform-prod-deploy
Sep 14, 2026
Merged

Brooooooklyn merged 10 commits into
mainfrom
platform-prod-deploy

Conversation

@Brooooooklyn

@Brooooooklyn Brooooooklyn commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Production had no ref to release from: platform-deploy.yml was manual dispatch only, so a release was whoever clicked "Run workflow". The old repo released on push to its prod branch; this restores that model on the monorepo.

What changed

platform-deploy.yml — deploy on push to prod

push:
  branches: [prod]
concurrency:
  group: platform-deploy-production
  cancel-in-progress: false

prod is the production ref — it points at the commit that is live, so advancing it is the release:

git push origin main:prod     # release whatever main is at

No path filter, deliberately: the old repo had none, and a push to prod is already deliberate. The concurrency group mirrors staging's (cancel-in-progress: false), so a deploy is never interrupted between the API migration and the workers.

schema-compat — the production guard, restored

The old repository's job, ported as-is: Phase 1 applies the candidate's migrations to a real Postgres 18; Phase 2 checks out prod and runs its platform-api-integration suite on that schema. It runs on PRs and, through require-schema-compat, on every production deploy (a push to prod checks itself, as before; a manual dispatch from another ref is checked against prod). Two integration tests only ever passed on PGlite; they are fixed so the suite holds on Postgres too.

Accepted limits, same as the old repository and stated in the job comment: the baseline is the prod branch head rather than the deployed revision, and released migrations are assumed append-only (drizzle never re-applies an applied entry and skips entries not newer than the last applied). A stricter guard (deploy-recorded baselines, an append-only checker) was tried on this branch and removed: more moving parts than the risk it covers.

Docs

platform/meta/operations.md (environments table, secrets note, workflow list) and platform/README.md — which platform/AGENTS.md symlinks to — now describe the branch model, and the agent guardrail becomes "do not push prod" instead of "deploys are manual".

First release notes

  • prod in this repo currently points at ab92e6d3 (2026-06-02, the SDK repo's old release branch), 1214 commits behind main. It is an ancestor of main, so advancing it is a fast-forward.
  • The old repo's prod (ea101e78) is also an ancestor of main, so no prod-only work is lost.
  • No database migrations ride along: all 26 files in platform/packages/api/drizzle are blob-identical between the old repo's prod and main, so the first release from here applies none.
  • Nothing deploys until someone pushes prod. That push is the trigger; there is no separate button.

Follow-ups this unblocks

  • Retire the old repo's Deploy Production (push to its prod) at the same moment, so only one repo can write production.
  • platform-ci.yml's schema-compat job is still inert with the comment "the merged repository's production ref has intentionally not been chosen yet" — prod is now that ref, so the job can be ported next.
  • The old repo's prod branch carried a ruleset (deletion, non_fast_forward, update, bypass for org admins / write role / the maintainers team). Nothing protects prod here yet, which matters more now that a push to it deploys production.

Verified

  • actionlint on the workflow: clean
  • vp fmt --check: clean

Note

High Risk
Production deploys become automatic on prod pushes and gate releases on migration compatibility against live prod code; mis-timed or breaking migrations can block or endanger production rollout.

Overview
Production releases now advance the prod branch instead of relying on manual workflow dispatch. Pushes to prod trigger platform-deploy.yml, with a non-cancelling concurrency group so deploys are not interrupted between API migration and worker rollout. The reusable CI workflow accepts require-schema-compat, and production deploy always sets it so the guard runs on every production trigger (including manual dispatch from another ref).

The schema-compat CI job is enabled (it was previously inert). It applies the candidate’s Drizzle migrations to Postgres 18, then checks out prod and runs platform-api-integration against that schema so live code must tolerate the new schema before ship. PRs get the same check; staging deploys skip it. If prod predates the monorepo platform tree, migrations still run but integration tests are skipped with a warning.

Docs (platform/README.md, platform/meta/operations.md) now describe push-to-prod as the production trigger and warn agents not to push prod.

Two integration tests were adjusted for real Postgres used by schema-compat: the rollback test models sequential rollbacks with a transaction hold instead of concurrent lock contention (Postgres lease/409 behavior differs from PGlite); the sandbox teardown-race test hooks cutover-ready: KV writes (post-commit, no open tx) so teardown ordering is deterministic on pooled Postgres.

Reviewed by Cursor Bugbot for commit 22e988b. Bugbot is set up for automated code reviews on this repo. Configure here.

Production had no ref to release from: the workflow was manual dispatch only,
so a release was whoever clicked "Run workflow", while the old repo released on
push to its `prod` branch.

`prod` is the production ref, so advancing it with `git push origin main:prod`
is the release. No path filter, because a push here is already deliberate, and
a concurrency group keeps one deploy running so a deploy is never interrupted
between the api migration and the workers.
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_4cf8a4f5-f64c-42c6-9f1e-1a73b4819d32)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-14T10:09:36.975622Z 22e988b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

The branch that deploys production is now protected, so say who can move it
rather than leaving the readme to imply anyone with push access can.
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_151a03ec-6264-4615-b852-442e79482ee3)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 56f230f7d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/platform-deploy.yml
`platform-deploy.yml` migrates before it deploys the api worker, so a push to
`prod` applies a new schema while the previous release is still serving. The
former repository checked that on every platform CI run; the job came over
inert here, waiting for the production ref to be chosen.

Migrate a real Postgres with the ref under test, then run the production ref's
integration tests against that schema. It runs on PRs, where the code under
test differs from what is live, and on the production deploy itself; staging
deploys skip it. Until `prod` is advanced past the platform merge there is no
baseline to compare against, so the job warns instead of failing.
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_81d1410c-04f0-4bec-9ff9-e4000aa9e72b)

Comment thread .github/workflows/platform-ci.yml

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9d30e83402

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/platform-ci.yml
The compatibility baseline was the `prod` branch, which on a push already
resolves to the commit being deployed, so the check never saw the code that is
live while its migrations run. Use the push event's previous commit instead;
PRs keep comparing against the deployed branch.

The production deploy also asks for the guard on every trigger, so a manual
dispatch from any ref cannot skip it, and the job pins TZ=UTC because the suite
compares Postgres timestamps against local time.
Both pin an interleaving that only PGlite's single-connection statement queue
produces, and both drive code rewritten after the merge (60afd36, e3bd516)
in ways that invalidated the trick while still passing on PGlite:

- sandbox teardown race: the fire-and-forget status flip opened a new session on
  a pool and lost to the deploy's warm connection, so the deploy committed live.
  Hook the `cutover-ready:` write instead — no transaction is open there — and
  await the flip and the routing delete.
- rollback-vs-rollback: Postgres grants the queued row lock at the other
  rollback's commit, so the loser read the rollback lease and answered 409.
  Hold the second rollback at the transaction seam until the first has fully
  finished, which is the serialization this test is about.

The suite now passes on PGlite and on Postgres 18 with the migrations applied.
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_e4d1e145-d00a-449e-b841-ab0073557819)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e11090621e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/platform-ci.yml
Comment thread .github/workflows/platform-ci.yml Outdated
Brooooooklyn and others added 3 commits September 14, 2026 17:21
`drizzle-kit migrate` applies only journal entries whose `when` is later than
the last entry it applied and never compares the stored hash, so production
skips an edited historical `.sql` and a new entry whose timestamp is not after
the last applied one. A fresh database applies both, which let the schema
guard pass on a migration set production would not apply.

`check-migration-set.ts` reads two drizzle directories and reports every way
the candidate is not an append-only extension of the released set: a released
file modified or removed, the released journal no longer a prefix, or a new
entry not newer than the one before it. The `schema-compat` job copies the
candidate's migrations and the checker aside before the baseline checkout and
runs it against the baseline's migrations. With that, migrating a fresh
database is the same operation as upgrading production's.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNXbwieSdsSErZwQ7HWRwG
The schema guard took `github.event.before` as the release it must keep
running. That is only the previous push to `prod`: it may have failed to
deploy, or been replaced while pending in the `platform-deploy-production`
concurrency group, which keeps one deploy running and one pending and lets a
newer pending run replace the older. The live release is the head of the last
successful `platform-deploy.yml` run, so the job now reads that from the
workflow's run history, which needs `actions: read` on the job and on the
deploy's `ci` job that calls it.

Until a production deploy has been recorded there is nothing to read, so the
job falls back to the previous `prod` commit on a push and to the `prod`
branch otherwise, and warns that it did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNXbwieSdsSErZwQ7HWRwG
The append-only check compared the candidate's migrations with the last
successful release's. That set can lag production's schema: the api deploy
migrates before it deploys the worker, so a push whose deploy failed afterwards
has still applied its migrations, and a retry could edit or replace them and
pass. The check now takes the previous `prod` commit on a push, and the `prod`
branch otherwise, as the released set — at or ahead of the live release's,
since `prod` only moves forward — checked out sparsely beside the code baseline
so the live release still decides which tests run. A released commit that
predates the platform carries nothing to extend, so the check says so and
skips; the tests keep their own gate.

The staging deploy calls the same workflow and granted its `ci` job only
`contents: read`, which a called job asking for `actions: read` exceeds. Grant
it there too, though staging never runs the guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNXbwieSdsSErZwQ7HWRwG
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_c0f4026e-2257-4a6b-899f-8d74bf24bad0)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7549e2c853

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/platform-ci.yml Outdated
Comment thread .github/workflows/platform-ci.yml Outdated
Comment thread .github/workflows/platform-ci.yml Outdated
The extra baselines and the migration-set checker added more moving parts
than the risk they cover. The old model — migrate a fresh Postgres with the
candidate's migrations, then run the `prod` branch's integration tests on it —
ran seven months without incident, so the guard goes back to that.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNXbwieSdsSErZwQ7HWRwG
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_8eb02a4f-e54e-4150-aea0-27981ed010a4)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8839699fd0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread platform/packages/api/test/integration/api-migrations.test.ts
The second rollback is held until the first has fully finished, so the test checks two rollbacks in sequence rather than row-lock contention, and its title and comments now say so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WNXbwieSdsSErZwQ7HWRwG
@cursor

cursor Bot commented Sep 14, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_0acc1f79-1152-46e2-b6d0-bf9659f955ba)

@Brooooooklyn
Brooooooklyn merged commit ea278e7 into main Sep 14, 2026
16 checks passed
@Brooooooklyn
Brooooooklyn deleted the platform-prod-deploy branch September 14, 2026 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant