Repository navigation
Redeploy main after a merge when prod has not converged on it #3848
Description
🤖 Generated by the Agentic Engineer
Part of #2838. It follows #3846, whose resolver merged in #3847.
Evidence
- Since feat(ci): tell whether prod has converged on main from signed provenance #3847,
scripts/resolve-prod-convergenceanswers from signed evidence whether prod's published artifact matchesmain. The answer isCONVERGED,BEHIND,DIVERGEDorUNKNOWN. - Nothing runs it. After the interleaving A merge-group heal can leave prod behind a merge that lands just after it #2838 describes, prod still stays behind
mainuntil an unrelated deploy happens to land. - The resolver needs registry read access to verify the attestation. That exists only on a runner with
packages: read, so its last acceptance point (a liveCONVERGEDread) is proven here, not in Tell whether prod has converged on main from the published artifact's signed provenance #3846.
Audience and impact
This serves the platform operator. It closes the last path by which prod can silently disagree with main after a merge.
Smallest useful change
After a merge lands on main, run the resolver against the digest :latest names:
CONVERGED: do nothing.BEHINDorDIVERGED: run the normal production deploy ofmain, through the same composite and the sharedprod-deploylock.UNKNOWN: do not deploy; fail visibly, so a broken check is never mistaken for a converged prod.
Because this job reaches production, it inherits every gate the existing deploy paths carry: the publication gate, the authorization and signing contracts, and the per-workflow coverage guards. It lands behind an opt-in switch, defaulting off, and is activated separately.
Acceptance criteria
- On a merge where prod already matches
main, the job readsCONVERGEDand does not deploy (proven on a real merge). - A
BEHINDorDIVERGEDverdict triggers exactly one deploy ofmain, and a second evaluation afterwards readsCONVERGED. -
UNKNOWNnever deploys and fails the job with the resolver's reason. - The job holds the shared
prod-deploylock and cannot run concurrently with a merge-group deploy or heal. - Tell whether prod has converged on main from the published artifact's signed provenance #3846's live
CONVERGEDacceptance point is recorded from this job's first real run.
Size
M: one workflow job plus contract-test updates, behind an opt-in switch.
Blocker: the next time production is behind main after a merge | outcome | last-verified 2026-10-17: not yet; the two bot-dispatched CD runs that succeeded since 2026-10-07 (10-12, 10-14) came from restore-stranded-prod.yaml, and the observer before the 10-12 one read DIVERGED and skipped its redeploy step. When the observer reads BEHIND, confirm it redeployed main and production converged, then close
Activity
chore: agent-claim #3848 nonce=63414cca9ef443b07a5ef2250891eb5f
fix(ci): fetch attested commits and keep docker stderr out of the digest
ci: test the convergence report on pull requests
devantler commentedon Sep 16, 2026
First slice merged: #3864 (
5e3049c9). After every push tomain, a read-only job reports the resolver's verdict.CONVERGEDbecomes a notice.BEHINDandDIVERGEDbecome warnings, because a queued merge group can briefly showDIVERGED.UNKNOWNfails the job. The job never deploys. Before resolving, it fetches every commit the provenance names, so an artifact left by a merge-group heal does not read asUNKNOWN. Its first live run readCONVERGED(run 35150608562).main→ readsCONVERGED, no deploy (real merge)BEHIND/DIVERGEDtriggers exactly one deploy ofmain, thenCONVERGEDUNKNOWNnever deploys and fails with the resolver's reasonprod-deploylockCONVERGEDpoint recordedNext slice: a deploy of
mainonBEHIND/DIVERGEDunderprod-deploy, behind a switch that defaults to off. It reaches production, so it carries the deploy lane's usual gates.devantler commentedon Sep 17, 2026
Status 2026-09-17: the deploy slice merged as #3870 (for #3869)
main→ readsCONVERGED, no deploy (real merge)CONVERGEDfor1dff6239, skipped the redeploy step, and dispatched no CD run.BEHINDtriggers exactly one deploy ofmain, thenCONVERGEDCDonmainonly onBEHIND, skips when an unfinished CD run already targetsmain's tip, and waits for its dispatched run to be listed before releasing the lock. Proven by the script's tests. A realBEHINDneeds the #2838 interleaving, so this stays open until one happens.DIVERGEDDIVERGED, so redeploying on it would undo a deploy that is about to land (decision recorded on #3869). It stays a warning.UNKNOWNnever deploys and fails with the resolver's reasonprod-deploylockprod-deploywithqueue: max.CONVERGEDpoint recordedRemaining: record the first live
BEHIND→ one CD run →CONVERGEDsequence. Nothing else is left to build here.devantler commentedon Sep 19, 2026
Status 2026-09-19: still waiting on a live
BEHIND. Nothing is left to build.Across the last 100
validate-main.yamlruns, 32 reached the🧭 Observe Production Convergencejob, and every one of them recordedCONVERGEDin its annotations. None recordedBEHIND,DIVERGEDorUNKNOWN. The only failed run in that window (35100863274, 2026-09-16) predates the observer.So the one open criterion (a real
BEHIND, then one CD run, thenCONVERGED) is waiting for the #2838 interleaving to actually happen. There is no code change to make until it does. Any later run can re-check with the check-run annotations on that job.