fix(promote): gate env pointer advance on in-scope deploy success - #632
Merged
Merged
Conversation
Promotion finalize advanced state.<env>.sha and state.<env>.version to the new commit even when an in-scope deploy terminally failed or was cancelled. With rollback_on_failure (the default), the auto-rollback job redeploys the environment's prior sha while finalize recorded the new one, so state asserted a version the environment was not running: a state/reality inversion in the default config. Finalize now gates the env-pointer advance on in-scope deploy success, mirroring the rollback finalize gate, and holds the pointer at its prior value when any in-scope deploy did not succeed. Per-deploy success rows are still recorded, so a partial success is not lost and a re-dispatch re-plans and retries only the failed deploy. A promotion whose deploys all succeed, or that advances an env with no deploys, is unchanged. Corrects two unit tests that encoded the old behavior (env advancing over a failed or cancelled deploy) and extends the promote-rollback runtime e2e scenario to assert the target pointer is held at the prior sha after a failed deploy, via an opt-in post-failure state sync. Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
joshua-temple
force-pushed
the
fix/promote-finalize-gate-on-deploy-success
branch
from
July 18, 2026 06:03
b63f89e to
bbad2e1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Promotion finalize advanced the environment pointer (
state.<env>.shaandstate.<env>.version) to the new commit even when an in-scope deploy terminally failed or was cancelled. Becauserollback_on_failuredefaults to true, the auto-rollback job redeploys the environment's prior sha while finalize recorded the new one. In the default config, state then asserts a version the environment is not running: a state/reality inversion. The run is red, but the erroneous state advance is silent, and a re-dispatch withprod.sha == uat.shais a silent no-op that never resurrects the failed deploy.The rollback finalize already does the right thing (
gateOnDeployResultsaborts the write onfailure/cancelled); promote was the outlier.Fix
Finalize now gates the env-pointer advance on in-scope deploy success (
inScopeDeployFailed), mirroring the rollback gate: the pointer is held at its prior value when any in-scope deploy terminally failed or was cancelled. Per-deploy success rows are still recorded individually, so a partial success is not lost and a re-dispatch re-plans and retries only the failed deploy. The in-scope set is the finalize's own deploy-result map, which is already scoped per component. Behavior is unchanged when every in-scope deploy succeeds, when a promotion advances an env with no deploys, and for--componentfinalize (siblings preserved).Verification
rollback_on_failure: false, and happy-path advance.failure/cancelleddeploy).promote-rollback-runtimee2e scenario to assert the target pointer is held at the prior sha after a failed deploy, via an opt-in post-failure state sync in the harness (finalize runs underalways(), so it commits state on a failed run; the harness previously could not observe that write).go build,go test,go test -race, andgolangci-lint runall clean; e2e module builds and vets; all scenarios parse;verify --own-reporeports no drift.