You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Status freeze persists on the scale and member-deletion early-returns (same class as #367) #371
#369 cures the freeze on the converged path (a healthy current == desired cluster whose pods all go down): the promote-after-converged pass now threads its transient requeue through updateStatus instead of returning early, so ReadyMembers / Available / Degraded stop advertising a last-healthy snapshot.
The same freeze class remains on every scale-related early return, which still skip updateStatus:
In-flight EtcdMember deletion — controllers/etcdcluster_controller.go deletion-wait loop returns RequeueAfter: 5s before updateStatus. With surviving peer CRs and etcd down, handleDeletion's removeMemberFromEtcd errors forever, reconciliationComplete is false (count mismatch), and deadlineExpired is false because ProgressDeadline was cleared at the prior convergence. ReadyMembers / Available=True/QuorumHealthy freeze exactly as in EtcdCluster status freezes at its last good value when etcd is unreachable #367. Repro: break-glass member deletion (or the memory-backed self-delete on pod loss), then all pods go down.
Any scale path — the deletion-wait, hasPendingMember → scaleUp, !allMembersReady, scaleDown, and the inner 5s/10s returns in scaleUp / completePendingMember / scaleDown all skip updateStatus. Healthy 3/3, replicas: 5, member 4 added as a learner, then a node shutdown takes all pods down: status advertises healthy with 0 ready pods for the full ProgressDeadlineSeconds, after which handleDeadlineExceeded writes Available=False/DeadlineExceeded but leaves ReadyMembers, Degraded, and the PDB untouched.
Design question to settle first
Both are the "derive member-based status on paths that early-return" problem. The durable shape (raised as the first open question on #367) is to derive the member-based status (ReadyMembers, Available/Degraded, BrokenMembers, PDB) before the scale fan-out — or on every non-error return — and keep it separate from the reconcile-state conditions that other writers own (the cert-manager gate, surfaceDiscoveryError, handleDeadlineExceeded).
The subtlety that makes this more than a mechanical refactor: during progression the ready/desired ratio counts not-yet-joined members against quorum, so a naive always-write flips Degraded on healthy scale-ups (the #369 review's item 1). #369 handles that for the converged path by withholding the health rewrite while Progressing=True && !reconciliationComplete; a general "write health on every return" needs the quorum denominator to be the established voter count during progression, not the desired replica count. Worth agreeing on the shape before landing the broader change.
Activity
fix(controllers): floor the progressing status withhold on quorum