Repository navigation
fix: Bring back every network presence when the controller starts - #844
Merged
Merged
Conversation
A presence that no hub binding declares and whose location publishes no interface comes back only when the wait its process set runs out. These tests run the presence controller under a real manager and replace it, or hand its lease to a standby, before the network is deleted or before the grace ends. On main nothing brings such a presence back: its grace never ends, and the deleted network's finalizer waits for its context for good. The same steps in one process pass. They also pin what reconciling the hub's copies must not change: a copy that changes alone brings no write, and a copy that goes ends nothing.
A presence that no hub binding declares and whose location publishes no interface came back only when the wait its process set ran out. A restart or a change of leader took that wait with it, so nothing looked at the presence again: its grace never ended, and since #828 a network deleted later waited for its context for good. The presence's key is the key of the hub's copy of its context, and the copy lasts as long as the presence does. So the controller now reconciles those copies as its primary type, with the bindings and the interface copies as secondary watches, and the list of copies on start brings back every presence.
Such a presence has no hub binding either, and may never have had one: with internal DNS on, the binding controller writes the context for a project-plane binding, network UID label included. Both fail on main after a restart and pass when the controller reconciles the hub's copies.
This was referenced Nov 3, 2026
scotwells
approved these changes
Nov 3, 2026
andrewb3000
approved these changes
Nov 3, 2026
vvoytovych
deleted the
fix/presence-reconciles-every-presence-on-start
branch
November 3, 2026 10:01
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since #828, a deleted Network waits until the presence controller ends each of its contexts, but the controller heard of a presence only through the hub bindings and the interface copies that cells publish (
SetupWithManager). A presence with neither, in its grace or declared only in its project, came back only when its wait ran out, and a restart or a change of leader takes that wait with it. So its grace never ended, and a Network deleted later stayed deleting: v0.34.0 deleted the context in the Network's finalizer, so this is new in #828 and not released yet. Staging holds two such contexts today, unclaimed since 2026-10-30 and three days past the 24-hour grace.Changes
For). The bindings and the interface copies stay as secondary watchesnetworkpresence_controller.goA presence's key is its hub copy's namespace and name. The replicator keeps that copy on the hub while the context exists and is not being deleted. So the initial list on start brings back every presence, and
RequeueAftergoes back to re-checking a wait, as the kubebuilder book recommends. A standby that takes the lease over gets the list too: client-go sends every cached object to a handler added after its informer started (shared_informer.go). The grace still comes from theunclaimed-sinceannotation, so a restart neither restarts it nor skips it.Corner cases
TestANetworkDeletedAfterARestartReachesAPresenceNothingDeclaresTestAPresenceNothingDeclaresEndsAtItsGraceAfterARestartTestANetworkDeletedAfterARestartReachesAPresenceDeclaredInItsProjectTestANewLeaderBringsBackEveryPresenceTestAPresenceWritesNothingWhenOnlyItsHubCopyChangesTestAPresenceOutlivesItsHubCopyGoingTestNetworkPresenceLeavesProjectPlaneBindingsAloneTestNetworkPresenceLeavesTheReplicatedHubCopyAloneCost
get,listandwatchon NetworkContexts, and the replicator watches them on the hub with the same identity.SetupWithManager: gocyclo 3 → 3, gocognit 3 → 3.Test plan
main, the restart, grace, declared-in-project, standby and copy-change tests fail, and the one-process control passes. All pass with the change, and the restart test passes on v0.34.0, whose finalizer deletes the context.golangci-lintv2.15.0 and CI's suites pass.mainboth stayed, 1 of 1 each. The e2e grace is 15 s, about one restart, so a deletion inside the grace after a restart is the envtest test's here, and staging's below.main's build, a network in our test project whose presence nothing declared was deleted at 09:36:29Z and was still deleting 90 s later. After the merge, its build started on the hub at 10:12:58Z, the presence controller ended that presence at 10:13:01Z, and the network went.Related to #664