Repository navigation
fix: Take a controller's owner from its namespace - #907
Merged
Merged
Conversation
A controller-health alert took its service only from the deployment owner map. A controller whose deployment the map does not name fired with no service, so it reached no team and fell to the catch-all channel: 17 controllers in 14 days, among them the cell controller manager, galactic, karmada and Flux. The platform resolves an owner by deployment, then by namespace; the rule stopped after the first step. Key changes: - Each alert joins namespace:datum_service:info on cluster and namespace when the deployment map does not name its deployment - A deployment both maps name keeps its deployment's owner and fires once; one that neither names still fires, without an owner - Fixtures for both, and for the floor warning; the test copy follows the shipped rules - The runbook says where the owner comes from
scotwells
approved these changes
Nov 12, 2026
ecv
approved these changes
Nov 12, 2026
ecv
left a comment
ecv
left a comment
Contributor
There was a problem hiding this comment.
Approve.
Controller health alerts for a deployment missing from the deployment owner map now take their team from the namespace owner map, so they reach a team channel instead of the catch-all.
namespace:datum_service:infoistopk by (cluster, namespace) (1, ...)in both infra estates, so theon (cluster, namespace) group_leftjoin is many-to-one and cannot error on duplicates.- The three branches are disjoint: the deployment owner wins, then the namespace owner, then no owner. A controller fires once.
- The expression edit resets the state of any of the three alerts firing at the release that carries it. Those alerts go pending again for their
forof 15m or 10m.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Controller health alerts take their owning team only from the platform's deployment owner map, so a controller missing from that map alerts with no owner and lands in the catch-all channel, which several controllers did in both environments over the last two weeks.
This change falls back to the platform's namespace owner map, the next step of the platform's ownership standard.
A controller that both maps name keeps its deployment's owner and alerts once, and one that neither names still alerts, without an owner.
Changes
Test plan