Repository navigation
feat: record events when an EventTrigger fails to deploy or an EventReport fails to be processed - #526
Merged
Conversation
…eport fails to be processed When deploying an EventTrigger to a cluster failed, or when the ClusterProfile instances could not be generated from an EventReport, the failure was only recorded in the status (`status.clusterInfo[].failureMessage` of the EventTrigger and `status.failureMessage` of the EventReport) and in the logs. Nothing showed in `kubectl describe` or in the events of the namespace. event-manager now records a Warning event for both. It uses the `EventTriggerFailed` reason defined in libsveltos `lib/eventreasons`. | When | Regarding | Action | Message | |---|---|---|---| | Deploying an EventTrigger to a cluster fails | the EventTrigger, with the cluster as the related object | `Deploying` | `Deploying to cluster Sveltos mgmt/prod-1 failed: <error>` | | Generating the ClusterProfile instances from an EventReport fails | the EventReport, in the namespace of the cluster | `Reconciling` | `Generating ClusterProfile instances failed: <error>` | The EventTrigger is cluster scoped, so its events are in the `default` namespace. The cluster is the related object because the events recorder merges events with the same reason, action and regarding object and keeps only the first note: without it, the failures of one EventTrigger in two clusters would be a single event. An event is recorded when the failure message is new or different from the one saved before, not on every retry: - Per cluster, the previous failure message of the cluster is kept in `status.clusterInfo` through the Provisioning step of a retry. A retry which fails the same way records nothing, a different error does. - For an EventReport, the event is recorded in `setEventReportFailureMessage`, which already skips the write when the message is unchanged. Nothing is recorded when the failure is cleared, or when only `AgentFailureMessage` changed. No event is recorded for expected states: a paused cluster, a cluster which is not ready or down, a cluster whose agent is not healthy (sveltoscluster-manager records that one), and a deployment which is still in progress. Those return before a deployment result exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When deploying an EventTrigger to a cluster failed, or when the ClusterProfile instances could not be generated from an EventReport, the failure was only recorded in the status (
status.clusterInfo[].failureMessageof the EventTrigger andstatus.failureMessageof the EventReport) and in the logs. Nothing showed inkubectl describeor in the events of the namespace.event-manager now records a Warning event for both. It uses the
EventTriggerFailedreason defined in libsveltoslib/eventreasons.DeployingDeploying to cluster Sveltos mgmt/prod-1 failed: <error>The EventTrigger is cluster scoped, so its events are in the
defaultnamespace. The cluster is the related object because the events recorder merges events with the same reason, action and regarding object and keeps only the first note: without it, the failures of one EventTrigger in two clusters would be a single event.An event is recorded when the failure message is new or different from the one saved before, not on every retry:
status.clusterInfothrough the Provisioning step of a retry. A retry which fails the same way records nothing, a different error does.setEventReportFailureMessage, which already skips the write when the message is unchanged. Nothing is recorded when the failure is cleared, or when onlyAgentFailureMessagechanged.No event is recorded for expected states: a paused cluster, a cluster which is not ready or down, a cluster whose agent is not healthy (sveltoscluster-manager records that one), and a deployment which is still in progress. Those return before a deployment result exists.