Skip to content

Add retention sweep outcome tracking and reporting - #5916

Merged
warwickschroeder merged 1 commit into
masterfrom
warwick/retention-purge-outcome
Sep 25, 2026
Merged

warwickschroeder merged 1 commit into
masterfrom
warwick/retention-purge-outcome

Conversation

@warwickschroeder

Copy link
Copy Markdown
Contributor

What this fixes

The manual retention purge status could not tell a purge that failed from one that worked: every pass swallowed its own error, and the finish time was stamped either way. This adds an outcome and an error to the status, and fixes three nearby ways a cancelled or finished sweep was misreported. It does not change what the sweep deletes, the hourly schedule, or the RavenDB 501 responses.

  • Outcome and error on the status. GET /api/maintenance/retention/purge/status gains last_outcome (succeeded, failed, cancelled) and last_error, one "<Pass>: <first line of the message>" entry per failed pass. A cancelled run keeps the errors of passes that failed before it. The full error stays in the log. New RetentionSweepOutcome and RetentionPurgeOutcome enums, mapped in RetentionApi.
  • One status snapshot per run. RetentionSweeper keeps a single volatile RetentionSweepCurrentStatus that is replaced whole, so a poll never mixes two runs. Starting a run clears the previous finish time and outcome. The status covers the hourly sweep as well as the manual one.
  • Cancelled between batches is cancelled. Both batch loops now throw on cancellation at the top instead of leaving the loop, so a pass cut short is recorded as cancelled in the metrics and the status, and the sweep stops there instead of running the later passes.
  • SQL Server cancellations are not failures. SqlClient reports a command cancelled mid-flight as a SqlException, so the type-based filters recorded a shutdown as a failed pass and logged it as an error. The catches in RunPass, SweepBody, the manual path and ExecuteAsync now go by the sweep's token. PS0019 is suppressed with that reason.
  • No 409 straight after a run finishes. The lock is released before the finished status is published, and the status is swapped with Interlocked.CompareExchange so a run that starts in that gap is not overwritten. A caller who sees is_running: false finds the lock free. A short wait in TryStartManualSweep was rejected because it let a fast second sweep start instead of being refused.

Tests

  • ServiceControl.Persistence.Tests/EFCore/RetentionSweepTests: outcome on success, failed passes (every error listed, first line only), cancellation before and during a query, cancellation between batches for failed messages and the event log, errors kept on cancel, the hourly path, and clearing at start. Two command interceptors drive the cancellation cases; the during-a-query test fails on SQL Server with the old filters.
  • ServiceControl.UnitTests/Infrastructure/RetentionApiTests (new file): every outcome maps to the API enum of the same name, and the error passes through.
  • ServiceControl.AcceptanceTests/WebApi/When_triggering_a_manual_retention_purge: a completed purge reports succeeded over HTTP.

@warwickschroeder
warwickschroeder merged commit af31ba1 into master Sep 25, 2026
36 checks passed
@warwickschroeder
warwickschroeder deleted the warwick/retention-purge-outcome branch September 25, 2026 00:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants