Skip to content

Properly await throughput recording in MonitoringService - #5784

Merged
johnsimons merged 1 commit into
masterfrom
john/bug_found
Aug 17, 2026
Merged

johnsimons merged 1 commit into
masterfrom
john/bug_found

Conversation

@johnsimons

Copy link
Copy Markdown
Member

Problem

MonitoringService.RecordMonitoringThroughput iterated the endpoints in an incoming
throughput message with List<T>.ForEach:

message.EndpointThroughputData?.ToList().ForEach(async e => { ... await ... });

ForEach takes an Action<T>, so the lambda was async void. That has three consequences:

  1. RecordMonitoringThroughput returned as soon as the first await suspended, so the
    transport handler acknowledged the message while the writes were still outstanding. If
    the process stopped in between, the data was gone and the message was not redelivered.
  2. Exceptions inside the lambda were unobserved instead of surfacing to the handler.
  3. Every endpoint in a message was written concurrently against the same data store.

This is the path where a Monitoring instance reports throughput to the primary instance, and
that data feeds the usage report customers send to Particular. Losing it means under reported
usage.

How it was found

This came out of the EF acceptance tests work. It was not caught by the existing unit
tests because each of them sends a message containing a single endpoint, so there is nothing
to lose.

Using List.ForEach with an async lambda resulted in async void calls that were not awaited, meaning the service could return before all endpoint data was persisted. Replacing this with a standard foreach loop ensures every operation is correctly awaited.
@johnsimons
johnsimons requested a review from rbev August 17, 2026 06:54
@johnsimons johnsimons self-assigned this Aug 17, 2026
@johnsimons
johnsimons merged commit bb12b9a into master Aug 17, 2026
105 of 106 checks passed
@johnsimons
johnsimons deleted the john/bug_found branch August 17, 2026 08:33
@rbev rbev added this to the 6.20.0 milestone Sep 3, 2026
@jpalac jpalac added the Bug label Sep 3, 2026
@jpalac jpalac modified the milestone: 6.20.0 Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants