Skip to content

fix(plannings): stop the index endpoint silently dropping workers - #1705

Merged
renemadsen merged 1 commit into
stablefrom
fix/plannings-index-concurrent-list-race
Sep 12, 2026
Merged

renemadsen merged 1 commit into
stablefrom
fix/plannings-index-concurrent-list-race

Conversation

@renemadsen

Copy link
Copy Markdown
Member

The bug

TimePlanningPlanningService.Index() — the endpoint behind the planning dashboard — fans out one async task per assigned site. Every task did this:

var result = new List<TimePlanningPlanningModel>();   // shared by all tasks
var tasks = assignedSites.Select(async dbAssignedSite => {
    ...                                               // several real awaits
    result.Add(siteModel);                            // UNSYNCHRONISED
    return siteModel;
}).ToList();
await Task.WhenAll(tasks).ConfigureAwait(false);      // return value DISCARDED

The Add runs after the awaits, so on arbitrary thread-pool threads, genuinely in parallel. List<T> is not thread-safe — concurrent Add loses elements.

The dashboard could return fewer workers than exist, with no error and nothing in the log. A worker silently missing from the grid is a missed shift, and it presents as a data problem rather than a race.

The fix

Task.WhenAll already returned exactly the data the shared list was collecting, and it was being thrown away. Build the result from it:

var siteModels = await Task.WhenAll(tasks).ConfigureAwait(false);

var result = siteModels
    .OfType<TimePlanningPlanningModel>()   // drops the "SDK site not found" nulls
    .OrderBy(x => Regex.Replace(x.SiteName, @"\d", ""))
    .ThenBy(x => x.SiteName)
    .ToList();

No lock, no ConcurrentBag — not sharing the state is less code than synchronising it. It also makes tie-ordering deterministic: OrderBy is stable, so sites with equal keys now fall back to input order instead of completion order.

.OfType<>() is a real filter here, not a no-op: the lambda's inferred return type is TimePlanningPlanningModel? under this file's #nullable enable.

Also fixed: innerDbContext was created per task and never disposed — one DbContext and pooled connection leaked per site per dashboard load. The two changes are load-bearing together; without the dispose, the new test's fan-out would exhaust the connection pool.

How this connects to the flaky e1m shard

This is the cause. The shard saw 14 of 16 rows, so the positional #cell3_0 selector addressed a different worker: the spec wrote five shifts, reopened what it thought was the same cell, and found it empty — Expected "01:03", Received "".

Confirmed against the failing run's log: the not found in Sites list warning, the only other path to a short row count, never fired.

Test changes

e1m/dashboard-edit-multishift.spec.ts now reads the worker from the first dialog it opens and asserts every later open matches, instead of trusting the row index. It deliberately does not hardcode which worker row 3 is — that depends on the server's collation of the seeded names (under da it is c d; under the host's en-US default it would be ag ah), so pinning a name would couple the spec to a derivation rather than to the property that matters: all five opens must hit the same row.

It also gates on the plannings/index POST that follows every assigned-site save, which it previously raced, and asserts the save reported success: true so a failed save can't surface as an opaque gate timeout.

PlanningServiceMultiShiftTests gains a contract test, not a regression test — stated plainly because it matters: it asserts every assigned site comes back, every time. A lost update can't be forced deterministically, so it repeats the call. It would not reliably fail against the old code, but it never passes wrongly. No workflow change needed; that class is already in shard c.

Known gaps, deliberate

  • b1m, c1m and d1m carry the identical #firstColumn3 / #cell3_0 construct with neither the gate nor the identity check. Out of scope by agreement; the race fix removes the cause for all of them.
  • The PlanRegistrationHelper holiday-config double-checked lock has no volatile — benign on x86/x64, not guaranteed on ARM. Separate follow-up.

Tests run only in CI, so CI is the first execution. Local verification was a clean dotnet build (0 errors) plus a TypeScript parse of the spec.

🤖 Generated with Claude Code

Index() fans out one async task per assigned site. Each task added its
row into a List<T> shared by every task -- after several awaits, so the
Add ran on arbitrary thread-pool threads, genuinely in parallel.
List<T> is not thread-safe: concurrent Add loses elements.

The dashboard could therefore return fewer workers than exist, with no
error and nothing in the log. A worker silently missing from the grid
is a missed shift, and it looks like a data problem rather than a race.

Task.WhenAll already returned exactly the data the shared list was
being used to collect, and that return value was discarded. Build the
result from it instead, dropping the nulls the "SDK site not found"
path returns -- which the old code never added either. No lock and no
ConcurrentBag: not sharing the state at all is less code than
synchronising it, and it makes tie-ordering deterministic, since
OrderBy is stable and ties now fall back to input order rather than
completion order.

Also dispose innerDbContext, which was created per task and never
disposed -- one DbContext and pooled connection leaked per site per
dashboard load. The two changes are load-bearing together: without the
dispose, the new test's fan-out would exhaust the connection pool.

This is what made the e1m Playwright shard flaky. It saw 14 of 16 rows,
so the positional #cell3_0 selector addressed a different worker; the
spec wrote five shifts, reopened what it thought was the same cell and
found it empty ("Expected 01:03, Received ''"). Confirmed against the
failing run's log, where the "not found in Sites list" warning -- the
only other path to a short row count -- never fired.

The e1m spec now reads the worker from the first dialog it opens and
asserts every later open matches, rather than trusting the row index.
It deliberately does not hardcode which worker row 3 is: that depends
on the server's collation of the seeded names, so pinning a name would
couple the spec to a derivation instead of to the property that
matters -- that all five opens hit the same row. It also gates on the
plannings/index POST that follows every assigned-site save, which it
previously raced.

The C# test is a contract test, not a regression test: it asserts that
every assigned site comes back, every time. A lost update cannot be
forced deterministically, so it repeats the call -- it would not
reliably fail against the old code, but it never passes wrongly.

Tests run only in CI, so none of this has executed locally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 11, 2026 11:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No unresolved issues were identified that would block approval.

Pull request overview

Fixes the planning dashboard race that dropped workers during concurrent site processing and prevents per-site DbContext leaks.

Changes:

  • Builds results from Task.WhenAll and adds deterministic ordering.
  • Disposes per-site DbContexts.
  • Adds backend contract coverage and strengthens the E2E worker-identity and reload checks.
File summaries
File Description
eFormAPI/Plugins/TimePlanning.Pn/TimePlanning.Pn/Services/TimePlanningPlanningService/TimePlanningPlanningService.cs Updated as part of this pull request.
eFormAPI/Plugins/TimePlanning.Pn/TimePlanning.Pn.Test/PlanningServiceMultiShiftTests.cs Updated as part of this pull request.
eform-client/playwright/e2e/plugins/time-planning-pn/e1m/dashboard-edit-multishift.spec.ts Updated as part of this pull request.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@renemadsen
renemadsen merged commit b8afe22 into stable Sep 12, 2026
77 of 78 checks passed
@renemadsen
renemadsen deleted the fix/plannings-index-concurrent-list-race branch September 12, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants