Skip to content

Fix flaky runtime and transport sync test waits - #1138

Merged
tnotheis merged 2 commits into
mainfrom
agent/fix-loki-identity-deletion-flakes
Aug 13, 2026
Merged

Fix flaky runtime and transport sync test waits#1138
tnotheis merged 2 commits into
mainfrom
agent/fix-loki-identity-deletion-flakes

Conversation

@tnotheis

@tnotheis tnotheis commented Aug 12, 2026

Copy link
Copy Markdown
Member

What changed

  • Replaced the fixed 15-iteration polling limit in the runtime test sync helpers with a 10s timeout budget.
  • Replaced the fixed 20-iteration polling limit in the transport test sync helper with a 60s timeout budget.
  • Capped per-iteration sleeps so tests stay responsive while giving Loki/MongoDB/Backbone event propagation enough time under CI load.
  • Included the timeout budget in thrown errors to make future sync wait failures easier to diagnose.

Why

The failing test-runtime-loki job timed out while waiting for identity-deletion peer events. When the cleanup hook timed out, later message tests inherited stale peer deletion state and failed with error.runtime.messages.peerIsInDeletion.

The same workflow run also had test-transport-mongodb failures in packages/transport/test/testHelpers/TestUtil.ts, where relationship reactivation tests hit syncUntil condition was not met. That helper had a separate fixed iteration budget, so this PR applies the same timeout-budget approach there too.

Validation

  • npm run build:node
  • USE_LOKIJS=true npx jest -i test/transport/identityDeletionPeerProcessing.test.ts test/transport/messages.test.ts passed identityDeletionPeerProcessing; the full local run hit an unrelated local Backbone recipient-limit failure in Messaging > send a Message to multiple recipients (count max 1 locally).
  • USE_LOKIJS=true npx jest -i test/transport/messages.test.ts -t 'Message errors for peers that are in deletion|Postponed Notifications for cancelled Identity deletion|Mark Message as un-/read|Message query'
  • npx prettier --check packages/runtime/test/lib/testUtils.ts
  • npx prettier --check packages/transport/test/testHelpers/TestUtil.ts
  • git diff --check

Notes:

  • npm run lint:tsc --workspace=@nmshd/runtime currently fails locally on existing dependency declaration issues in node_modules plus test/lib/AdminApiClient.ts, unrelated to this change.
  • npm run lint:tsc --workspace=@nmshd/transport currently fails locally on existing Loki declaration globals plus test/testHelpers/AdminApiClient.ts, unrelated to this change.
  • I could not rerun the Docker-backed local transport MongoDB Jest target after this second commit because the local sandbox was denied access to the Docker API socket, even after requesting filesystem access to the socket.

@tnotheis
tnotheis marked this pull request as ready for review August 12, 2026 14:02
@tnotheis

Copy link
Copy Markdown
Member Author

@britsta @Milena-Czierlinski I asked Codex to fix the flaky tests. This is what it suggested. I'd say we merge it to main to see if it helps. The changes are not that absurd.

@tnotheis tnotheis added the test This pull request contains only new or changed tests label Aug 12, 2026
@tnotheis tnotheis self-assigned this Aug 12, 2026
@tnotheis tnotheis changed the title Fix flaky runtime sync test waits Fix flaky runtime and transport sync test waits Aug 12, 2026

@britsta britsta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if that helped 🙏

@tnotheis
tnotheis merged commit ac8c032 into main Aug 13, 2026
15 checks passed
@tnotheis
tnotheis deleted the agent/fix-loki-identity-deletion-flakes branch August 13, 2026 07:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test This pull request contains only new or changed tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants