Skip to content

fix(cdp): wait on the resolver job row, not the mock counter - #95850

Draft
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixcdp-wait-on-the-resolver-job-row-not-d17157
Draft

fix(cdp): wait on the resolver job row, not the mock counter#95850
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/fixcdp-wait-on-the-resolver-job-row-not-d17157

Conversation

@posthog

@posthog posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Problem

  • Engineers on the Node.js serial Tests 3/3 shard get a red build from a racing assertion, three times in the last week. Each hit blocks a merge and costs a rerun of the shard.
  • The case "audience fetch failure: resolver reschedules with backoff" polls fetchAttempts, a counter the mocked audience fetch increments when the fetch starts.
  • The resolver writes status = 'available' later, after its catch block calls job.reschedule and the worker flushes to Postgres.
  • The single row read after the poll can therefore still see running. The other assertions in the case hold either way, so exactly one line flakes.
  • The sibling case above it polls putAttempts the same way. It is a latent second instance.

Changes

  • Both cases now wait on the database state they assert, so the shard stops going red on a wait that finished early. The row read moves inside waitForExpect.
  • The audience fetch case also asserts state.attempts >= 1. Without it the poll could pass on the job's initial row, which is already available with a null cursor and zero counters.
  • No retry wrapper, and no change to the resolver. The race is in the test-side wait logic only.
  • The counter assertions stay, after the poll, to keep the "the mock was actually reached" fact in the case.
  • Nothing user-visible changes. This is a test-only diff.

How did you test this code?

  • Ran the batch resolver e2e cases six times in a row against a local dev stack. All 13 passed each time.
  • The failure mode is timing dependent, so a green local run is evidence of no regression, not proof the race is gone. The argument for the fix is structural: the assertion and the wait now read the same row.
  • Not run: the rest of the serial shard, and the parallel suites. CI covers those.
Local run command
npx jest --config jest.serial.config.js --runInBand --forceExit \
  --testPathPatterns='cdp/workflows-e2e.serial.test.ts' -t 'resolver'

Automatic notifications

  • Publish to changelog?

Docs update

None. The change touches no user-facing behavior.

🤖 Agent context

Autonomy: Fully autonomous

  • Written by Claude Opus 5 in a PostHog Desktop cloud task, from a self-driving inbox report on the flaky shard.
  • Skills invoked: /writing-pr-descriptions, /simplify, /writing-simplified-technical-english.
  • No duplicate: gh pr list --state open --search "workflows-e2e serial flaky resolver" found nothing that fixes this.
  • The state.attempts assertion was not in the report's plan. Moving the read alone would have let the audience fetch case pass on the job's initial row, which already matches every assertion in it.
  • /simplify found nothing to cut. The duplicated cyclotron_jobs query matches two sibling cases in the same file, so no helper was extracted.
  • Rebase: PR fix(cdp): give batch workflow audience resolution its own timeout #93606 edits this file at line ~3994, well clear of these hunks.
  • Public artifact: nothing here comes from a non-public source. The diff is test wait logic.

Created with PostHog Desktop from this inbox report.

Two batch resolver e2e cases polled a counter that the mocked fetch bumps when the request starts, then read cyclotron_jobs once. The reschedule reaches Postgres after that, so the status read could still see 'running'.

Move the row read inside waitForExpect in both cases. Add an attempts assertion to the audience fetch case, because the job also starts out available.

Generated-By: PostHog Desktop
Task-Id: c071f9b1-4feb-4834-a6f4-fb65da280a89
@trunk-io

trunk-io Bot commented Sep 7, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@posthog

posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

🦔 PostHog Review reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A happy dog on a sunny path

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Test-only change to a serial e2e spec that fixes a real race by waiting on the DB row instead of a mock counter; diff matches the description exactly and carries no production or infra risk.

Gate mechanics and policy version
Gate Result
prerequisites all clear
deny-list no deny categories matched
size 0L, 0F substantive, 73L/1F incl. docs/generated/snapshots — within ceiling
tier T0 auto-approve: T0-deterministic (73L, 1F, single-area, fix)
stamphog 2.0.0b4 .stamphog/policy.yml @ 7fbf176 · reviewed head 7fbf176

@trunk-io

trunk-io Bot commented Sep 7, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants