Skip to content

chore(flags): wait for the test server in rate-limit tests - #95854

Draft
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/flakyflags-wait-for-the-test-server-a7ef45
Draft

chore(flags): wait for the test server in rate-limit tests#95854
posthog[bot] wants to merge 1 commit into
masterfrom
posthog-self-driving/flakyflags-wait-for-the-test-server-a7ef45

Conversation

@posthog

@posthog posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Problem

  • test_rate_limit_replenishment was quarantined again on 2026-09-07, so the /flags rate-limit guard no longer blocks a regression in CI.
  • The test posts to /flags right after ServerHandle::for_config_*, which starts the server in tokio::spawn. No test in this file waits for the server.
  • When startup fails inside that task (a dependency the runner cannot reach, a missing local asset), serve returns early and drops the listener. The first request then fails with a bare transport error and no assertion text — which is what the Trunk event for this test shows.
  • The previous fix (65ab79a) removed real-time sleeps with a fake clock. Timing is now deterministic, so startup is the remaining source of failure.

Changes

  • Every test in rust/feature-flags/tests/test_rate_limiting.rs calls server.wait_until_ready().await before its first request. The helper already existed and is used by four other test files.
  • A startup failure now reports Server failed to become ready within 5 seconds instead of a transport error, so the next flake names its own cause.
  • All 20 tests in the file get the line, not only the quarantined one. They share the same race and the same silent failure mode.
  • The readiness route sits outside the rate-limited router, so the extra request consumes no token or IP bucket capacity. Rate-limit assertions are unchanged.

How did you test this code?

Automated only. No manual product testing applies to a test-only change.

  • cargo test -p feature-flags --test test_rate_limiting — 20 passed.
  • cargo test -p feature-flags --test test_rate_limiting test_rate_limit_replenishment — 8 consecutive runs, all passed.
  • No new test was added. The change makes an existing regression guard trustworthy again.
Failure-mode comparison, forced by removing a local startup asset

Server init fails in both runs. The difference is what the test reports.

Branch Reported failure
master client error (SendRequest)connection errorConnection reset by peer (os error 104)
this PR Server failed to become ready within 5 seconds

Not checked: whether Trunk clears the quarantine. That needs several green runs on master after this lands.

Automatic notifications

  • Publish to changelog?

Docs update

None. The change touches test code only.

🤖 Agent context

Autonomy: Fully autonomous

  • Written by Claude Opus in PostHog Desktop, from an inbox report about the quarantined test. No repo skill applied to a Rust test-only diff.
  • No duplicate: gh pr list --state open --search for test_rate_limiting and for the flaky test name returned nothing.
  • Considered and rejected: keeping the fix to the one quarantined test. The other 19 tests build the server the same way, so a partial fix leaves the same trap for the next report.
  • Also examined the rate-limiter cleanup task and the governor fake-clock arithmetic as alternative causes. Both are deterministic here, so neither explains a flake after the clock fix.
  • The startup failure mode was confirmed locally by removing a startup asset and running the test with and without the change, not inferred from a CI log — the Trunk event carries no error text.
  • Public artifact: nothing in this PR comes from the session beyond what the repository and the linked report already hold.

Created with PostHog Desktop from this inbox report.

The rate-limiting integration tests posted to /flags immediately after
spawning the server. If the spawned task failed during startup, the
listener was dropped and the first request failed with a bare connection
error, which gives no clue about the cause.

Each test now polls /_readiness first. The readiness route is outside the
rate-limited router, so it consumes no tokens.

Generated-By: PostHog Desktop
Task-Id: 4fc356bd-b80d-49db-8ac4-ed1418a38cec
@trunk-io

trunk-io Bot commented Sep 7, 2026

Copy link
Copy Markdown

Merging to master in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@posthog

posthog Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor Author

🦔 PostHog Review reviewed this pull request

Nothing worth raising this time, so here's a calming picture instead:

A happy dog on a sunny path

@posthog posthog Bot added the skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs label Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

🤖 CI report

Trunk lane — non-backend lane

This PR is assigned to the non-backend lane. It does not run backend Python tests and may merge in parallel with PRs in other lanes.

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Test-only, additive change: adds an existing readiness-wait helper before the first request in 20 rate-limit tests, matching the description exactly. No risky territory touched, no outstanding comments or in-flight reviews.

Gate mechanics and policy version
Gate Result
prerequisites all clear
deny-list no deny categories matched
size 0L, 0F substantive, 20L/1F incl. docs/generated/snapshots — within ceiling
tier T0 auto-approve: T0-deterministic (20L, 1F, single-area, chore)
stamphog 2.0.0b4 .stamphog/policy.yml @ 124c132 · reviewed head 124c132

@github-project-automation github-project-automation Bot moved this to Approved in Feature Flags Sep 7, 2026
@trunk-io

trunk-io Bot commented Sep 7, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature/feature-flags Feature Tag: Feature flags skip-agent-review Save $$$, skip auto agent reviews (Greptile) — use for trivial or chore PRs team/feature-flags

Projects

Status: Approved

Development

Successfully merging this pull request may close these issues.

0 participants