A paused feed's checkpoint is proved by order, not by waiting 50ms - #764
Merged
Merged
Conversation
TestInvariantF1APausedFeedDoesNotMoveTheCheckpoint scripted one page, waited for the queue to pause, slept 50ms and asserted the checkpoint was absent. That is a window, not an order: a save that happens but happens slowly is a pass, so the test could false-pass and could never false-fail. It is worse than a loose window. The pause the test waits for happens INSIDE the page — the feed blocks offering the page's second event — so PageDelivered has not run for any page at the moment of the assertion. The test never observes a completed page while paused at all, so it has no guard against a per-page save, fast or slow. The walk is two pages now. The entry is present-class, so the feed holds every page's position and saves once: after the walk reaches its frozen head and the drain has accepted every event. The second poll is the barrier — the feed issues it only after every event of the first page was accepted and PageDelivered ran for it — so the checkpoint is read at three points the feed's own progress orders: paused mid-page-one, after the walk followed `next` into page two, and paused mid-page-two with the second page's first event already in the ledger. Each read is sequenced after any save that moved with a page, rather than 50ms after a clock. No sleep is left in the test. A test that cannot fail cannot be proved by reverting it, so the evidence is the other way round. With intake's PageDelivered observer made to save the page's position durably there and then — a paused feed really moving the checkpoint — the old test passes 20/20 plain and 10/10 under -race, and the new one fails 20/20 and 10/10 on "a page delivered in full is still not a position saved". With that regression removed the new test passes 30/30 plain and 20/20 under -race. invariants_test.go had exactly one sleep, this one. The rest of the file asserts after something synchronous has returned or after a WaitGroup, so they are ordered already.
Contributor
There was a problem hiding this comment.
🟢 Approved
The revised test deterministically verifies the intended checkpoint ordering without timing-based assertions.
Pull request overview
Makes the paused-feed checkpoint invariant deterministic by replacing a timing window with feed-progress barriers.
Changes:
- Exercises a two-page feed walk with pauses mid-page.
- Checks checkpoint ordering at three deterministic points.
[!TIP]
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or rungh pr ready --undo.
Click "Ready for review" or rungh pr readyto reengage.
File summaries
| File | Description |
|---|---|
internal/connector/invariants_test.go |
Reworks the paused-feed checkpoint invariant test around observable ordering. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An invariant test that could only ever pass.
TestInvariantF1APausedFeedDoesNotMoveTheCheckpointwaited for the queue to pause, slept 50ms, and asserted the checkpoint was absent. That is a window, not an order: a save that happens but happens slowly is a pass. The test could false-pass and could never false-fail, which is why nothing has ever reported it — and why it would stay quiet through exactly the regression it exists to catch.Originally tracked in A paused-feed test asserts absence after a sleep, so it can only pass wrongly, found while fixing The feed test reads the walk's position once the walk has saved it and deliberately left alone there so as not to bury that PR's evidence. The test arrived with Take the account event feed into a durable ledger.
It is worse than a loose window, and that is the part worth knowing. The pause the test waits for happens inside the page: the feed blocks offering the page's second event, so
PageDeliveredhas not run for any page at the moment of the assertion. The test never observes a completed page while paused at all. Its 50ms is not a weak guard against a per-page save — there is no guard, at any speed.The invariant is an order and is now tested as one. The walk is two pages. The entry is present-class, so the feed holds every page's position and saves once, after the walk reaches its frozen head and the drain has accepted every event. The second poll is the barrier: the feed issues it only after every event of the first page was accepted and the page was announced delivered. So the checkpoint is read at three points the feed's own progress puts in order — paused mid-page-one, after the walk followed
nextinto page two, and paused mid-page-two with the second page's first event already in the ledger — and each read is sequenced after any save that moved with a page rather than 50ms after a clock. The sleep is gone.Proving a change to a test that cannot fail
Reverting it and watching it go red proves nothing here, because the old test passes by construction. So the evidence runs the other way: break the production code and see which test notices. With intake's
PageDeliveredobserver made to save the page's position durably there and then — a paused feed really moving the checkpoint, so that a crash while paused would resume after a page whose events are still in the queue:-raceThe new test fails on
a page delivered in full is still not a position saved. With the regression removed, it passes 30/30 plain and 20/20 under-race.invariants_test.gohad exactly one sleep, this one. Everything else in the file asserts after something synchronous has returned or after aWaitGroup, so it is ordered already.