Repository navigation
fix(workflow-host): a lone recall function is never shown waiting behind others - #137
Merged
Merged
Conversation
…o event starts, not 61 The test published 61 events one after another into the host's SQLite file while the host's follower started a run for each on a second connection to the same file, so its length grew with the start rate and with the machine's load: 94 ms alone, 507 to 643 ms beside 64 busy threads, and 5,049 ms against the 5,000 ms limit in the full gate. The test now records that the workflow was started 59 times in the current minute, as the backlog test of start-rates.test.ts records the starts already waiting, and then publishes two events: the every schedule still starts its run, the first event makes the 60th start and the second waits for a later minute. A schedule start counted in the rate leaves both events waiting, and an event start left out of it starts both, and each fails the test. That 60 starts are admitted one by one stays proven by start-rates.test.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its first fold The test waited for the recall function's standing to read rebuilding, which the standing also reads before the projector has added the view's row (recall-standing.ts answers rebuilding when no view is kept). The projector adds the row as waiting and moves it to rebuilding in a second write (view-reconciling.ts), so the recall the test made next answered whichever of three details held at that moment: not begun, waits behind other recall functions, or is building. Alone, one run in eight saw the standing before the row existed, and beside 64 busy threads one run in twelve failed on the waiting detail. The test now waits until the gate holds the rebuild's first fold, which the projector asks for only after it has set the row to rebuilding and which holds the view at 0 folded events, so the building detail is the one the situation gives. The test of a function that waits behind another, which shows the waiting detail, waits the same way, with the bound the standing waits use rather than vi.waitFor's default second. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ind others The projector wrote a new or renewed view as waiting and then, in a second write, moved the views that took a slot to rebuilding. Between the two writes a brain's only recall function read as waiting, and a run of it answered that it waits behind other recall functions of the brain being built, when there were none. A new or renewed view is now written in the phase its place gives it: the slots are counted over the views still building and the fresh ones, in the order saved, before anything is written, and a dropped view is removed first, so a fresh view never takes a slot a dropped one still holds. The rows suite watches the row after every write a reconcile makes and sees a view built alone as rebuilding from its first write, new and renewed; on the earlier code it sees waiting after each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d fold, not a clock The test measured that the recall answered within 2,000 ms. The gate already holds the rebuild's first fold until after the answer, so the rebuild cannot finish before it: a recall that waited for its view would never answer, and the answer's detail says the view has folded 0 events. The wall-clock bound added only a dependence on the machine's load, and is removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er's row changes When the view holding a slot was retired or renewed in the reconcile that freed it, the view waiting behind it kept reading waiting beside the free slot until the phase writes after the next read of the rows: a retired holder's row was deleted first, and a renewed holder left both rows waiting for a moment. A still-building view that takes a slot is now moved to rebuilding first, before the drops and before the new and renewed rows. With the drops first, the rows suite sees the waiting view alone and waiting between the delete and its move; moving it first leaves the retired holder and it both rebuilding until the delete, which no standing shows, since a retired function has none. The suite records the rows after every write of a reconcile, with the holder retired while another view is saved and renewed while another waits, and no row reads waiting while fewer rows rebuild than there are slots. Since a view that takes a slot now moves with the other changes, the writes after the rows are read again only move back to waiting the rows past the slots, as when the brain may build fewer views at once than it did; the suite now covers that case too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two tests passed alone and failed when every package's tests ran at once. Each now waits for, or sets up, the state it asserts, so how busy the machine is no longer changes the result.
The start rate of a workflow with several triggers
packages/workflow-host/src/triggers/several-triggers.test.tsproved that a workflow's event trigger starts it 60 times a minute and that its schedule's start is not counted. To get there it published 61 events one after another into the host's SQLite file, while the host's follower started a run for each over a second connection to the same file. So the test's length grew with the limit and with the load on the machine: 94 ms alone, 507 to 643 ms beside 64 busy threads, and 5,049 ms against the 5,000 ms limit in the full gate. It never waited on the clock; the work was the 61 deliveries.The test now records that the workflow was already started 59 times in the current minute, the way
start-rates.test.tsrecords starts already waiting in its backlog test. It then saves the workflow and publishes two events. The every schedule starts its run, the first event makes the 60th start and the second waits for a later minute. If a schedule start were counted, both events would wait. If event starts were not counted, both would start. Each of those faults, put intoreactions/reacting.tsorreactions/start-rates.ts, fails the assertion.start-rates.test.tsstill proves that 60 starts are admitted one by one. The test takes 37 ms alone and 166 to 338 ms beside 64 busy threads.A recall function whose view is being built
packages/server/src/recall/recall-versions.test.tswaited for the recall function's standing to readrebuilding. The standing also readsrebuildingbefore the projector has added the view's row (primitives/recollection/src/primitive/recall-standing.ts). The projector added the row aswaitingand moved it torebuildingwith a second write (packages/workflow-host/src/projector/view-reconciling.ts). So the recall the test made next could answer any of three details, depending on how far the projector had got: not begun, waits behind other recall functions, or is building. Alone, one run in eight returned from the wait before the row existed. Beside 64 busy threads, one run in twelve failed on the waiting detail.The test now waits until the test's fold gate holds the rebuild's first fold. The projector asks for that fold only after it has set the row to
rebuilding, and while the fold is held the view stays at 0 folded events, so the "is building" detail is the only one possible. The waiting detail is a real state too: it is what a function shows when another function in its brain holds the only rebuild slot. The test for that state was already deterministic in its order. It now waits for the held fold through the same helper,foldsHeld, bounded like the standing waits instead of byvi.waitFor's default of one second.The same test also checked that the answer came within 2,000 ms. The held fold already proves what that bound stood for: the rebuild cannot finish while its first fold is held, so a recall that waited for its view would never answer, and the answer says the view has folded 0 events. The wall-clock bound is removed.
A lone recall function is never shown waiting
Between the projector's two writes, a brain's only recall function read as
waiting, and a run of it answered that it "waits to build its view of version 1, behind other recall functions of this brain being built", when there were none. The projector now writes a new or renewed view in the phase its place gives it. It counts the slots over the views still building and the new ones, in the order saved, before it writes anything. It removes a dropped view first, so a new view never takes a slot that a dropped one still holds. The rows suite, on SQLite and PostgreSQL, reads the row after every write a reconcile makes. It sees a view built alone asrebuildingfrom its first write, both new and renewed; on the earlier code it seeswaitingafter each. The README's account of the projector says so.The same holds when the view holding a slot is retired or renewed while another waits. The waiting view is moved to
rebuildingbefore the holder's row is deleted or reset, and before any new row is written. Otherwise it would readwaitingbeside a free slot until a later write. The rows suite covers both cases: the holder retired while another view is saved, and the holder renewed while another waits. After every write it finds no row waiting while fewer views rebuild than there are slots. Deleting the holder first would fail the retirement case, because the waiting view readswaitingwith nothing rebuilding between the delete and its own move. Moving it first means the retired holder and the promoted view both readrebuildinguntil the delete. A retired function shows no standing, so no one sees that moment. The writes after the rows are read again now only move rows past the slots back towaiting, as when the brain may build fewer views at once than before. The rows suite covers that case as well.Checks
pnpm checkwithout PostgreSQL: passed, 55 of 55 tasks, with 5,188 tests passing and the 194 PostgreSQL tests skipped.