fix(scripts): give the render-smoke overlap test room for process startup - #2178
Conversation
There was a problem hiding this comment.
Pull request overview
Expands timing margins in the render-smoke overlap regression test to prevent load-related flakes.
Changes:
- Increases render timeout to 1200 ms.
- Increases survival window to 1500 ms.
- Documents the timing relationship being tested.
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
|
Correct on both counts, and thank you — the fix was right for the wrong reason. Fixed in 30beabb. You are right that this path never touches Re-measured the way the test actually spawns (plain So the old 150ms budget had roughly 90ms of slack idle, not the ~36ms I claimed. That is genuinely useful to know rather than a wording nit: it explains why this presented as an occasional flake instead of an obvious bug, and it means the cause is scheduling delay under load — hitting both the child's boot and the parent's timer callback during a full The comment now says that, names Worth noting the fixture is still verified by mutation rather than by going green: removing the |
bb7ecf0 to
d446de5
Compare
…rtup The test gave itself a 150ms budget that had to cover `script(1)` PTY allocation plus Node boot: `paintAt` is measured from the child's first JS tick, while `renderTimer` is armed in the parent at spawn. Measured idle, that startup is 66-114ms — ~36ms of slack at best — and under the load of a full `local:gate` run it goes over, so the render timer fires before the paint and the run is reported as "did not render", which is precisely the misdiagnosis this test exists to catch. It failed three gate runs in a row while passing standalone every time. The constants are widened rather than the assertion changed: the shape under test is the overlap (paint < timeoutMs < paint + surviveMs), and 1200/1500 keeps it with real margin on both sides. Verified by mutation — removing the `clearTimeout(renderTimer)` this test guards makes it fail at 1204ms, i.e. the timer still fires inside the survival window. Closes #2177 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev Signed-off-by: cliffhall <cliff@futurescale.com>
…ment Copilot round 1. The comment blamed `script(1)` PTY allocation, but this test's `stub()` spawns `process.execPath` directly — the PTY wrapper lives in `smoke-tui.mjs`, the caller — so the fixture never pays that cost and the 66-114ms figure (measured through the wrapper) was not this path's. Re-measured without it: Node boot alone is 46-59ms idle, which fits the old 150ms budget comfortably. That is *why* it read as a flake rather than a bug — what actually exhausted the budget was scheduling delay under a full gate run, on both the child's boot and the parent's timer. The fix is unchanged; only the reasoning was wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev Signed-off-by: cliffhall <cliff@futurescale.com>
474c43c to
7083c6c
Compare
Closes #2177
scripts/lib/render-smoke.test.mjs→ "a paint landing just under the deadline is not failed by the render timer" failed three consecutivenpm run local:gateruns while passing every time standalone, which failstest:scripts→validate→ the whole gate.Cause
The test gave itself a 150ms budget that has to cover process startup:
paintAtis measured from the child's first JS tick;renderTimeris armed in the parent, at spawn (render-smoke.mjs:324). So those 150ms have to absorbscript(1)PTY allocation plus Node boot before the child's own timer even starts.Measured on this machine, idle, 8 samples of spawn → first byte of child JS output through the same PTY wrapper:
~36ms of slack at best with nothing else running. Under gate load it goes over, the render timer fires before the paint, and the run is reported as
did not render "MCP Servers" within 400ms— which is exactly the misdiagnosis this test exists to catch.Fix
Widen the constants, not the assertion. The shape under test is an overlap, and only an overlap:
timeoutMs: 1200/surviveMs: 1500keeps that with real margin on both sides — worst-case paint lands near 365ms (well under 1200), and 1200 still falls inside a survival window that closes no earlier than ~1750ms. The comment now spells out why both margins are wide and warns against tightening them back for speed.Verified by mutation, not just by going green
A widened timing test can pass vacuously, so I checked it still fails for the right reason. Removing the
clearTimeout(renderTimer)on first paint (render-smoke.mjs:267) — the line this test guards:It fails at 1204ms, i.e. the render timer still fires inside the survival window. Restored, 15/15 pass.
Verification
npm run local:gategreen end to end, withtest:scriptsat 292/292.Cost: about a second of extra runtime on one test.
🤖 Generated with Claude Code
https://claude.ai/code/session_013hzwzS8UvBsr4yZm7o7sev