Skip to content

ci: cmd/codeaf finishes inside its touched leg (busy case 200 ms, codeaf leg 25m) - #2296

Merged
AbirAbbas merged 3 commits into
workstreams/mainfrom
workstreams/eng-ci
Nov 3, 2026
Merged

AbirAbbas merged 3 commits into
workstreams/mainfrom
workstreams/eng-ci

Conversation

@AbirAbbas

@AbirAbbas AbirAbbas commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The PR gate's touched codeaf leg on workstreams/main hits the 15-minute package timeout (panic: test timed out after 15m0s), so every cmd/codeaf test after the cut goes unchecked. This is ENG-CI of the Workstreams plan.

Measured with a per-test JSON run of the whole package on the CI runner at dd749e133:

time
whole package 1065 s (17.8 min), 3128 top-level tests
tests that predate the workstream work 1052.7 s; 60 tests of 5 s or more take 792 s (slowest: TestAppRelayOutageO1O2O3O4O5 93 s, TestAppRelayRestartP2UnknownUntilPeerReturnsThenActuallyOffline 66 s, TestC27NewChatByDefaultStaysQuiet 55 s, TestMemoryPausedJobLifecycle 39 s)
workstream tests 11.2 s, of which 10.04 s is one subtest: TestAppWorkstreamWireConformance/errors/busy waiting out SQLite's real 10 s busy timeout

The base branch's own run at 19f4df464 times out the same way, so the base alone is over the limit.

Changes

  1. tests: the conformance busy case sets PRAGMA busy_timeout to 200 ms on the backend's own single store connection, reads it back and fails unless it took, then holds the same real writer on a second handle. The refusal still goes SQLITE_BUSY -> ErrBusy -> wire busy through the real store and dispatch; no synthetic error. Production keeps its 10 s. Measured after, same base: the busy subtest 10.04 s -> 0.22 s (passes), the workstream tests 11.2 s -> 1.8 s; with app: workstream agent prompt and tools (proposals, drafts, its own pages) #2289's agent tests also in, 42 workstream tests take 2.0 s.
  2. ci: only the codeaf leg passes --timeout 25m to touched-verdict.sh (a single expression on the existing line, with a comment giving the measurement). tui3, session and rest pass no --timeout and keep the Makefile's 15m. A timeout is still red. Sharding cmd/codeaf failed in ci: touched packages runs in parallel legs and re-checks a failure before blaming the change #1734, so it is not sharded here. docs/rules/ci.md records the exception and that it goes once the suite is back under 15 minutes.

No t.Parallel() and no seeding changes: the rest of the workstream tests total about 1.2 s.

Validation contract

  • (a) The busy case still gets the wire busy code from a real lock held by a second handle, through the backend's real store and error mapping. Test: TestAppWorkstreamWireConformance/errors/busy (assertions unchanged).
  • (b) The backend's handle waits 200 ms, and the test fails if that is not in effect. Test: the read-back check in workstreamWireHoldWriter.
  • (c) Production keeps the 10 s busy wait: no diff under internal/workspace/ or in any non-test Go file.
  • (d) Only the codeaf leg gets 25 minutes: the one ci.yml line; the touched codeaf leg on this PR finishes without the timeout.
  • (e) docs/rules/ci.md states the exception and why.

CI on this PR

touched codeaf ran the whole package in 1035 s (17.3 min) with --timeout 25m: no timeout. It is red only on failures also red on workstreams/main's own runs: TestAppChatRowsR2SavedTurnTime and TestAppChatStartupR3QuietRelease (attribution inconclusive, as on the base) and TestRecipeDoorWritesTheWorktreeWhenOnlyTheCheckoutIsHeld (already failing on the base). light gate is red only on the internal/session laws ratchet (TestNoRoadInTheTaskEngineHasMoreEndingsThanItsLedgerRow, TestNoRoadInTheMemoryEngineHasMoreEndingsThanTheCeiling), also red on the base. For comparison, the codeaf leg on workstreams/main at dd749e133 and 4dcd5150f timed out at 15m.

Left for the base's owners

The real fix is the base's long real-time waits (the 60 tests of 5 s or more, many of them exactly 10, 15, 20, 25 or 30 s). Once the suite is back under 15 minutes, drop the 25m line. Full check runs cmd/codeaf under the same 15m and is not changed here.

🤖 Generated with Claude Code

AbirAbbas and others added 3 commits November 3, 2026 03:25
The busy error case held a real writer on a second handle and let the
backend's store wait out the full 10-second SQLite busy timeout. That one
subtest was 10.0 of the workstream tests' 11.2 seconds in cmd/codeaf.

It now sets PRAGMA busy_timeout to 200 ms on the backend's own single
connection, reads it back and fails unless it took, then holds the same
real writer. The refusal still travels SQLITE_BUSY -> ErrBusy -> wire
busy through the real store and dispatch. Production keeps its 10 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmd/codeaf runs unsharded (sharding it failed in #1734). Its serial suite
measured 1065 s (17.8 min) on the 4-vCPU runner on 2026-11-03, 1053 s of
it in tests that predate the workstream work, so the leg hit the
Makefile's 15-minute timeout and every test after the cut went unchecked.
The base branch's own runs time out the same way.

Only the codeaf leg passes --timeout 25m; tui3, session and rest keep the
Makefile's 15m. A timeout is still red. docs/rules/ci.md records the
exception and that it goes once the suite is back under 15 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@AbirAbbas
AbirAbbas merged commit c54b647 into workstreams/main Nov 3, 2026
2 of 6 checks passed
@AbirAbbas
AbirAbbas deleted the workstreams/eng-ci branch November 3, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant