Skip to content

Flaky CI: TestServer_should_Write*HookURL*/MCPURL* tests timeout under go test -race load #271

Description

@tstapler

Migrated from TylerStaplerAtFanatics/stapler-squad#192
Originally created by @tstapler on 2026-07-25T23:41:20Z


Summary

server/server_integration_test.go's hook-URL/MCP-URL integration tests intermittently fail in CI with 30-second timeouts, unrelated to any specific code change. Observed 3 consecutive failures while shipping PR #190 (a completely unrelated backlog-feature change to session/backlog_*.go and server/services/backlog_service.go — this test file was never touched).

Failing tests

  • TestServer_should_WriteUnchangedHookURL_When_StartedOnExplicitPort
  • TestServer_should_WriteRealPortIntoSessionHooksAndMCPURL_When_StartedWithPortZeroThenSessionCreated

Failure mode (both): waitForPermissionRequestHookCommand/waitForLiveInstance time out after 30s waiting for CreateSession's asynchronous hook-injection (a real tmux session spin-up + .claude/settings.local.json write) to complete.

Root cause (diagnosed while shipping PR #190)

  • CI runs these tests via go test -race ./... (.github/workflows/build.yml:149), and the race detector adds substantial (2-10x) overhead.
  • The test file's own comment already documents this exact risk: "Timeouts are generous (vs. the sub-second local case) because CI runs this under go test -race alongside the rest of the server/session/config suite, which can push tmux session startup + hook injection close to a tighter deadline under load."
  • Locally, make test uses plain go test -short (no -race) and these tests pass reliably — the race detector + full-suite concurrent resource contention on the shared GitHub Actions runner is what's pushing the already-"generous" 30s timeout over the edge, at least intermittently under current CI load.

Evidence this is pre-existing / environmental, not a regression

  • Neither failing test, nor server/server_integration_test.go, nor the hook-injection code path (server/services/approval_handler.go) was touched by PR fix(test): disable gc.auto in gogitstore fixtures to stop concurrent-repack pack-count flakes #190.
  • The PR's own full local test suite (including -race-free make test) passed cleanly multiple times.
  • 3 consecutive CI reruns of the same job all failed on this same file, with escalating failure count (1 test failed on run 2, 2 tests failed on run 3) — consistent with a marginal-timeout issue that gets worse under variable CI load, not a deterministic bug in the PR's own diff.

Suggested next steps (not yet investigated in depth)

  • Consider whether -race is actually needed for this specific integration-test package, or whether it could run without -race (or in isolation from the rest of ./...) to remove the CPU/memory contention that's pushing it over the timeout.
  • Consider whether the underlying hook-injection pipeline (tmux spin-up → .claude/settings.local.json write) has room to be made faster/more deterministic rather than just widening the timeout further.
  • Check GitHub Actions runner sizing/concurrency settings — if this job runs alongside many other jobs on a shared/smaller runner, that's a likely contributor.

🤖 Filed via Claude Code while investigating a CI flake blocking PR #190.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions