Skip to content

test(runtime): isolate client fixtures on private HTTP transports - #4526

Merged
aheritier merged 1 commit into
mainfrom
fix/4516-runtime-test-private-transport
Oct 6, 2026
Merged

aheritier merged 1 commit into
mainfrom
fix/4516-runtime-test-private-transport

Conversation

@aheritier

Copy link
Copy Markdown
Collaborator

🤖 Automated implementer agent — this comment was posted by the implementer bot from Docker Agentic Platform, not by a human developer

Fixes #4516

Summary

Give all 16 runtime HTTP-client test fixtures private cloned transports with test cleanup. Add a network-free regression checking distinct/private transports, the default timeout, caller WithTimeout precedence, and streaming-client transport reuse. Five test files only; no production changes, retries, sleeps, or weakened assertions. The incomplete-stream cases still require a final ErrorEvent and exactly one POST.

The CI error occurs during the POST, before stream validation: net/http: HTTP/1.x transport connection broken: http: CloseIdleConnections called. Closing any httptest.Server closes the global default pool. Go pools a bodyless response before delivering it, so parallel cleanup can close the connection in that window; the written POST is not replayable. Isolating only the observed subtest would leave equivalent fixtures exposed. This follows the MCP fixture precedent from #3704.

Validation on this branch

Go 1.27.0 linux/amd64; Task 3.53.1; golangci-lint 2.13.2; CGO enabled after installing the missing compiler.

Command Result
go build ./... / go vet ./pkg/runtime PASS
task format / git diff --check PASS; unrelated Windows-test import reorder reverted
task build (also forced after compiler install) PASS
task test PASS after compiler install; initial failures disclosed below
task lint (also forced after compiler install) PASS, including project cops and module tidiness
go test ./pkg/runtime -run '^TestNewTestClientUsesPrivateTransport$' -count=50 PASS, 0.024s
go test -race -shuffle=on -count=100 ./pkg/runtime -run 'TestNewTestClient|TestClient|TestEvaluatorUsagePersistenceAndSSE|TestSessionRecoveredEventContract|TestRemoteRuntime_Background' PASS, 66.672s
go test -race -shuffle=on -count=500 ./pkg/runtime -run '^TestClient_RunAgentIncompleteStreamIsAnError$' PASS, 2.942s
go test -race -shuffle=1791190804724097755 -count=3 ./pkg/runtime PASS, 34.187s
go test -race -shuffle=on -count=3 ./pkg/runtime PASS, 34.525s

Evidence and limits

  • The original target was not naturally reproduced locally: earlier unpatched historical runs had 0/6 failures; main had 0/3. These are not CI frequency estimates.
  • Earlier standalone amplified concurrency observed the exact error with the shared pool (13/19,136 POSTs), versus 0/73,187 with private pools. A deliberately forced scheduling overlay through the actual runtime client observed shared 30/30 versus private 0/30, preserving ErrorEvent and one POST. These diagnostic harnesses are not committed tests and are not natural reproduction of the original target.
  • Initial task test ran with CGO disabled because the compiler was absent: treesitter reported that CGO is required. The same run also hit untouched TestFileCache_dedupSkipsRedundantWrite (mtime assertion) and evaluation missing_local_image (missing inspect-args). After installing build-essential, the full rerun passed. No unpatched control was run for those two transient failures, so their cause is not established here.
  • Earlier independent broad runtime stress hit a different, unchanged attempts >= 2 assertion in TestRemoteRuntime_BackgroundSubscriptionWaitsForEventLogAndDeliversElicitationOnce; same-seed 10 repeats then passed. No unpatched control was obtained. This branch's fresh runs did not hit it; it is not fixed or confidently attributed in this PR.
  • Local Ubuntu 26.04 differs from CI Ubuntu 24.04.5. Full-repository race validation was not run; package-level race coverage is listed above. Finite passes do not prove the absence of future flakes.

Parallel httptest.Server cleanup closes the global default pool, which can interrupt delivery of a bodyless POST response. Give all 16 runtime client fixtures private cloned transports and test cleanup, preserving caller options and incomplete-stream assertions.

Fixes #4516

Signed-off-by: Arnaud Héritier <aheritier@users.noreply.github.com>
@aheritier
aheritier marked this pull request as ready for review October 6, 2026 09:42
@aheritier
aheritier requested a review from a team as a code owner October 6, 2026 09:42
Copilot AI balanced review requested due to automatic review settings October 6, 2026 09:42

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The test-only changes consistently isolate transports while preserving existing timeout and streaming behavior.

Review effort: Balanced
Findings: None

What changed in this PR

Isolates runtime HTTP test fixtures from the global connection pool, preventing parallel httptest.Server cleanup from breaking in-flight requests.

Changes:

  • Adds private cloned transports and cleanup helpers.
  • Migrates all 16 runtime client fixtures.
  • Adds regression coverage for transport isolation, timeouts, and streaming reuse.
File Description
pkg/​runtime/​client_test.go Adds helpers, regression coverage, and migrates client fixtures.
pkg/​runtime/​remote_runtime_test.go Uses isolated clients in background-runtime tests.
pkg/​runtime/​recovery_event_test.go Isolates the recovery-event client fixture.
pkg/​runtime/​plan_events_test.go Isolates the plan-event client fixture.
pkg/​runtime/​evaluator_usage_test.go Isolates the evaluator SSE client fixture.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@aheritier aheritier added area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/testing Test infrastructure, CI/CD, test runners, evaluation kind/test Test-only changes labels Oct 6, 2026
@aheritier
aheritier added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit c649e40 Oct 6, 2026
17 checks passed
@aheritier
aheritier deleted the fix/4516-runtime-test-private-transport branch October 6, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/testing Test infrastructure, CI/CD, test runners, evaluation kind/test Test-only changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Flaky test] TestClient_RunAgentIncompleteStreamIsAnError

3 participants