perf: ship the shared agent-build session default-on with a kill switch - #1395
Merged
philmerrell merged 2 commits intoSep 30, 2026
Merged
Conversation
…kill switch The dev A/B (docs/specs/turn-latency-preamble.md PR-6, 2026-09-30, 15 cold first turns per arm) met the rule written before the data: `agent_build` median 869ms -> 372ms, first token at the client 4211ms -> 3564ms, decision metric ranges 601-813 vs 171-289 with no overlap and no stage regressed. `AGENT_BUILD_EXPERIMENT` and its arms are replaced by `AGENT_BUILD_SHARED_SESSION_ENABLED` (unset means on; only `false` disables, and then warm-up skips the shared step too). The `session_id` plumbing that existed only to pick an arm goes from `ModelConfig.to_bedrock_config`, `AgentFactory` and `ChatAgent`; `turn_prelude` stamps `sharedSession` instead of `buildArm`. The experiment script is retired with the flag it needed. No CDK entry: the Runtime is at 48 of 50 environment variables and unset means on. Nothing reaches the prompt, `toolConfig` or restored history; nothing runs on the request path that did not before. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#1377 shipped one process-wide boto3 session for the first-turn agent build, off by default behind a two-arm dev A/B with the decision rule written before the data. The A/B ran on dev on 2026-09-30 (15 cold first turns per arm, interleaved, every turn its own Runtime process, all 30 ok, arm labels verified server-side, sessions soft-deleted afterwards) and met the rule decisively. This PR ships it default-on.
Readout (median / p75, ms):
agent_build.session_mgr_clientsagent_build.session_mgr(network)agent_build.strands_agentagent_build.finalize(restore)agent_build(group)Decision metric (
session_mgr_clients+session_mgr): control ranged 601 to 813, the shared arm 171 to 289. The distributions do not overlap, and no stage regressed (session_mgr's +10 is inside its own spread). Warm-up did its part: Runtime logs show every fresh container building the three shared clients in 15 to 53 ms and reading the strategy ids at warm-up (median 193 ms, one outlier at 3.2 s, all ok), on the daemon thread and never on a request.What changed
AGENT_BUILD_SHARED_SESSION_ENABLED(agent_build_shared_session_enabled): unset or empty means on, only the literalfalsedisables, house style. Off, every SDK builds its own session exactly as before and warm-up skips the shared step, including the one strategy-id connection.AGENT_BUILD_EXPERIMENT,AGENT_BUILD_ARMS,agent_build_experiment_armandmemory_shared_clients_enabledare gone; a Runtime still carrying the old variable reads as nothing (test pinned).AGENT_STATUS_ENABLED,MID_TURN_STEERING_ENABLED, ...) rather than taking a slot. Added to the docs-site runtime-only table.session_idplumbing that only picked an arm is removed fromModelConfig.to_bedrock_config,AgentFactory._create_bedrock_model/create_agentandChatAgent._discover_strategy_idskeepsshared_session=as part of its cache key so a switched-off process still makes its own call.turn_preludestampssharedSession(a property, never a dimension) in place ofbuildArm;processBuildsstays.abmode it needed. The readout lives indocs/specs/turn-latency-preamble.mdPR-6, now marked shipped, which also keeps the record of the withdrawn off-loop arm.Nothing about the mechanism changes: the session, its
client()sharing, the warm-up placement, the SDKMemoryClientrebinding and theBedrockModelboto_sessionhand-off are exactly what the A/B measured.Cost / TTFT
toolConfigor restored history.GetMemoryread, unchanged.After merge
AGENT_BUILD_EXPERIMENThas already been removed from the dev Runtime out of band (back to 48 variables, image unchanged), so dev runs the plain build until this deploys and the default-on session after.turn-path-ttft.md§5 P2.Tests
PYTHONPATHpointed at the branch).falseis on;falsein any case is off; the retired experiment variable is ignored.MemoryClientthe shared session by default and nothing when off;BedrockModelreceivesboto_sessionand noregion_nameby default, neither when off, and the shared session resolves the same region as a fresh one;reset_cached_clientsdrops the session.🤖 Generated with Claude Code