perf: A/B the first-turn agent build (shared boto3 session built at warm-up), default off

[philmerrell]

Why

After #1374, dev showed the first-turn title waiting on the agent build, not on anything in the title path. "✅ Generated title" is logged in the same millisecond as turn_prelude on every first turn. The build takes 1.0–1.2s, and agent_build.session_mgr alone is 616–738ms.

Two findings shaped this PR:

• The SDK builds its Memory clients twice, every time. AgentCoreMemorySessionManager.__init__ builds a MemoryClient (a fresh boto3.Session plus two clients), then a second fresh session plus two clients that replace the first pair. A fresh session re-parses every service model it touches, so that is about 360ms of client construction per session manager on a laptop, and turn-latency-preamble.md PR-3 showed client construction is ~30x slower on the container. Two more fresh sessions sit next to it on the same first turn: _discover_strategy_ids builds its own MemoryClient, and Strands' BedrockModel builds its own boto3.Session.
• Every conversation runs in its own Runtime process. service.instance.id differs per session in the runtime logs, so every first turn is a cold build. Laptop numbers can't settle whether the change helps there, so it ships off behind a dev A/B.

This PR was narrowed on 2026-09-30 against the assessment in docs/specs/turn-path-ttft.md §4 (PR #1388): the instrumentation and the shared_clients arm stay, the off-loop arm is withdrawn, and the shared session is built at warm-up instead of by the first build.

What changed

One per-session experiment flag, AGENT_BUILD_EXPERIMENT (agent_build_experiment_arm). Unset or empty means control for every session, so no environment changes behavior on merge.

Arm	Change
control	today's build
shared_clients	one process-wide boto3 session (apis.shared.aws_clients.shared_boto_session) whose client() returns one client per configuration (pool size 50, SDK user agents kept), handed to everything on the build that would otherwise construct a fresh session: the SDK session manager (boto_session=), _discover_strategy_ids (MemoryClient(boto3_session=)), and Strands' BedrockModel (boto_session=, and then no region_name, which Strands rejects beside a session). The SDK constructs its throwaway MemoryClient with no session, so the SDK module's MemoryClient name is rebound to a subclass that takes the shared session only while the factory builds a shared-arm manager (a contextvar). A test fails if an SDK upgrade stops going through that seam; the rebinding is a workaround for a one-line upstream bug and should have an end date.

Built at warm-up, not by the first turn. The session, its bedrock-agentcore, bedrock-agentcore-control and bedrock-runtime clients, and the strategy ids are all built in warmup.py on the startup daemon thread, so the arm's first turn — the only turn the experiment is about — finds the service models parsed and the ids cached. The strategy-id discovery is a control-plane read of static configuration, and it opens the one connection warm-up otherwise avoids. That is accepted per turn-path-ttft.md §5 P2: the alternative is paying the read on the turn this exists to shorten, the call is idempotent so botocore retries a connection error, and on Runtime V2 the restore's first turn is the thing to watch for a pool holding a socket that did not survive the snapshot. The discovery cache is keyed by arm on purpose, so warm-up primes the shared entry and the control arm's first turn still does exactly what it did before.

Withdrawn: the off-loop arm. As opened, a third arm ran create_agent under asyncio.to_thread so the route could emit a first-turn title mid-build, and it needed hardening the frozen loop used to provide for free: process-wide build serialization, a lock on ExternalMCPIntegration's maps and singleton, and a consumer pin on every external MCP client handed to a build. All of that is gone, with its tests, and the route's title race reverts to the plain awaited build. A thread does not make a synchronous build faster (CPU-bound parts contend on the GIL, network-bound parts take the same time), so the arm could not reduce time to first token and its A/B would have read as noise; the concurrency it would have bought is reachable inside the constructor without any of the hardening (§5 P3a: session_mgr and tools on two threads of one executor). A title before prepared is not worth shipping locks for.

Measurement, unchanged:

• turn_prelude gains buildArm and processBuilds (1 on a first turn), which are EMF properties, never dimensions.
• Two new sub-stages. agent_build.session_mgr_clients (client setup) now precedes agent_build.session_mgr, which becomes network only. agent_build.strands_agent (Strands' Agent construction, including MCP load_tools) now precedes agent_build.finalize, which becomes the restore only.
• scripts/experiment_agent_build_arms.py drives interleaved first turns per arm (rotating order, session ids stratified into the two arms) and joins client-side frame timings to each turn_prelude.

Considered and rejected

• Skipping read_session's two list_events for a known-new session. Strands skips read_agent when a session looks new, so a wrong "new" (turn 1's metadata pre-create failed, turn 2 re-creates it) would silently drop conversation history.
• A CDK entry for the flag. The dev Runtime is at 48 of its 50 environment variables, and a temporary experiment shouldn't take a slot. It is set out of band: backend.yml preserves it and platform.yml resets it.
• Warming the shared session only when the arm is on. Warm-up cannot know a session's arm (ab hashes the session id), and building clients on an unused session costs nothing on the request path, so it is unconditional.

Cost / TTFT

• Nothing reaches the prompt, toolConfig or restored history. The session, the clients and the strategy ids are transport; the prompt bytes on both arms are identical to develop.
• What the control arm pays on its request path: two extra mark_stage calls (session_mgr_clients in read_session, strands_agent in initialize) and the two property stamps on turn_prelude. Unchanged from the PR as opened. Control never touches the shared session, and its own strategy-id discovery on the first turn is the same call it made before (separate cache entry).
• What every deployment pays at startup, on the warm-up daemon thread and never on a request: three client constructions on the shared session and one control-plane GetMemory read for the strategy ids.
• The arm's TTFT effect is exactly what the A/B measures. Expected from the laptop arithmetic in turn-path-ttft.md §5 P2: session_mgr_clients + session_mgr drop by the parses they no longer do and the discovery call, finalize by one parse; the container ratio is unknown until measured.

Plan after merge

1. Set AGENT_BUILD_EXPERIMENT=ab on the dev Runtime out of band (update-agent-runtime; backend.yml deploys preserve it, a platform.yml deploy resets it).
2. Run scripts/experiment_agent_build_arms.py --per-arm 15 --cleanup against dev.
3. Apply the decision rule written into turn-latency-preamble.md PR-6 before the data: ship shared_clients as default-on with an =false kill switch if its session_mgr_clients + session_mgr median beats control's by more than the run-to-run spread and no stage regresses. Otherwise remove the arm and keep the instrumentation. Either way, watch the first Runtime V2 restore's logs for the warm-up connection.

Tests

• Backend full suite from the branch: 10998 passed, 3 skipped (run from the worktree with PYTHONPATH pointed at the branch).
• Shared session: warm-up builds each client once on the shared session and discovers the strategy ids once with shared_session=True, skips both without a region, and survives a failing client; the factory hands the SDK constructor and MemoryClient the shared session on the arm and nothing off it; warm_strategy_ids and the arm's first turn hit the same cache entry; BedrockModel receives boto_session and no region_name on the arm (two models share one bedrock-runtime client, and the arm resolves the same region as a fresh session), and neither off it; reset_cached_clients drops the shared session; a None keyword (Strands' endpoint_url=None) does not defeat the sharing while a real override is never shared.
• Shared clients, as before: real TurnBasedSessionManager construction with only SDK network stubbed. Later managers build zero clients, concurrent first builds share one client, control keeps per-manager clients, and the MemoryClient seam is inert elsewhere.
• Arm assignment: default control, forced arms, the withdrawn arm's name means control, stable and roughly even two-way ab hashing.

🤖 Generated with Claude Code