perf: push the first-turn session title as soon as it exists

[philmerrell]

Why

The conversation title is generated by a Nova Micro side-channel call that runs alongside the agent stream. The call itself is quick, but several delays sat between Nova answering and the sidebar/top-nav showing the name.

What changed

Fix 1: poll the title every 100ms, not only between agent events. The one-shot _session_title_sse check only ran when the agent stream yielded. A title that finished during the model's time-to-first-token or a long tool call waited until the next event. The check now also runs in the coordinator's existing live status merge (_merge_agent_status, 100ms), passed per turn as poll_side_frame the same way turn_lease is. That merge runs every step in one context, so the OTel detach fix is unaffected. The between-event check stays, because it is the only route while AGENT_STATUS_LIVE_DRAIN_ENABLED=false.

Fix 2: stop waiting on the DynamoDB write. generate_conversation_title awaited update_session_title before returning, and the stream only emits a finished task. The write now runs as a background task, held by a strong reference so it can't be garbage-collected.

Fix 3: write the title in one round trip. update_session_title did a GSI lookup followed by update_item. Every row is now born with the static S#{session_id} SK, so the write is one keyed update_item guarded by attribute_exists(PK). It falls back to the GSI only for a legacy timestamped SK, and the guard keeps it from creating a ghost row when the row is missing.

Fix 4: build the Bedrock client once. A bedrock-runtime client per title cost ~250ms on first use in a process (measured locally), and that time ran on the event loop. It also meant a new TLS handshake every time. The client is now cached and built lazily under a lock inside the worker thread.

Fix 5 (SPA): faster fallback. When the stream outruns title generation, the post-close metadata read sees the placeholder. It used to retry once after 1.5s; it now retries up to 5 times at 400ms, the same overall window, and stops early once the title arrived another way.

Deliberately not done

• Holding the stream open after done to wait for a late title. The SPA clears loading and the Stop button on close, not on done, so a hold would show "responding" for longer on every fast first turn. Fix 5 covers this case instead.
• Starting the title task earlier. The preamble ahead of it is about 22–37ms, so there is little to gain.

Cost / TTFT

• Nothing is added to the prompt or the cacheable prefix, and the model call is unchanged.
• No work is added before the first token. Fix 4 takes up to ~250ms of client build off the event loop, which can land on a cold container's first-turn preamble.

Tests

• Backend full suite: 10939 passed, 3 skipped.
• New tests:
  • The title reaches the client during agent-stream silence.
  • The poll still runs without a status hook.
  • A failing poll never breaks a turn.
  • The title returns before its write lands.
  • The client is reused across titles.
  • Static-SK write, legacy-SK fallback, and no ghost row when the row is missing.
• SPA chat-http.service.spec.ts: 19 passed, 3 new covering retry-until-found, bounded give-up, and stop-when-applied.

Verify on dev after merge

Timestamp the session_title frame against done on:

• a short first turn (should mostly arrive via SSE now, or within ~400ms of close), and
• a tool-heavy first turn (should arrive mid-tool, not at the next event).

🤖 Generated with Claude Code