Skip to content

Page giant turns' work and collapse the active-turn frontier#738

Open
SawyerHood wants to merge 3 commits into
mainfrom
bb/fresh-problem-solution-thr_7rrht6txvp
Open

Page giant turns' work and collapse the active-turn frontier#738
SawyerHood wants to merge 3 commits into
mainfrom
bb/fresh-problem-solution-thr_7rrht6txvp

Conversation

@SawyerHood

Copy link
Copy Markdown
Collaborator

Problem

Very long threads — especially a single agent turn running for hours — became progressively expensive to load and update. The worst sample turn (thr_n8ptu7bf8f) is 335 minutes / 7,025 events / 27 MB:

  • Active turns never collapsed: every event rendered as a flat row, the server re-projected the whole turn per delta tick, and the un-virtualized list grew without bound → browser freezes.
  • Expanding a completed giant turn fetched the entire turn's detail un-truncated in one response (~14 MB projected for the largest segment).
  • The conversation outline re-read every event in the thread on each update.

#711 tried to bound responses by raw row/byte budgets and was reverted in #722 because the cut could fall mid-turn behind a generic "Load older" control. This change keeps that constraint front and center: top-level segment pagination is untouched — a visible response always starts with its initiating user message — and all new bounding happens inside a turn's "Worked for…" section.

What changed

Paged turn work ("Show earlier work")

  • GET /threads/:id/timeline/turn-summary-details gains workItemLimit (newest-N page + earlierCursor) and afterSeq (exact catch-up range). Windows resolve from the indexed item/completed sequence; items straddling a window edge still project fully from their completion payload; paged rows get the timeline's 32 KB output truncation. Omitting both params keeps today's full, un-truncated detail (small turns, CLI).
  • LazyTurnRowBody pages large turns (partial rows, or completed turns above 60 work items) through immutable per-window segment queries with a nested "Show earlier work" control. Expansion shows newest work first; revealed windows are keyed by summary-row id so they survive collapse/expand, live-turn growth, and the completion transition; prepends anchor scroll exactly like "Load older messages".

Active-turn frontier collapse ("Worked so far")

  • Once a running turn accumulates ~48 finished work items, its settled prefix collapses into a partial turn row (partial: true, title "Worked so far (2h 5m)"). Running commands, waiting approvals, unanswered questions, and the most recent ~24–47 finished items stay flat below it.
  • The frontier derives from indexed completion counts only and moves in 24-item chunks (hysteresis), so row identity churns at most once per chunk and delta payloads stay small. On completion the row converges to the normal "Worked for…" summary at the same row id; steer boundaries split partial summaries exactly like completed ones.

Targeted conversation outline

  • Re-lands Bound large thread timeline responses #711's outline event selection, plus turn-lifecycle/error events so completed-turn grouping matches the timeline and interim assistant messages don't leak into the minimap.

Measured (copy of the 39 MB sample thread)

Scenario Before After
Latest page while the giant turn is active ~7,000 rows / 27 MB 69 rows / ~420 KB
Expanding the 980-item turn segment 13.9 MB, un-truncated, one shot ~210 KB / ~15 ms per 40-item window
Full page walk of that segment 24 windows, 378 ms total, full coverage, no dupes
Conversation outline full-thread read per update 49 ms / 108 KB
HTTP paged vs full detail (485-item turn) 2.3 MB / 81 ms 94 KB / 15 ms

Validation

  • New unit tests: partial collapse (frontier partition, settled-work rules, id stability across completion), work-page builder (children-only completed windows, straddled items, pending-work drop/finalize), server paging (cursor walk, catch-up ranges, degraded stale windows, chunk hysteresis), outline/timeline row-id equivalence.
  • pnpm exec turbo run typecheck across the repo; full test suites for @bb/server, @bb/thread-view, @bb/db, @bb/app, @bb/server-contract, @bb/cli, @bb/sdk.
  • Real-data verification against a copy of the live DB (numbers above), plus end-to-end UI QA in the dev app: expanded the 1h38m turn, paged earlier work, and confirmed the scroll anchor holds to the pixel.

Follow-up (documented in plans/long-thread-timeline-pagination.md)

  • Bounded event selection: a cache-miss timeline build still reads/parses every event in the window (~600 ms on the sample thread, once per appended event batch). The collapse frontier is already computed from indexed queries, so the selection can skip collapsed-prefix payloads next.
  • Optional per-item "show full output" fetch for truncated outputs inside paged windows.
  • List virtualization if profiling still warrants it after this lands.

🤖 Generated with Claude Code

@SawyerHood

Copy link
Copy Markdown
Collaborator Author

Ran one multi-angle review round over the diff (8 finder angles + verification) and pushed the fixes in ff042a3. Highlights of what it caught:

  • Cache determinism: the collapse-frontier queries weren't bounded by the build's maxSeq, so a response cached at revision N could bake in a frontier derived from newer events; the conversation outline also built without frontiers, breaking the minimap's row-id contract for running giant turns (jump-to-message would page the whole thread hunting for a collapsed row). Both now share the same bounded frontier.
  • Index reality check: EXPLAIN QUERY PLAN showed the completion-count/nth/descending queries doing per-row table seeks (or whole-thread scans for the OFFSET variant). Added a partial covering index (thread_id, turn_id, sequence, item_kind) WHERE type='item/completed' via drizzle-kit.
  • UX edges: "Show earlier work" no longer flickers out while its page loads; groupable steer messages are no longer dropped from paged windows; an empty settled prefix no longer emits a bogus "Worked so far" row; expansions refetch once on turn completion so interrupted work appears finalized instead of vanishing.
  • Contract compliance: the SDK now forwards workItemLimit/afterSeq (they were accepted-but-ignored, which AGENTS.md forbids). A dedicated bb CLI surface for paging turn work is noted as follow-up.

Full suites green after the fixes (the one failing app test is the pre-existing flaky promptbox slash-token test; it passes in isolation).

@SawyerHood
SawyerHood force-pushed the bb/fresh-problem-solution-thr_7rrht6txvp branch from 732405c to 2be3bf5 Compare July 16, 2026 13:49
SawyerHood and others added 3 commits July 16, 2026 15:29
Very long single turns (observed: 335 min / 7,025 events / 27 MB) froze the
browser: active turns rendered every event as a flat row, and expanding a
completed turn's "Worked for…" section fetched the entire un-truncated turn in
one response. Top-level segment pagination stays untouched — every visible
response keeps its initiating user message — and all new bounding happens
inside a turn's work section, the lesson from the #711 revert.

Server:
- turn-summary-details gains work-item paging: `workItemLimit` serves the
  newest N work items with an `earlierCursor` for progressively revealing
  older windows; `afterSeq` serves an exact catch-up range. Windows resolve
  from the indexed item/completed sequence, items straddling a window edge
  project fully from their completion payload, and paged rows get the same
  32 KB output truncation as the timeline (full detail stays un-truncated).
- Active turns past ~48 finished work items collapse their settled prefix
  into a partial "Worked so far" turn row (`partial: true`); recent work,
  running commands, approvals, and questions stay flat below it. The collapse
  frontier is derived from indexed completion counts and moves in 24-item
  chunks, so row identity churns at most once per chunk and the row converges
  to the normal completed "Worked for…" row at the same id.
- Conversation outline selects only message-producing and turn-lifecycle
  events instead of reading every event (re-land of the #711 outline query,
  plus lifecycle events so completed-turn grouping matches the timeline).

App:
- LazyTurnRowBody pages large turns (partial rows and >60-item completed
  turns) through immutable per-window segment queries with a nested
  "Show earlier work" control; revealed work is keyed by summary-row id so it
  survives collapse/expand, live growth, and the completion transition, and
  prepending anchors scroll like "Load older messages".

Verified against a copy of the 39 MB sample thread: the active giant turn's
latest page drops from ~7,000 rows / 27 MB to 69 rows / ~420 KB; paged
expansion serves ~210 KB windows in ~15 ms with full coverage and no
duplicates; the outline builds in ~50 ms. UI exercised end-to-end in the dev
app (expand, page, scroll stability).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Correctness:
- Bound the collapse-frontier queries (active turn lookup, completion count,
  nth completion) by the build's maxSeq so a response cached per
  (thread, maxSeq) never bakes in turn state from events appended after that
  revision, and pass the same frontiers into the conversation outline so its
  items always correspond to top-level timeline rows (the minimap's
  jump-to-message contract).
- Guard the partial collapse against an empty settled prefix: previously the
  single-group fast path fell back to turn.summaryCount and whole-turn bounds,
  emitting a bogus "Worked so far" row above the same rows rendered flat.
- Keep agent/system-initiated steer messages in paged work windows: they fold
  into summary children for completed turns, so dropping every user row made
  them unreachable in paged expansions.
- Treat NULL-item_kind legacy completions like the outline query does (fall
  back to the payload's item type) so reasoning completions on old threads
  don't inflate page windows or the collapse frontier.
- Keep the "Show earlier work" control mounted in a loading state while the
  next-older window fetches instead of flickering out mid-load.
- Re-key segment windows (generation bump) when a turn finishes so work that
  never completed reappears finalized as interrupted instead of silently
  vanishing from an already-open expansion.
- Include `partial` in the turn-row render signature; forward
  workItemLimit/afterSeq through the SDK so the paging contract isn't
  accepted-but-ignored there.

Efficiency:
- Add a partial covering index (thread_id, turn_id, sequence, item_kind)
  WHERE type='item/completed' — EXPLAIN QUERY PLAN showed the frontier and
  paging queries doing per-row table seeks (or whole-thread scans for the
  OFFSET variant) against the existing indexes. Generated via drizzle-kit.
- Gate the outline's json_extract fallback to NULL-item_kind rows so typed
  tool/command events never parse their payloads; include plan items, which
  also project to assistant text.

Cleanup:
- Extract the duplicated turn-window event selection pipeline into a shared
  helper used by both the full-detail and paged paths; fold the pending-row
  predicates into one type guard; drop a redundant client-side re-sort and
  simplify the segment row merge to Map semantics.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@SawyerHood
SawyerHood force-pushed the bb/fresh-problem-solution-thr_7rrht6txvp branch from 2be3bf5 to ee850f1 Compare July 16, 2026 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant