Repository navigation
plan: A5 viewport — phase 2 — usable tail-first F5 and real-Wasm #924 close gate #960
Description
Plan header
| Field | Value |
|---|---|
| Status | HANDOFF-READY — implement after #959 lands |
| Date | 2026-09-07 |
| Type | phase 2 — one issue / one PR |
| Parent / source | #958, #553 |
| Depends on | #959, bounded best-effort read/transport |
| Closes bug | #924 only when the actual production-controller real-Wasm int gate is green and this PR ships |
| Baseline / branch | main b31ca512e7ddee162fa8313cc52f549822e2d23a; branch feat/a5-tail-cold-attach after dependency |
| Layers | Host controller/parser/apply; backend narrow metadata sync; existing Wasm paint exports |
| Reusability / ops | Scoped injected services; no admin migrate/backfill/env cutover; existing Git deploy/int artifact |
| Living docs | Session/stream/limits/feature-divide, AGENTS, README, SECURITY |
Review notes — pragmatic revision (2026-09-07)
Operator explicitly chose best effort, sensible bounded cost, incremental usability over exact history. This replaces this issue's previous certified-checkpoint/whole-system-CAS dependency. Parent and #959 now define recent sampled display and tail skipping. Do not revive the removed archive/proof/transaction work.
| Finding | Resolution |
|---|---|
| Legacy restoration cannot be perfectly aligned | No exact merge. Adopt sample/head or retain cached paint, mark partial, begin fresh continuation rows after handoff. |
| Cold JSON→GET can replay backlog | Consume #959's one negotiated snapshot-first SSE response, not a second cold raw GET. |
| Partial snapshots can overwrite canonical history | Separate display from durable snapshots; all new production host persistence uses a narrow metadata-only API, no transcript body upload. |
| Queue still needs reload continuity | Persist the existing host-known desired queue in meta.queueMirror, explicit [] clear. No transaction journal/exactly-once claim. |
| Whole Redis CAS rewrite was scope inflation | No source-generation/CAS/worker B7/B8 refactor. Use a small field-patch store primitive that changes only allowed metadata fields in the current stored envelope; see below. |
Scores: correctness4, performance5, architecture4, testing5, security4, reusability5, parent adherence5, layers5, docs5, cap governance5, harness UX/layout4. Administrative cloud ops N/A. Verdict HANDOFF-READY subject to #959 merge, not blocked on a design question. No code/tests run during review.
Goals / scope
- G1: real running F5/new tab/login/switch lands on recent useful rows, no prefix rewind or historical chronological replay.
- G2: no sampled historical reasoning paint; post-handoff reasoning is live, even if continuing an old segment.
- G3: Busy/Stop restored promptly for running/cancelling; explicit Stop works during recovery; restore never starts a second turn.
- G4: partial displays never upload as full transcript or become model promptHistory; menus/host-known queue remain usable.
In: production shared controller, #959 parser/negotiation, attach/apply lifecycle, metadata PATCH + queueMirror, tests/docs.
Out: native exports, scrollbar/cache format, exact dedup/terminal commit proof, global CAS, changing worker persistence/inference/queue semantics, SDK changes.
Baseline / ownership
Read on stated main: turnAttach.ts cold decisions hardcode zero; harnessChat.ts calls prefix wipe; turnApply.ts increments callback cursor and assumes last visible live row; HarnessHost cold adopt resets heap then kicks attach; session repo whole-snapshot PUT serves metadata updates too. Existing queue helpers maintain host-known mirror/FIFO; body sanitization drops empty queue whereas the new metadata mirror needs an explicit empty tombstone. Existing Redis store has get/set, no narrow field-patch method; memory adapter must match new method.
| Concern | Files |
|---|---|
| Pure production controller | new lib/viewportBootstrap.ts, lib/sessionBoot.ts, app/harness/HarnessHost.tsx |
| Negotiated clients/parser | lib/turnApi.ts, #959 lib/viewportStreamProtocol.ts, lib/sessionRepository.ts |
| Apply/lifecycle | lib/turnAttach.ts, lib/turnApply.ts, lib/harnessChat.ts, existing queue/detach helpers |
| Metadata API | app/api/sessions/[id]/envelope/route.ts PATCH, new lib/sessions/patchSessionMeta.ts |
| Store primitive/carrier | lib/sessions/{sessionStore,redisSessionStore,memorySessionStore}.ts, lib/sessionCloudCaps.ts, lib/turnQueue.ts |
| Int | int/{driver,stores,f5-attach.int.test,quota.int.test}.ts, real int/loadBridge.ts |
1. Production cold controller
Extract a framework-independent controller used by HarnessHost and int tests; do not copy its decisions into test driver. States: cached → recovering → attached, or terminal/error/detached. Inputs: session epoch/id, bridge, scoped clients, abort, local writers.
- Cached/local paint immediately; avoid full repo.get/reconstruction before cold attach. Read small envelope for liveness, then
viewportVersion=1&hydrate=tailfor a live run. Envelope/snapshotturnStreamCursoris only a hint; never trust it as a complete display prefix. - On
viewport_state, set live run id and Busy/Stop immediately. No blank/prompt-only rebuild. Keep composer draft/focus and existing FIFO. Cancelling remains live until actual terminal status. - On snapshot
replace:true, paint rows once through canonical image/math scheduling and queue-preserving ring replace.replace:falsekeeps cached rows. Add one neutral incomplete-history System note when needed. Snapshot rows are already projected; no second full-history fold/coalescer. - Set heap next index to snapshot.resumeIndex; reset live assistant/tool/thinking mutable buffers and start fresh live continuation rows. Never update the last restored tool/card by name or append full
done.textover sampled text. Modest fragmented/duplicate rows accepted; no historical wipe/replay. turn_eventadvances absolute cursor only after parsing/apply; typed skipped records advance without arbitrary paint. Index unknown after decoder/transport failure → detach and preserve last applied state. Hot GET resumes from same-heap raw index; cold reopen resamples, not callback count.- Negotiate indexed durable POST for normal new turns too, so later hot cursor has one domain. New turn starts at raw origin legitimately; cold attach doesn't. Legacy injected
/api/agent/test clients stay explicit unindexed paths. - Cold Send-while-running attaches without a new POST, preserving current follow-up-note behavior. True terminal state permits the next actual prompt. No auto-retry that reruns tools after a durable start.
- Terminal stored done/error and synthetic viewport_end control lifecycle using actual run status; do not demand a successful exact terminal Blob proof. Retain displayed content, stop busy, follow existing success/error/queue-promote rules. On later idle reload the bounded latest head is best effort, never full replace upload.
- All switch/New/Clear/unmount/logout epochs ignore late data and abort readers only. Explicit Stop uses existing cancel API then releases UI appropriately; no recovery-side cancellation. Preserve FIFO/pause/no auto-drain on Stop/error.
- No automatic full-history materialization in new production path. For rare next-turn missing model seed, send raw prompt and omit partial promptHistory; server's normal modelMessages/compaction selection remains unchanged. Legacy complete-snapshot test path stays supported.
2. Partial-view persistence firewall
Use explicit historyComplete:false/view type, not length-based inference. Central guard rejects passing any view to put, trimForCloudPut, mint/upload/flatten path. Apply across success/done/error/Stop, subscription failure, queue/menu/cwd/provider/usage updates and 409 adoption. Read-only view remains separate even if it happens to contain all rows.
Replace new production host metadata calls with PATCH /api/sessions/:id/envelope:
{selectedModel?: string|null, reasoningEffort?: string|null, queueMirror?: string[]}.
Only these fields in this cut; unknown keys reject (including messages, transcriptPointer, turnRunId/turnStatus, arbitrary meta and worker-owned pointers). Persona selection remains existing New-session creation semantics, not a new editable PATCH feature. Worker-owned cwd/bind/provider/usage/notes/model/compaction state reads locally and stays server-written, not echoed by partial host. Standard session/user/tenant validation, invalid fields400, unauth401, missing/foreign404, oversized full meta413, unavailable503. Return only small sanitized acknowledgement/carriers, not arbitrary full meta/secret content.
Narrow field patch — not a history transaction
Add patchEnvelopeMetaFields(key, changes) to session store and injected Redis adapter. On Redis use one small Lua operation that reads the current envelope, changes only the validated selectedModel/reasoningEffort/queueMirror fields, advances updatedAt server-side, preserves every other field and existing TTL. Missing row returns notfound and must not create/resurrect it. Validate the same scope/record and full serialized1 MiB meta/2 MiB envelope rails inside the primitive; malformed existing data fails closed. Changes already schema-sanitized by pure helper; Lua enforces the allowlist too. Memory implementation same semantics. This avoids a stale host read copying an older transcriptPointer over the current one, without adding source-base CAS, retries, candidate journals, worker protocol changes or multi-store transactions. Normal complete legacy writers remain unchanged.
A single field-patch operation is idempotent desired-state/LWW. Existing concurrent old worker writes may later overwrite same host metadata; accepted cross-version residual, preserve local queue/preferences and reassert on later normal coalesced updates. Do not rewrite every worker, retry forever or rerun inference. On response loss retrying the same desired array does not append a duplicate command. No history bytes are read, written or replaced by PATCH.
3. Queue mirror
New reserved scalar key queueMirror stores JSON of the existing bounded queue including valid []. Add typed parser preserving the difference between absent/invalid and present empty; use existing16×5000 sanitization/multiplicity, but serialize empty array instead of dropping it. Excess invalid client input rejects rather than silently converting to a different desired queue.
- Copy forward current queueMirror on omit in ordinary new-code envelope upserts (compatibility like other reserved host-carried exceptions); explicit new valid value wins. Old deployed workers are not magically upgraded; live FIFO/local mirror protects best-effort continuity.
- Reads prefer valid present queueMirror (including
[]), else legacy Blob queue. Do not resurrect Blob queue after a new explicit empty mirror. - Existing host queue helpers compute desired mirror. Strip the promoted prompt once before POST, schedule that desired state through the coalesced metadata channel, restore head on a failed pre-start send as today. No worker-side extra strip of the new mirror, avoiding duplicate-text double removal.
- One in-flight metadata request per session channel, latest desired state wins; no tight retry loop. On a later normal update send newest full desired mirror. Queue edits inside Wasm that host cannot observe remain the existing documented limitation.
- On cold activation rearm exactly once; snapshot/live ring replacement doesn't rearm. Current pause/promote/band interaction unchanged.
Caps
No existing cap changes, no new history-retention quota. Parent/operator best-effort defaults inherited.
| Value | Limit / rationale | Location |
|---|---|---|
| Inherited recovery | 2048 frames /8 MiB sample /5000ms optional work /1000ms final probe /one head | #959 caps; this phase consumes, not retries |
| Snapshot / row / ring | 2 MiB /262144B /2048 rows | Existing and phase1 view rails; include omission note |
| New queueMirror carrier | Existing16×5000 chars JSON | Inside unchanged1 MiB meta; worst escape~480KiB, combined meta may reject without clipping unrelated fields |
| PATCH body/envelope | Existing2 MiB Function body,1 MiB meta, below4.5 MB real ceiling | Existing caps; reject overbudget, no source body transport |
| Channels | One in-flight metadata write and one attach per session epoch | Coalesce latest desired state, no new retry attempt/time budget |
| Other turn/queue/route/protocol | All unchanged | No new native export or step |
Testing — #924 close gate
No tests run in planning. All cases ordinary green assertions, real protocol-current Wasm fails closed.
| # | Case | Required evidence |
|---|---|---|
| R1 | Long running F5/new tab/switch from cached tail | Production controller+actual client parser+apply→real Wasm ring stays recent throughout state/snapshot/attach; old prefix wipe unreachable |
| R2 | Large historical reasoning sample, omitted gap; new post-H reasoning | No historical reasoning body delivered/painted; post-H thinking does paint; no exact legacy segment proof required |
| R3 | running/cancelling during delayed recovery | Busy+Stop restored before snapshot, accepted explicit cancel works; server H not naked envelope/local C; no second POST |
| R4 | Idle/completed head restore, stream drop hot resume, EOF/no progress, synthetic end, known skipped frame | No idle stream; raw cursor correct; no spin or fake history completeness |
| R5 | Missing prompt/head/corrupt history, partial tools/oversize row, done aggregate | Useful partial or cached paint; no full done duplication, no global history reconstruction |
| R6 | User Stop/error/success/menu/queue/cwd updates after partial restore | Spy on mint/upload/PUT: zero transcript uploads; only allowed metadata PATCH; canonical Blob unchanged |
| R7 | Metadata field patch races worker pointer update/delete; invalid/foreign/malicious fields; combined meta pressure | Redis Lua and memory parity: current transcript pointer/worker siblings unchanged, no recreate, no keys leaked; no retry loop |
| R8 | queueMirror absent/invalid/[], legacy fallback, repeated text, strip-before-send/failed-start restore, response loss, coalescing | No ordinary double strip or duplicate append; explicit[] overrides old queue; best-effort cross-tab LWW documented |
| R9 | Switch/Clear/unmount during I/O, queue rearm/pause/promote, late acknowledgement | Old epoch cannot repaint/update selected view/rearm old FIFO; abort≠cancel |
| R10 | Required gates | npm test, npm run test:int, npm run typecheck, npm run build; keep #959 foundation green |
| R11 | Hosted browser desktop/~390px | F5/Stop/new thinking/two-turn conversation; composer/status geometry stable, no React fallback |
Rewrite old unconditional startIndex:0 cold expectations. Unit helper-only tests, old harnessChat.wasm-int or screenshots alone don't satisfy R1–R3. #924 closes only with these production-path real-ring tests and shipping this PR.
Docs / operations / implementation order
| Surface | Change |
|---|---|
docs/session-model.md |
Active best-effort tail policy, partial-write firewall, metadata PATCH/queueMirror and accepted LWW residual |
docs/agent-stream.md |
Negotiated production cold/indexed apply, lifecycle and post-handoff thinking |
docs/feature-divide.md |
Host I/O / canvas lifecycle and no second UI |
docs/harness-limits.md |
Recovery budgets/excerpts, no full-history accuracy claim |
AGENTS.md |
Controller/store primitive ownership and real-Wasm no-replay gate |
README.md |
Reload resumes recent view without historical replay; link guide |
SECURITY.md |
Allowlisted owned metadata PATCH, no client transcript pointer or secrets |
.env.example, native protocol guide |
N/A no new env/secret/export |
No administrative mutate/migration/backfill; existing Git deployment and int-durable current artifact. Small new metadata lazily appears; old readers ignore it; source Blob unchanged. No new dashboard setting/laptop ops. Build/compile only usual agent/CI path; no native changes planned.
Implementation: parser/client and partial types → metadata field primitive/PATCH/queue tests → shared controller + all cold callsites → apply/finalization/queue lifecycle → real-Wasm int → docs/gates. Exactly one PR.
DoD / residuals
- G1–G4, R1–R10 green; R11 recorded; actual production path not a parallel fixture.
- No chronological cold replay or prefix wipe; Busy/Stop works while recovering, post-H thinking works.
- Partial rows cannot full-PUT or seed promptHistory; narrow field patch cannot change current pointer/body/worker fields.
- QueueMirror/empty tombstone and ordinary duplicate semantics tested, no transactional journal.
- Existing caps/inference/worker behavior/native protocol unchanged; docs complete.
- PR
Fixes #960,Fixes #924only with gate evidence;Refs #958 #553, no source-spike close yet.
Open questions: none. Accepted: imperfect recent history, fragmented continuation rows, missed legacy model context when no server seed, best-effort cross-tab/old-worker preference/queue synchronization. Not accepted: cross-session data exposure, accidental partial transcript replacement, hidden rerun/cancel, or failed no-replay integration gates.
Refinement vs parent / old draft
The global source-generation/CAS and exact terminal-commit requirements are removed under the operator's best-effort decision. The replacement is a small allowlisted field-only patch, not a full-envelope read-copy-upsert, so a preference update cannot roll back a concurrently advanced transcript pointer. Parent map is aligned to this concrete refinement. No dependency on perfect legacy history remains.
Activity
[-]plan: A5 viewport — phase 2 — replay-free cold attach and real-Wasm #924 close gate[/-][+]plan: A5 viewport — phase 2 — usable tail-first F5 and real-Wasm #924 close gate[/+]fix(viewport): allowlist decoded snapshot carriers
docs(viewport): lock stored done/error as producer-terminal
btipling commentedon Sep 7, 2026
Plan to implement A5 viewport phase 2 — tail-first cold attach + partial-view firewall + queueMirror (plan issue #960) (Status: IN PROGRESS)
Last updated: 2026-09-07
Open design decisions (most complex → least)
historyComplete:falseview guard)patchEnvelopeMetaFieldsstore primitive + PATCH routemeta.queueMirrorcarrier + tombstone[]#1 — cold production controller (✅ decided)
turnAttach.decideAttachClassreturns{kind:'cold', startIndex:0};harnessChat.tscold attach (dedup branch) wipes the ring through last user and replays; the host must instead negotiateviewportVersion=1&hydrate=tail, consume the one snapshot-first SSE, and paint through the existing ring/queue-preserving seam.lib/turnAttach.ts:120–169(cold = startIndex 0),lib/harnessChat.ts:1293–1310(cold dedupprefixThroughLastUser+ full re-hydrate),app/harness/HarnessHost.tsx:628–646(kickColdAttach),lib/viewportStreamProtocol.ts(ViewportModecold/indexed),app/api/turns/[runId]/stream/route.ts:234–259(server viewport pipe exists on plan: A5 viewport — phase 1 — bounded best-effort tail read and snapshot-first attach #959).lib/viewportAttach.ts+ publicviewportAttachTurnStream(runId, opts)inlib/turnApi.ts. GET coldviewportVersion=1&hydrate=tail, indexedstartIndex. DecodeViewportStreamDecoderrecords:viewport_state→ shared apply lifecycle;viewport_snapshot→ explicit viewport view/package via newlib/viewportSession.ts(viewportSnapshotToView(rows, historyComplete=false)— explicit type, not a heal);turn_event→ sharedcreateApplyTurnEventonEvent;viewport_end→ result;viewport_error→ failure. Wire inrunHarnessTurnviaopts.attach.viewportAttach, defaulting to the new client.viewport_state→ Busy/Stop restore timing; snapshotreplacevs cached rows;resumeIndex→ heap cursor seeding → resolve in implement.int/f5-attach.int.test.tsrewrite + new real-Wasm rows (R1–R3).#2 — partial-view firewall (🔴 queued)
repo.put(whole-snapshot) today.lib/sessionRepository.tstrimForCloudPut/put(:830,:1304); snapshot restore paths inlib/sessionBoot.ts.SessionSnapshot.historyComplete?: booleanexplicit marker + central guardassertCompleteForCloudPutrejecting any view atput/mint/upload/flatten; default-undefined = complete for legacy paths — CHOSEN (explicit type, not length inference).#3 — patchEnvelopeMetaFields (🔴 queued)
lib/sessions/redisSessionStore.ts:374–424upsertEnvelope(read→replace→set),lib/sessions/memorySessionStore.ts:109+,app/api/sessions/[id]/envelope/route.ts(PUT only).patchEnvelopeMetaFields(key, changes)onServerSessionStore(Redis: one small Lua eval that reads, patches allowlisted fields, bumpsupdatedAt, preserves TTL, fails closed on missing/corrupt) + memory parity +PATCH /api/sessions/[id]/enveloperoute with{selectedModel?, reasoningEffort?, queueMirror?}allowlist — CHOSEN.transcriptPointerover the current one.#4 — queueMirror (🔴 queued)
sanitizeQueuedrops[]to unset; new mirror needs an explicit empty tombstone.lib/turnQueue.ts:63–77; envelope meta reserved keyslib/sessions/sessionStore.ts:83–104.queueMirror(JSON string, present-empty valid), typed parser preserving absent/invalid vs[], reads prefer valid mirror else legacy blob queue, host strip-before-POST + coalesced metadata channel — CHOSEN.#5 — apply/finalization lifecycle (🔴 queued)
turnApply.tsassumes a live-streaming cursor; snapshot rows anddone.textover sampled text must not double-paint.lib/turnApply.ts:583–598(donefinalize), heap cursorbumpStreamCursor(:456).snapshot.resumeIndex; fresh live continuation rows; never append fulldone.textover restored rows;viewport_enddrives lifecycle.#6 — Real-Wasm int gates + docs (🔴 queued)
int/, rewrite unconditionalstartIndex:0cold expectations; docs per plan table (session-model, agent-stream, feature-divide, harness-limits, AGENTS, README, SECURITY)..env.example/native protocol: N/A.Progress log (append-only)
viewportAttachTurnStream(newlib/viewportAttach.ts+ public client inlib/turnApi.ts), wires viarunHarnessTurnopts.attach.viewportAttach; newlib/viewportSession.tsmarks the viewhistoryComplete:falseexplicitly (never persisting it marks decision 2.2 Install and register GitHub Actions self-hosted runner #2). Status → IN PROGRESS. 2.1 Provision DigitalOcean droplet for builds #1 designing.feat/a5-tail-cold-attachoff updatedmain(f167422, includes merged plan: A5 viewport — phase 1 — bounded best-effort tail read and snapshot-first attach #959/PR feat(viewport): bounded best-effort tail read and snapshot-first attach (plan #959) #963). Status → IN PROGRESS. 2.1 Provision DigitalOcean droplet for builds #1 designing.Plan delta / living-docs note
None yet.
test(viewport): gold-typo viewportAttach unit gate comments (plan #960)