Skip to content

chore: reconcile main into dev (unblock 0.4.0 promotion) - #498

Merged
thedancingdeveloper merged 27 commits into
devfrom
reconcile-040
Sep 1, 2026
Merged

chore: reconcile main into dev (unblock 0.4.0 promotion)#498
thedancingdeveloper merged 27 commits into
devfrom
reconcile-040

Conversation

@thedancingdeveloper

Copy link
Copy Markdown
Contributor

Purpose: make main an ancestor of dev so the fast-forward dev→main promotion (promote.yml) can run for the 0.4.0 release. main and dev diverged at the 0.3.0 base and carried the 0.3.1-era work as parallel rebase-merged duplicates, so main was not reachable from dev.

What this is: a single git merge -s ours origin/main commit on top of dev. It keeps dev's tree byte-for-byte (0 file changes) and only records main as a second parent. Verified safe: every file main changed is already changed on dev — there is no main-only content to lose.

⚠️ Must be landed as a MERGE COMMIT, not rebase. A rebase-merge would drop the main parent and defeat the reconcile. This repo has merge-commit PRs disabled, so landing requires an admin: either temporarily enable "Allow merge commits" and merge with Create a merge commit, or admin-push reconcile-040 to dev directly (git push origin reconcile-040:dev).

After it lands (git merge-base --is-ancestor origin/main origin/dev returns true), the 0.4.0 promotion resumes.

🤖 Generated with Claude Code

thedancingdeveloper and others added 27 commits August 29, 2026 20:54
The tagged release builds the merged `vogt-stack` image prod deploys, but it
passed `INSTALL_CADASTRE_MCP=true` and never `INSTALL_AI_CLIENTS`, so the
Dockerfile default (false) stood: v0.3.0 shipped with no `claude` or `codex`.
Production registered the `Claude Code (protected)` / `Codex (protected)`
session templates from config.rs yet could not start them — the CLIs were not
in the image. `build.yml` learned this for dev/prod (#23); `release.yml` had
not.

- release.yml: pass INSTALL_AI_CLIENTS=true and INSTALL_THECLAWBAY=true to both
  the candidate and pushed stack builds; extend the smoke step to run claude,
  codex and theclawbay so a missing CLI fails the release (NFR-Q7). Flutter
  stays out by design — a release is the `lean` pod variant (#184).
- deploy-production.yml: resolve the stack image by its published tag
  (`vogt-stack:${TAG#v}`), since docker/metadata-action strips the `v`; the old
  `:$TAG` lookup would fail the signed-digest resolve for a v-prefixed tag.
- test_deploy.py: guard the release build the way the dev build is guarded.
- Bump to 0.3.1 across pyproject, package, __init__ and build.yml; changelog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
The version lives in more places than the first commit touched: the ci.yml
product-version gate arg, the deploy compose/overlay/.env image tags, the
deployment docs, and the product-version test's expected value. Matches the
scope of the 0.3.0 bump commit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
…s operator-private

A reader following DEPLOYMENT.md could pin a plain release image for
production and get a pod whose Claude Code / Codex session templates register
but cannot start — the binaries are only ever added by the §3.2
VOGT_INSTALL_AI_CLIENTS build flag, which no published image sets. §3.2 now
says so directly, and §7 states the boundary: the production desired-state and
overlays live in the operator's own repository (#204), the signed release
digests are CLI-free, and the maintainer's estate is a private deployment, not
a turnkey scenario reproducible from this tree.
…#466)

`connect()` reassigned `ws` without closing the previous socket, and several
paths reach it while a socket is still open or connecting: the reactive
park/resume effect, `onWake`, the cache-replay `.then()`, and — since the
recent silent-socket work — `recycleSocket()`, which calls `ws.close()` *and*
`scheduleReconnect(100)` so the socket's own delayed `close` event can spawn a
second `connect()`. Every leftover socket keeps its `message` listener attached
and writes the same PTY output into the one xterm instance, so lines render two
or three times. Seen live on dev and prod.

Make `connect()` the single owner of the socket: drop any existing socket
first, then capture the new socket in a local `socket` and guard every handler
with `if (ws !== socket) return;` so a replaced or parked socket's late
open/message/close/error events are no-ops. One terminal now has exactly one
live socket at a time. Only the racy leftover path closes a socket here; healthy
sockets are untouched (all `connect()` callers already gate on a
closed/absent socket or a prior park). PWA typecheck and the 817 web tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
#466)

Non-selected terminal sessions were force-dropped and then flooded with a
replay on switch. When a subscriber fell >1024 broadcast chunks behind, the
outbound task sent a `Lag` frame and closed the socket; the client tore down
and reattached, and the server replayed scrollback — a large/full replay that
is heavy main-thread work, which stalled the neighbouring panes' sockets and
made them lag too. The loop fed itself.

Two changes to the outbound task, server-side only (the client already handles
mid-stream snapshots and tracks its resume cursor):

- Coalesce: drain the broadcast chunks already queued (`try_recv`) into one
  WebSocket frame, bounded at 256 KiB. A chatty session now wakes the client's
  single message-draining thread far fewer times, which is what was starving
  the other panes.
- In-band lag recovery: on `Lagged`, re-snapshot from the client's exact
  last-delivered position and stream it as the normal SnapshotStart → payload
  → SnapshotDone sequence on the *same* socket — a `reset:false` delta when the
  cursor is still in the scrollback window, a `reset:true` reload when it has
  aged out. The socket never closes, so there is no reconnect/replay cascade. A
  circuit breaker (5 consecutive resyncs with no live send) falls back to the
  old Lag-and-close so a hopelessly-behind client can't drive unbounded
  resnapshotting.

Pure `coalesce` helper is unit-tested; fmt, clippy (-D warnings) and the ws
tests pass. Deliberately out of scope (follow-ups): decoupling client socket
draining from the render thread (Web Worker) and pausing hidden panes'
subscriptions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
…es (#459)

Relax the ureq/url version floors in voice/tts/Cargo.toml and update the
shared voice/Cargo.lock so the sidecar no longer resolves the vulnerable
transitive crates:

- rustls-webpki 0.102.8 -> 0.103.15  (clears RUSTSEC-2026-0104 high panic-on-CRL,
  RUSTSEC-2026-0049, RUSTSEC-2026-0098, RUSTSEC-2026-0099)
- idna 0.4.0 -> 1.1.0                 (clears RUSTSEC-2024-0421)
- ureq 2.9.7 -> 2.12.1               (kept on the 2.x line; no ureq-3 API use)
- url 2.4.1 -> 2.5.8

ureq/url are not referenced from tts/src; they exist only to steer the shared
lock. cargo-audit against voice/Cargo.lock goes from 5 vulnerabilities to 0
(only the pre-existing, unrelated audiopus_sys "unmaintained" notice remains).
Voice CI (fmt, clippy -D warnings, test) passes locally with the updated lock.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
promote.yml has never succeeded: the self-hosted [self-hosted] runner has no
gh CLI, so `gh api` / `gh run list` / `gh pr list` / `gh pr create` all die
with `gh: command not found` (exit 127) before any gate runs.

Replace every gh call with a small curl+jq `api()` helper hitting
$GITHUB_API_URL (curl and jq are both present on the runner and already used
by deploy-production.yml). PR creation still authenticates as
VOGT_PROMOTION_TOKEN so the opened PR receives its normal checks.

Also add `actions: read` to the job permissions: the dev deployment-receipt
gate reads actions/workflows runs and run artifacts, which the previous
permission block (contents/checks/pull-requests only) did not grant. That gap
was never reached before because the run failed earlier on the missing gh.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
…ls (#460)

test_promotion_is_fast_forward_only_and_never_pushes_a_branch pinned the old
`gh pr create` string; #460 replaced the gh CLI with curl+REST. Update the
assertions to the new contract — the PR is opened via POST .../pulls with the
promotion token, `gh pr create`/`gh api`/`gh run list` are gone, and
`actions: read` is granted for the receipt lookups — while keeping the
never-pushes-a-branch and fast-forward-only invariants.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F4VhQKnLKBA847LSZpPPds
Picking a preset (e.g. Shell) fired promptUser unconditionally even
though buildDefaultSessionName had already produced a valid default
name, contradicting the documented "no name prompt unless Shift"
behaviour in terminalNaming.ts. Create the session immediately with the
default name and only prompt when Shift is held on the preset, matching
the + Session action.

Fixes #472

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BMbeugEzMF6K26Hq3X3dsh
History was the archived-only view: it listed only exited sessions and
told the reader their running ones would "appear here once they exit".
Make History the one place that lists every session, live and dead, with
liveness as a filterable facet rather than a gate (frontend half of #477;
the backend history union is deferred to #475).

- Union the live session registry into the archived list, keyed by id
  with the live entry winning on conflict, keeping created_at DESC order,
  and badge each row Live or Exited.
- Serve a live row's detail from the registry (no archive DB lookup that
  would 404) and route its replay preview through GET /api/history/{id}/log,
  which tails the on-disk log directly and works for a running session.
- Give the Status filter real meaning: running / exited / unfinished,
  wired to the merged data; the previously-dead "unfinished" case now
  matches archived rows with no exit code.
- Fix now-wrong copy: the "appear here once they exit" note and the
  "No archived sessions." empty state, plus the USER_GUIDE History section.
- Hide archive-only Export/Delete for live rows and note that live output
  is not yet in the search index (search stays archived-only).

Updates historyTruth vitest expectations and adds a union + status-filter
test; realigns the pinned Playwright assertion to the new truth.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
…ows, and backfill (#475)

The History tab recorded nothing on long-lived deployments: a row was only
written when a child exited while the engine was alive, but prod sessions are
agent shells that never `exit`, and the engine had no graceful shutdown, so
every redeploy SIGKILLed the PTYs and archived nothing. Raw transcripts
persisted on disk; only the index row was missing.

- Graceful shutdown: `serve_forever` now wraps axum with
  `with_graceful_shutdown` on SIGTERM/SIGINT and drains every live session to
  history before exit (`SessionRegistry::drain_to_history` ->
  `pty::archive_live_session`), setting `ended_at` and the child's exit code
  (NULL == terminated/unknown).
- Provisional row at spawn: the registry writes a metadata row when it inserts
  a session (`ended_at`/`exit_code` NULL), finalized by the exit waiter. The
  `archive_session` upsert is guarded with COALESCE/MAX so a late provisional
  write can never NULL a completed row or shrink its scrollback.
- Startup backfill: `SessionHistory::backfill_orphaned_logs` indexes raw
  `session-logs/*.log` files with no row, inserting NULL-exit rows and making
  the recovered transcript searchable, without duplicating existing rows.
- The `unfinished` history filter is now real: NULL exit codes exist, and the
  existing frontend filter selects them.
- Docs: correct ENGINE.md — disabled history routes answer 404, not 500.

Tests: integration coverage for the shutdown drain of a never-exited session,
the provisional-row lifecycle (present at spawn, not clobbered after finalize),
and the startup backfill (recover + searchable + idempotent).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
…w races (#475)

The provisional row written at spawn (exit_code NULL) now appears in the
history APIs before the exit waiter finalizes it, so history poll loops that
broke on first appearance and then asserted a concrete outcome could observe
the provisional state and fail deterministically in CI:

- `exited_sessions_are_archived_searchable_and_deletable` asserted exit_code
  == 7 but caught the NULL provisional row first.
- `archived_history_log_preview_and_download_work` read the tail log before the
  reader thread had flushed the full transcript (only guaranteed once finalized).
- `archived_history_cleanup_removes_old_sessions_and_logs` could delete the
  provisional row before finalize, whose upsert then re-inserted it, breaking
  the subsequent 404 assert.

Each loop now waits for a non-NULL (or concrete) exit_code before proceeding,
keeping the same deadline discipline. Product behavior is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
A cold terminal attach (a fresh browser with no cache) sends no
`resume_from`, so the server took the full-snapshot path and shipped the
entire scrollback ring (up to DEFAULT_SCROLLBACK_BYTES, 4 MiB) which the
client then replayed uncapped, making first-open of every session slow.
REPLAY_TAIL_MAX_BYTES (1 MiB) already bounded this, but only on the
IndexedDB cache-restore path.

Let the client send a `snapshot_tail_bytes` hint on the WS auth frame when
`resume_from` is absent. The server trims the full snapshot to at most that
many trailing bytes at `snapshot_for_attach`, aligned forward to a
ground-state boundary (past a newline, else the next UTF-8 codepoint start)
so replay never begins mid-escape-sequence or mid-codepoint. Warm reattach
via `resume_from` (the `reset: false` delta path) is untouched — the tail
cap is ignored whenever a cursor is present, including the aged-out full
reset — so it stays byte-for-byte unchanged. The trimmed front does not move
the returned position, so the live stream still resumes with no gap.

Client: `openAttach` sends the hint set to REPLAY_TAIL_MAX_BYTES on a cold
attach only, keeping one source of truth for the budget.

Tests: contract round-trip for the hint; scrollback `snapshot_tail` unit
tests (newline seam, UTF-8 boundary, whole-buffer passthrough); two engine
integration tests (cold attach caps ≤ N with reset:true; warm resume is not
narrowed); and a vitest asserting the cold auth frame carries the hint and
the warm one does not.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
In a fresh browser every session took the expensive cold-attach path the
first time it was clicked, one at a time, while the reader waited. Add a
background pre-warm coordinator that, after boot and once the active pane
has finished its own replay, warms the most-recently-active unopened
sessions so a later click restores from cache and reattaches with a cheap
`resume_from` delta instead of a cold full snapshot.

- New `terminalPrewarm.ts`: a WebSocket-only worker. Each warm attach is a
  single bounded cold attach that requests only the snapshot tail (the #474
  cap that `openAttach` already sends when no `resume_from` is present),
  writes the tail + absolute `outputPosition` to the same IndexedDB cache
  the restore path reads, then closes its socket. It never mounts an xterm
  and never runs the Terminal park/connect state machine, so it carries no
  risk to the #466 no-socket-leak invariant — every warm attach owns exactly
  one socket and always closes it.
- Warms up to `MAX_CACHED_SESSIONS` (8), most-recently-active first,
  strictly sequentially (concurrency 1), re-checking the gate between each.
- Foreground preemption: a warm attach never touches the shared FIFO replay
  queue, so it structurally cannot sit ahead of the pane the reader clicked;
  on top of that, the loop pauses while any foreground pane holds the
  replay gate (`beginForegroundReplay`/`foregroundReplayActive`), wired into
  Terminal.tsx's initial-load / snapshot-done / park / resume / cleanup
  lifecycle. This also gates constraint 5 (no warm-up until the active pane
  finishes its replay).
- Respects document visibility: paused while hidden, kicked when visible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
The redesign hides `.terminal-workspace-roster` on phone and replaces it
with the swipeable mobile pager, so the broadcast-state badge ("Input
fan-out") is no longer surfaced on the phone composition. Scope that
visibility assertion to the desktop project, where the roster still owns
the pane list.

Re-baseline the two desktop demo screenshots the redesign shifted:
demo-inbox-768 (phone bottom-nav spacing) and demo-agent-390 (the
390px terminal now renders the mobile pager). Regenerated -linux
snapshots on a Linux host to match the self-hosted CI runner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
Link the two public demo sites — vogt-demo (desktop PWA) and
vogt-mobile-demo (mobile app showcase) — from the README intro.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
Bump the canonical product version across the gated manifests
(pyproject, src/vogt/__init__.py, web + mobile package.json,
build.yml VOGT_PRODUCT_VERSION), the CI version gate, deploy manifests,
docs image-tag references, and uv.lock; add the 0.3.1 CHANGELOG entry.

check_product_version.py 0.3.1 passes. Does NOT include the previously
reverted 'bake AI clients into the release image' change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013uH61idJmpL3z1PP3Bf9ka
main and dev diverged at the 0.3.0 base and carried the 0.3.1-era work as
parallel rebase-merged duplicates, so main was not an ancestor of dev and the
fast-forward dev->main promotion could not run. dev is a verified content
superset of main (no file main changed is untouched on dev), so this records
main as an ancestor while keeping dev's tree byte-for-byte (-s ours).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N7eWQbHi8HyLyhaRA2piEn
@thedancingdeveloper
thedancingdeveloper merged commit d2c73de into dev Sep 1, 2026
13 checks passed
@thedancingdeveloper
thedancingdeveloper deleted the reconcile-040 branch September 1, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant