Motivation
Operators can search archived terminal output in the History GUI, and the engine serves a real FTS5 search over it — but agents cannot reach any of it. The vogt MCP session surface (session.list/start/stop/attention/outcome) is metadata-only; the engine adapter deliberately drops scrollback (src/vogt/adapters/engine/client.py:64). Even for the GUI, "history" means dead sessions only: output is FTS-indexed exactly once, at session death (engine/server/src/pty.rs:446-500), so a live session is invisible to search (see #477, #475).
Goal: all history — live and archived — searchable by agents (MCP), scripts (HTTP API), and operators (GUI), without letting the history store eat the disk.
Full design (with code references and rejected alternatives) lives in the operator-local, git-ignored docs/local/SESSION_HISTORY_SEARCH_DESIGN.md.
Key facts
- Auth already suffices: history GETs need no capability (
auth.rs:426-428 gates only DELETE/POST behind history-write); the vogt-core engine token works today.
- Live output is already on disk from byte one (
state_dir/session-logs/{uuid}.log), and GET /api/history/{id}/log reads it with no DB lookup — it already answers for live sessions. This design promotes that quirk to a feature.
- No retention exists. Nothing calls cleanup.
DB size / performance (prod, 2026-08-31)
archived_session_count: 62
log_bytes: 1,841,138,753 (~1.84 GB — avg ~30 MB/session)
db_bytes: 202,596,352 (~203 MB FTS database)
62 sessions → ~2 GB already, growing unbounded. Design decisions driven by this:
- Live search = on-demand bounded scan (last 256 KiB of each live session's log, config-tunable), NOT incremental FTS indexing — periodic re-index is a whole-output DELETE+INSERT per tick, ~O(n²) write amplification for ~30 MB sessions, plus DB growth while sessions run. The scan costs zero DB writes; per-search cost is O(live_sessions × cap) — negligible.
- Retention sweeper:
history_retention_days config (default 30, 0 = forever) + periodic sweeper calling the existing cleanup_old_sessions, mirroring the assistant-log sweeper (config.rs:274, app.rs:210-214). Bounds both history.db and session-logs/.
- History GETs are un-rate-limited and capability-free — accepted for single-operator use; flag for the open-source review (a public deployment wants a
history-read capability / GET limits).
- Future work: cap indexed bytes per session; compress or cap raw
.log files (90% of the footprint).
Surface
Three new vogt-core read ops (registry auto-generates MCP + CLI + REST + OpenAPI):
| Operation |
MCP tool |
Params |
session.search_output |
session_search_output |
q, limit, include_live (default true) |
session.log_tail |
session_log_tail |
id, tail_bytes (≤256 KiB), strip_ansi (default true) |
session.history_list |
session_history_list |
limit, offset |
Results carry the engine: str | null degradation field (FR-E9 style) and a live: bool marker on search hits.
Work slices
Motivation
Operators can search archived terminal output in the History GUI, and the engine serves a real FTS5 search over it — but agents cannot reach any of it. The vogt MCP session surface (
session.list/start/stop/attention/outcome) is metadata-only; the engine adapter deliberately drops scrollback (src/vogt/adapters/engine/client.py:64). Even for the GUI, "history" means dead sessions only: output is FTS-indexed exactly once, at session death (engine/server/src/pty.rs:446-500), so a live session is invisible to search (see #477, #475).Goal: all history — live and archived — searchable by agents (MCP), scripts (HTTP API), and operators (GUI), without letting the history store eat the disk.
Full design (with code references and rejected alternatives) lives in the operator-local, git-ignored
docs/local/SESSION_HISTORY_SEARCH_DESIGN.md.Key facts
auth.rs:426-428gates only DELETE/POST behindhistory-write); the vogt-core engine token works today.state_dir/session-logs/{uuid}.log), andGET /api/history/{id}/logreads it with no DB lookup — it already answers for live sessions. This design promotes that quirk to a feature.DB size / performance (prod, 2026-08-31)
62 sessions → ~2 GB already, growing unbounded. Design decisions driven by this:
history_retention_daysconfig (default 30, 0 = forever) + periodic sweeper calling the existingcleanup_old_sessions, mirroring the assistant-log sweeper (config.rs:274,app.rs:210-214). Bounds bothhistory.dbandsession-logs/.history-readcapability / GET limits)..logfiles (90% of the footprint).Surface
Three new vogt-core read ops (registry auto-generates MCP + CLI + REST + OpenAPI):
session.search_outputsession_search_outputq,limit,include_live(default true)session.log_tailsession_log_tailid,tail_bytes(≤256 KiB),strip_ansi(default true)session.history_listsession_history_listlimit,offsetResults carry the
engine: str | nulldegradation field (FR-E9 style) and alive: boolmarker on search hits.Work slices
include_liveonGET /api/history/search(bounded live-log scan, ANSI-stripped,live: trueresults)strip_ansiparam onGET /api/history/{id}/log(reuse archive-path stripper)history_retention_daysconfig + sweeper (mirror assistant-log pattern)scope="read")livebadge; replay via existing log route + readable-transcript renderer) — closes the History tab should list ALL sessions — live and dead — not only archived ones #477 gapCURATED_READSopt-in (vogt_tools.rs);docs/ENGINE.mdhistory section refresh