From db53d0fadd1770ed0e2fd76315889daaa826be74 Mon Sep 17 00:00:00 2001 From: thedancingdeveloper <306930456+thedancingdeveloper@users.noreply.github.com> Date: Mon, 31 Aug 2026 23:05:51 +0000 Subject: [PATCH] chore(public): retire historical docs, estate references and compat framing Prepare the public tree as an open-source project that documents only its current state: - Move completed design/workplan records (COMPARE, DEMO_SITE_PLAN, FORGE_FRONTEND_DESIGN, IMPORT-PLAYBOOK, VOICE_POC, VOICE_DELIVERY, the M0-M14 ROADMAP) out of the tree; they survive in the operator's git-ignored docs/local/. ROADMAP.md is rewritten as a short current-state roadmap (designed-but-unbuilt, pending cleanup, deliberate deferrals). - Remove opensource.md. With no released users there is no compatibility contract to publish; the legacy MYDEVENV2_* names become pending housekeeping (ROADMAP.md) instead of documented aliases, and CONTRIBUTING now forbids adding new ones. - Trim delivery narration from DESIGN.md, SCHEMA.md, ENGINE.md, USER_GUIDE.md, CUSTOMISATION.md and DEPLOYMENT.md: revision/date preambles, milestone (M-number) and revision (r-number) references, the stale buildless-GUI layer entry, and the dated open-questions log (now "Standing decisions"). - Scrub *.sprooty.com and other estate hostnames from tracked files (ci.yml comment, capacitor.config.ts, smoke_merged_stack.sh usage, identity.py docstring, test_connect.py fixtures); the test_public_delivery.py denylist keeps enforcing the rule. - Drop scripts/pin_estate_image.py (orphaned; its consumer lives in the private ops repository) and its test. - Add AI_POLICY.md: this stack is written by AI agents for AI-forward developers, and what that means for use, contribution and maintenance. Verified: uv run pytest (1296 passed, 25 skipped), mypy, ruff, scripts/check_docs.py. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_0161etTkUPyRoUXnHQ8xgKUu --- .github/workflows/ci.yml | 2 +- AI_POLICY.md | 51 + CHANGELOG.md | 4 +- README.md | 7 +- docs/COMPARE.md | 293 ------ docs/CONTRIBUTING.md | 17 +- docs/CUSTOMISATION.md | 9 +- docs/DEMO_SITE_PLAN.md | 470 --------- docs/DEPLOYMENT.md | 2 +- docs/DESIGN.md | 256 ++--- docs/ENGINE.md | 48 +- docs/FORGE_FRONTEND_DESIGN.md | 269 ------ docs/GETTING_STARTED.md | 4 +- docs/IMPORT-PLAYBOOK.md | 247 ----- docs/ROADMAP.md | 1272 +------------------------ docs/SCHEMA.md | 39 +- docs/USER_GUIDE.md | 18 +- docs/VOICE_DELIVERY.md | 105 -- docs/VOICE_POC.md | 374 -------- engine/README.md | 1 - mobile/capacitor.config.ts | 8 +- opensource.md | 155 --- scripts/check_docs.py | 10 +- scripts/pin_estate_image.py | 63 -- scripts/smoke_merged_stack.sh | 2 +- src/vogt/application/identity.py | 16 +- tests/test_connect.py | 10 +- tests/test_mobile_identity.py | 7 +- tests/test_pin_estate_image.py | 53 -- web/src/__tests__/voiceRepair.test.ts | 2 +- 30 files changed, 248 insertions(+), 3566 deletions(-) create mode 100644 AI_POLICY.md delete mode 100644 docs/COMPARE.md delete mode 100644 docs/DEMO_SITE_PLAN.md delete mode 100644 docs/FORGE_FRONTEND_DESIGN.md delete mode 100644 docs/IMPORT-PLAYBOOK.md delete mode 100644 docs/VOICE_DELIVERY.md delete mode 100644 docs/VOICE_POC.md delete mode 100644 opensource.md delete mode 100644 scripts/pin_estate_image.py delete mode 100644 tests/test_pin_estate_image.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bf9350fd..792db3e6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -600,7 +600,7 @@ jobs: # repo/environment variable `VOGT_ANDROID_SERVER_URL` and only falls back # to loopback when that is unset. Unset, the artifact stays a build-proof # that points at nothing (installing it shows the offline page). Set the - # variable to a real front door — e.g. `https://vogt-dev.sprooty.com` — + # variable to a real front door — e.g. `https://vogt.example.com` — # and the same job publishes an APK that is actually installable and # loads that Vogt. It stays debug-signed (below), so it is a sideload # channel, not a store/release artifact; a signed release APK is still a diff --git a/AI_POLICY.md b/AI_POLICY.md new file mode 100644 index 00000000..f23cb7e1 --- /dev/null +++ b/AI_POLICY.md @@ -0,0 +1,51 @@ +# AI policy + +This stack was written by AI agents, for AI-forward developers. + +That sentence is a statement of fact, not a disclaimer. Vogt's code, +documentation, and tests were produced by AI coding agents working under the +direction and review of the human maintainer, who is accountable for every +line the repository ships. It is also a statement of intent: Vogt exists to +run product work that people and agents do together, so agent surfaces — +MCP, the operation registry, `docs/AGENT_GUIDE.md`, terminal sessions an +agent can drive — are first-class product, not integrations bolted onto a +human tool. + +## What that means for using Vogt + +- The documentation is written to be read by people and agents alike. + `AGENTS.md` is the repository's working contract for agents; + `docs/AGENT_GUIDE.md` is the guide for agents running work *through* a + deployed Vogt. +- Transport parity is a design rule: anything the CLI or REST surface can + do, MCP can do, with tests asserting they agree. An agent is never a + second-class client. + +## What that means for contributing + +AI-assisted and AI-authored contributions are welcome and expected — most of +this repository was built that way. The bar does not move either direction +because an agent was involved: + +- **You are accountable for what you submit.** Run the checks in + `docs/CONTRIBUTING.md`, understand the change, and be able to answer + review questions about it. "The agent wrote it" is not an answer. +- **Unreviewed agent output is not a contribution.** Bulk or speculative + pull requests, issues generated without reading the code, and changes + their author cannot explain will be closed without detailed review. +- **The same quality rules apply.** Tests, transport parity, the audited + write path, and the documented layer order bind agents and people + equally. +- **You must hold the rights to what you submit.** Contributions are + accepted under the repository's MIT licence; submitting output you are + not entitled to license is your responsibility, not the project's. + +Disclosure of AI involvement in a contribution is not required. It is the +default assumption here. + +## What that means for maintenance + +Agents triage, review, and land work in this repository as a matter of +routine. Decisions about scope, releases, and security remain with the human +maintainer, and security reports go to the channel named in `SECURITY.md` — +not to an automated surface. diff --git a/CHANGELOG.md b/CHANGELOG.md index ff8ba053..9e6e43cb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,7 @@ All notable changes to Vogt are documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) once released — pre-1.0, a minor version bump may still carry a breaking -change, per the compatibility policy in [`opensource.md`](opensource.md). +change. This file starts from the public 0.2.0 baseline; earlier history lives in the git log rather than being reconstructed here. @@ -56,7 +56,7 @@ migration is required. The first release since the merged core+engine stack reached production. No operation was renamed or removed; no schema migration is required (the declared schema stays at 0015, the observed schema at 0004). Pre-1.0, this remains a -minor bump per the compatibility policy in [`opensource.md`](opensource.md). +minor bump. ### Added diff --git a/README.md b/README.md index c214ae70..3cd0bf97 100644 --- a/README.md +++ b/README.md @@ -155,8 +155,7 @@ points: configuration, Compose overlays, image extension, running behind your own front door, and the optional integrations above. The Rust engine and Capacitor mobile shell (`engine/`, `web/`, `mobile/`) are the largest worked example of those extension points; the engine is the stack's front half, and -the mobile shell wraps its PWA. See [`opensource.md`](opensource.md) for the -public/private boundary and compatibility policy. +the mobile shell wraps its PWA. ## Development @@ -198,8 +197,8 @@ workflow and [`docs/CONFIG.md`](docs/CONFIG.md) for every setting. what it owns, how to build and run it, its wire contract, the assistant. - [Design outline](docs/DESIGN.md) — architecture and domain decisions. - [Contributing](docs/CONTRIBUTING.md) — workflow and checks. -- [Public boundary and compatibility](opensource.md) — what is supported, - what is optional, which legacy names remain as aliases. +- [AI policy](AI_POLICY.md) — how this stack is built: by AI agents, for + AI-forward developers, and what that means for contributions. ## Licence diff --git a/docs/COMPARE.md b/docs/COMPARE.md deleted file mode 100644 index b2a5d456..00000000 --- a/docs/COMPARE.md +++ /dev/null @@ -1,293 +0,0 @@ -# Vogt and ProPR — a comparison, and what is worth taking - -*Read on 2026-08-16 against `integry/propr` at `d537c25` (v1.1.0), its -Docusaurus documentation set under `docs/docs/`, and this repository's -`README.md`, `docs/ENGINE.md` and the requirements gap register.* - -**What this document is.** A comparison against one adjacent product, and a -ranked list of features it has that Vogt could adopt. It is **not a backlog and -nothing in it is scheduled** — the same rule the requirements gap register -states for itself. An item here becomes work when it becomes a numbered -requirement, and not before. Where an item touches a requirement that already -exists, or a decision already taken, this document says so by ID rather than -re-opening it. - -**Freshness.** Every claim about ProPR below was read from that commit's -documentation or source on that date. Every claim about Vogt was read from this -tree on the same day. Neither is re-checked on a timer, which is the same -contract everything else in this repository offers. - ---- - -## 1. What the two products are - -They are not competitors. They occupy adjacent halves of the same problem and -overlap in one place. - -**Vogt is a tracker with a doing-surface.** Projects, backlog, bugs, ranking, -drift, contract, audit, and — since the session-engine merge — PTY sessions bound to -work items. Its centre of gravity is *knowing what is true, with provenance and -freshness*. Its principles are in `README.md`: observed-first, reports never -enforces, never goes looking, declared vs observed always separated, every -answer carries provenance, transport parity, writes are first-class, -forge-optional, MCP by default. - -**ProPR is a pipeline with no tracker.** It turns a GitHub issue into a -reviewed, merged pull request, deterministically and repeatedly — its README -claims 2,100+ merged PRs since May 2025, 690+ in its own repository. It has no -backlog of its own, no ranking, no drift, no contract, no audit-with-reason, no -MCP surface, and no declared-versus-observed distinction. GitHub issues, PRs and -labels **are** its data model and its state machine. - -The overlap is the session engine. Everything worth importing sits there. - -### 1.1 Feature-by-feature - -| Capability | Vogt | ProPR | -|---|---|---| -| Backlog with explainable ranking | Yes (FR-V*) | No | -| Cross-project rollup | Yes | No — per-repository stats only | -| Declared vs observed, drift | Yes (M3) | No | -| Contract checks | Yes (M3) | No | -| Audit with a mandatory reason | Yes, every write | No | -| MCP server | Yes, generated from the registry | No | -| Transport parity (CLI/REST/MCP/GUI) | Yes, asserted by test | No — UI and CLI are separate clients | -| Works with no forge at all | Yes (principle 8) | No — a GitHub App is required | -| Terminal sessions, watchable/steerable | Yes (FR-E1, FR-E2) | No — logs and streamed output only | -| Scheduled agent runs | Yes (FR-E7: manual/interval/daily) | No | -| Per-run git worktree isolation | **No** | Yes | -| Branch + commit + push + open PR from a run | **No** | Yes | -| Cost and token accounting | **No** | Yes, per call and aggregated | -| Provider capacity/rate-limit visibility | **No** | Yes, via optional Agent Tank | -| Multi-model fan-out on one item | **No** | Yes, one PR per `llm-*` label | -| Plan-before-execute drafting surface | **No** | Yes (Planner Studio) | -| Repository indexing and summaries | Partial — collectors, not summaries | Yes | -| Scored, structured AI review | **No** | Yes (`/review`, `Score: N/10`) | -| Queue with retry, backoff, correlation IDs | **No** | Yes (Redis/BullMQ) | - ---- - -## 2. How ProPR works, in enough detail to judge the imports - -**The three-phase pipeline.** Worker execution splits into phases and the agent -participates only in the middle one: - -1. **Pre-agent setup (ProPR):** pull the job from the queue, update the base - branch, create an isolated git worktree, create the task branch, prepare the - prompt and context. -2. **Agent implementation (agent):** run the selected agent CLI inside a - container against that worktree. The agent edits files; it does not push, - branch, or open pull requests. -3. **Post-agent finalisation (ProPR):** inspect changed files, commit, push, - create or update the PR with issue linking, update labels and task state. - -Because phases 1 and 3 are deterministic code rather than agent decisions, a -failure is attributable: phase 1 or 3 is a git/GitHub problem, phase 2 is an -agent problem. Branches are named -`/---`. - -**Intake** is a GitHub label. An issue tagged with a trigger label (`AI`, -`propr`) plus `llm--` routes to that agent; several model labels -fan out to one job, worktree, branch and PR *per model*. - -**Follow-up** stays in the pull request. Plain comments are picked up as -instructions (line-level ones carry file, line and diff hunk; attached images -reach the agent; comments arriving during a running job are batched). Slash -commands cover the rest: `/review` (read-only, severity-tagged findings and a -`Score: N/10` line), `/fix` (applies the review suggestions a human left -standing), `/merge`, `/switch`, `/use`, and `/ultrafix` (alternates review and -fix until the score reaches a goal, with a PR label as circuit breaker). - -**Planner Studio** is the plan-first entry point: Define & Context (repository, -model, context level, attachments, a context preview and an estimated issue -count *before* generation) → Review Plan (edit, delete, restore, undo/redo, -refinement chat) → Execution (per-issue status, pause/resume, links to the -resulting PRs). Finalising creates GitHub issues. - -**Repository knowledge** indexes monitored repositories into file and -repository summaries, with a fallback model and quota-aware cooldown so -indexing degrades rather than fails; a background worker rescans every 5 -minutes and reindexes daily. On top of the index sit repository chat, -categorised improvement suggestions, and repository todos. - -**Observability** is a task record (trigger, repository, branch, agent, model, -state, commits, PR, failure point), the exact prompt and logs, a completion -comment posted back to the PR, and a metrics trail: every model call lands in -an `llm_logs` table with execution type, model, agent alias, input/output and -cache token counts, a USD estimate, duration and correlation ID, aggregated per -repository, per model and per day, with a high-cost alert above -`LLM_COST_THRESHOLD_USD`. - -**Agent Tank** is a separate optional local service that reads each agent CLI's -own `/usage` output for subscription capacity, surfaced as usage bars and a -per-call usage delta. It is explicitly best-effort: unreachable means the bars -hide and the work completes normally. - ---- - -## 3. Tier 1 — worth building, and each closes something already owed - -### 3.1 One git worktree and branch per run - -ProPR reuses one clone per repository and creates a worktree per task; branch -names carry the model identifier so concurrent runs never collide. - -Vogt opens a session **in the project's registered root path**, and -`engine/server/src/git.rs` operates on that checkout directly. That is exactly -the hole gap **FR-E11** names: *"Two agents can edit one -checkout concurrently and neither is told. No audit row records the loss, -because both writes are legitimate."* - -Worktree-per-session does not *detect* the collision — it makes it impossible. -This is the highest-value single import in this document: it closes an owed -`could-have` by construction, and items 3.2 and 3.4 depend on it. - -### 3.2 A deterministic post-agent phase - -Vogt's session lifecycle ends at *PTY exited, exit code recorded, findings -appended* (`ENGINE.md` §7). A finalisation phase — inspect changed files, -commit, push, open a PR linked to the work item, transition the item — turns -"WI-7 had a session" into "WI-7 produced a reviewable change." - -It sits inside M5's existing rule that forge write-back is **additive and -forward-only**: opening a pull request adds and destroys nothing. It also makes -failures phase-attributable, which run records cannot do today — they carry -`exit_code` and a derived summary, not a failing phase. - -### 3.3 Cost and token accounting - -Vogt has no concept of what work costs. This is the most natural import on the -list, because it is already a Vogt-shaped sentence: *"WI-12 has cost $34 across -6 sessions, last observed 4 minutes ago."* Cost is an observation with a source -and an age, exactly like everything else here. - -Two consequences beyond the number itself: it makes an expensive loop visible -before the bill does, and — uniquely among the items in this document — a cost -observation can enter the ranking weights. - -Note ProPR's own honesty about the limits, which any Vogt version inherits: the -figure is an *estimate* from token counts and published per-model rates, with -OpenRouter as a fallback feed; provider options that change the rate but are -not reported by the CLI are not included; token counts are null for some -agents. That is a freshness-and-trust problem, and this product has a vocabulary -for it. - -### 3.4 Multi-agent fan-out on one work item - -Several `llm-*` labels on one ProPR issue produce one run, worktree, branch and -PR per model; you compare, merge one, close the rest. - -Vogt already has labels, work items and a session registry. With 3.1 in place -this is mostly wiring, and it is differentiated for a *tracker*: comparing -several implementations of one unit is a ranking problem, and ranking is what -this product does. - ---- - -## 4. Tier 2 — good fit, more design work - -### 4.1 Repository knowledge as a collector - -ProPR's indexing, summaries, repository chat and improvement suggestions map -onto machinery Vogt already has. A `repo-summary` collector produces -**observations** with freshness and trust; improvement suggestions land as -**collected subjects in the ranked backlog**, visible by default and -authoritative only once adopted (principle 1). Nothing about it needs a new -architectural concept, and the quota-aware fallback is a good pattern to copy -verbatim: degrade the model, keep the sweep moving. - -### 4.2 A planning surface that fans one request into several work items - -Planner Studio's shape — draft, preview the context and cost *before* -generating, review and edit, then commit — is a checkpoint Vogt does not have. -Vogt has work items, relations and briefs, but no drafting-then-committing -surface. - -**Check this against a closed decision before designing it.** §7.3 records -"natural-language task drafting" as **withdrawn**, but that withdrawal was -about *agent tasks*, where interval and daily schedules covered the motivating -cases. Fanning one goal into several related, reviewable work items is a -different thing. It should be adopted or refused explicitly, not inherited by -resemblance in either direction. - -### 4.3 Review as a scored observation - -`/review` emits a fixed structure — severity-tagged findings (🔴 Critical, 🟡 -Warning, 🟢 Suggestion, ✅ Positive) and a closing `Score: N/10` — and touches -no files; `/fix` then applies only what a human left standing. The reusable -part is the *structured, scored, read-only* output: a review score with a date -and a source is an observation, and it gives the backlog a quality signal it -does not currently have. - -The human-edits-between-review-and-fix step is worth keeping too. It is the -same shape as the assistant's on-screen approval (FR-T*): the machine proposes, -a person prunes, and only then does anything change. - -### 4.4 Provider capacity as an observation - -Agent Tank reads each CLI's own reported limits — nothing scraped, nothing -estimated — and ProPR degrades to "no capacity bars" when it is absent. That -best-effort contract is already this product's default posture. Cheap to add, -and it makes *"why did nothing run last night"* answerable. - ---- - -## 5. Tier 3 — take the mechanism, not the feature - -- **Queue mechanics.** Correlation IDs traced across daemon, worker and agent; - exponential backoff with jitter on transient git and GitHub failures; comment - batching while a job for the same subject runs; a worker concurrency cap; - queue-depth statistics. Vogt's scheduler is `manual`/`interval`/`daily` with - no queue, no concurrency limit and no retry. **Take the ideas, not Redis** — - a broker dependency breaks the single-node zero-dependency SQLite - self-hosting story that `README.md` puts in the stack line. -- **Comment-driven intake.** Vogt already collects GitHub issues and PRs - read-only (M5) and consolidates them at import (M7); treating a comment as - *intake* is a small extension of the same adapter. It carries a real tension - with `ENGINE.md` §8's *"it never decides to run anything"*. The argument that - a person's comment is a person's act is available and probably correct — but - it has to be made in the requirement, not discovered afterwards. -- **Signed system tasks.** ProPR authorises reverts with `SYSTEM_TASK_SECRET` - so a destructive operation cannot be injected through ordinary intake. Worth - copying for any future destructive Vogt write. -- **Author gating on intake.** Bot accounts ignored by construction, plus an - allowlist and a blocklist. Any comment-driven path needs the equivalent - before it needs anything else. - ---- - -## 6. What not to take, and why - -- **GitHub as the substrate.** ProPR cannot start without a GitHub App; issues, - PRs and labels are its database. That is the direct negation of principle 8 - (forge-optional), which is load-bearing here — plain folders and local git - are first-class in Vogt and the GitHub adapter only ever *adds*. -- **Label-triggered autonomous pickup.** ProPR's entire intake is "label an - issue and an agent starts." Autonomous work pickup is deferred by name as - a non-requirement, and named again in `ENGINE.md` §8 as the surviving core - of a reversed non-goal. Adopt the *routing* idea — a label selects agent and - model — without the *triggering* idea. -- **Authoritative state living in forge labels.** ProPR's - `-processing` → `-done` → `-failed-*` lifecycle is - precisely the pattern Vogt's `forge_state_mismatch` drift exists to catch. - Anything imported must arrive as observations with freshness and trust, or as - audited writes with a reason — never as a second state machine living - upstream. -- **Docker-socket-per-agent-run.** A container per run per agent image, with - direct socket access; their own hardening firewall ships disabled because - applying it would need `--privileged`. Vogt's PTY sessions are lighter and - better audited. -- **Redis**, per §5 above. - ---- - -## 7. The short answer - -If exactly one thing is taken: **worktree-per-session (§3.1), plus the -deterministic finalisation phase (§3.2)**. Together they close FR-E11 by -construction and turn a Vogt work item from a card sitting next to a terminal -into a card that produces a pull request — which is the whole of what ProPR has -demonstrated works at volume. - -If a second: **cost accounting (§3.3)**, because it is the only item here that -can feed the ranking. diff --git a/docs/CONTRIBUTING.md b/docs/CONTRIBUTING.md index f0384783..e9e521a0 100644 --- a/docs/CONTRIBUTING.md +++ b/docs/CONTRIBUTING.md @@ -74,11 +74,10 @@ The supported public product is the Python core, Rust session engine, and Solid PWA together; the core remains independently supported over CLI, REST, and MCP. The engine and PWA have their own toolchains (`engine/AGENTS.md`, [`ENGINE.md`](ENGINE.md)) and image, and a core-only change need not require -those optional toolchains unless it changes their contracts. Read -[`opensource.md`](../opensource.md) before changing packaging or deployment -files — `tests/test_public_delivery.py` pins that the public example stays -self-contained, and a change that introduces a private path, registry, secret -broker, or external service into it will fail there. +those optional toolchains unless it changes their contracts. Before changing +packaging or deployment files, know that `tests/test_public_delivery.py` pins +the public example as self-contained: a change that introduces a private +path, registry, secret broker, or external service into it will fail there. Vogt is currently a single-maintainer project. Protected branches therefore require the test and policy checks, resolved review conversations, linear @@ -94,7 +93,7 @@ transport-parity rule, logging — are in [`AGENTS.md`](../AGENTS.md). ## Pull requests Describe the behavior change, the surfaces it affects, and the verification -commands run. If a change keeps a compatibility alias for a -historical identifier, explain its migration/removal plan and record it in -[`opensource.md`](../opensource.md) rather than hiding the alias in a generic -example. +commands run. Do not add compatibility aliases for historical identifiers: +no released installation depends on the legacy names still in the tree, and +removing them outright is preferred over extending them (see +[`ROADMAP.md`](ROADMAP.md) "Pending cleanup"). diff --git a/docs/CUSTOMISATION.md b/docs/CUSTOMISATION.md index d049623f..0e812b9a 100644 --- a/docs/CUSTOMISATION.md +++ b/docs/CUSTOMISATION.md @@ -260,9 +260,10 @@ model and request details. A fronted deployment needs a token the front door presents to the core, so audit rows name the actor who acted rather than "the proxy". The core is what -validates it — which used to mean it could only be *minted* by a running -core, and a first deploy went: start up, watch `/api/vogt` answer 401, exec -into the core, mint a token, paste it into your configuration, deploy again. +validates it — which, without a bootstrap path, would mean it could only be +*minted* by a running core, making a first deploy: start up, watch +`/api/vogt` answer 401, exec into the core, mint a token, paste it into your +configuration, deploy again. The second deploy is not free either: it restarts the pod and takes every open terminal session with it. @@ -430,7 +431,7 @@ tailnet and no maintainer integrations, so it runs on any host unchanged. The maintainer's own estate layers its host mounts, tailnet, and secret integrations on top of this same base. That overlay is not tracked in this -repository (#204) — a deployment tied to one operator's paths and addresses +repository — a deployment tied to one operator's paths and addresses does not belong in a public tree — and lives in the operator's private ops repository instead. Treat `engine.overlay.yml` as the pattern: every estate-specific value it would add is an environment value or a mount an diff --git a/docs/DEMO_SITE_PLAN.md b/docs/DEMO_SITE_PLAN.md deleted file mode 100644 index b9c691b9..00000000 --- a/docs/DEMO_SITE_PLAN.md +++ /dev/null @@ -1,470 +0,0 @@ -# Demo site implementation plan - -**Status:** implemented on `feat/demo-site`; acceptance evidence is in the -tests and delivery gates named below -**Target base:** `dev` -**Implementation base SHA:** `a2369cf6e286fbef1ce562a7d8be0b75bab57ffa` -**Reviewed:** 2026-08-24 -**Visual reference:** the PWA served by -[`vogt-dev.sprooty.com`](https://vogt-dev.sprooty.com/), from the matching -`dev` commit - -This is an implementation plan, not a second product specification. Built -architecture remains in [`DESIGN.md`](./DESIGN.md), delivery status remains in -[`ROADMAP.md`](./ROADMAP.md), and the issue tracker decides whether this work -is scheduled. - -## 1. Outcome - -Deliver a safe, resettable demo site that: - -- renders the **same Solid PWA artifact** as the `dev` live site rather than a - screenshot recreation or a second set of demo components; -- gives every shipped route, surface tab, navigation menu, picker and major - disclosure representative data; -- includes scripted terminal sessions in idle, running, waiting and exited - states; -- opens directly into representative two-pane and nested three-pane terminal - layouts, while leaving the existing split, retarget, maximise, broadcast, - detach and drag-to-split controls usable; -- lets visitors exercise writes against private, per-browser ephemeral state, - with no forge write, shell execution, shared mutation or secret; -- works at desktop and phone widths and makes its demo status unmistakable; -- can prove which `dev` commit its UI came from and cannot silently drift onto - a stale bundle. - -The primary acceptance statement is: - -> For one `dev` commit, build `web/dist/` once. The live-capable image and the -> demo image consume those exact hashed assets. Only the runtime transport and -> data differ; the rendered application components and CSS do not. - -The demo artifact also serves `mobile-demo.html`. It is a presentation shell, -not another application: its phone frame loads the same origin's PWA at the -responsive Sessions, terminal and Assistant routes. This matches the actual -Capacitor architecture, where the Android WebView loads the deployed PWA -directly; native-only push and microphone plumbing are stated as exclusions. -For a dedicated mobile hostname, the same static image may select the showcase -as its root document. Its frame names `/index.html` explicitly, so this mode -does not recurse and does not require a second build or image stream. - -## 2. Review findings - -### 2.1 The UI already has most of the necessary presentation - -- `web/src/routes.ts` declares 19 application routes: the six stable places, - terminal/editor/git/history/tasks/GUI/Assistant tools, setup, settings and - work-item detail. -- `web/src/App.tsx` owns the desktop Places rail, running-session menus, recent - places, Files, phone bottom navigation, the phone More sheet, Settings, - keyboard help and the command palette. -- `web/src/TerminalWorkspace.tsx` and `web/src/terminalLayout.ts` already - support horizontal and vertical splits, nested split trees, existing-session - composition, drag-to-split, pane retargeting, maximise/restore, broadcast, - detach and persistence in `vogt.terminalLayouts.v1`. -- The PWA already has meaningful sub-views that a demo must populate: Backlog - and Bugs; Audit trail and Notifications; project overview, dependencies, - drift and import; Git worktree/log/diff; History search/replay; task runs, - findings and gates; Assistant transcript and pending action; Settings - sections and saved workspace layouts. - -The demo should therefore provide data and transport behavior to the existing -components. It should not fork their markup. - -### 2.2 The browser test fixtures are useful prior art, not a deployable demo - -`web/tests/browser/gui.spec.ts` already intercepts a broad set of engine and -Vogt requests, and `page.routeWebSocket` proves a terminal can be fed an -ordered `snapshot-start` / binary output / `snapshot-done` sequence without a -PTY. The fixtures are intentionally per-test, incomplete and embedded in a -large test file. Shipping that harness would make the demo dependent on -Playwright and would leave many routes in their empty state. - -Extract reusable **fixture builders and protocol transcripts**, not the -Playwright interception layer. Tests and the demo runtime should consume the -same typed scenario catalogue through their own adapters. - -### 2.3 A real public PTY is the wrong demo mechanism - -The engine's real sessions are shell processes with writable input. Publishing -a shared token or bypassing auth to let anonymous visitors use them would turn -the demo into remote command execution. Running a real core with one shared -database would also let visitors change one another's board, audit and forge -state. - -Terminal output must therefore be simulated in the browser, and all mutable -demo data must be isolated per browser. Direct calls to the demo host's real -engine API must remain unauthorized. - -### 2.4 Artifact provenance must be part of parity - -At review time the deployed development site served the JS/CSS produced by -commit `aad55d0`, while `origin/dev` had advanced. That is normal for a -digest-pinned deployment, but it means “looks like dev” cannot be an informal -claim. The demo must expose its source commit and asset manifest, and the -deployment check must fail when the advertised commit and the served assets do -not agree. - -## 3. Architecture decision - -### 3.1 One PWA, two runtime transports - -Add a narrow transport seam beneath the existing API clients: - -```text -Solid surfaces - -> api.ts / vogtApi.ts / installApi.ts / push.ts - -> RuntimeTransport - |- NetworkTransport -> real HTTP + SSE + WebSocket - `- DemoTransport -> per-browser DemoStore + scripted streams -``` - -`NetworkTransport` preserves today's behavior. `DemoTransport` is selected -only when the origin serves a valid `/demo-manifest.json`. On ordinary Vogt -deployments that file is absent, so production takes the network path without -an operator setting or guessed hostname. - -The manifest should contain only non-secret presentation metadata: - -```json -{ - "schema": 1, - "enabled": true, - "source_ref": "dev", - "source_sha": "<40 hex characters>", - "scenario": "full-estate-v1" -} -``` - -Application boot must resolve this manifest before deciding whether a saved -credential exists. Demo mode installs a non-secret sentinel credential in the -same browser storage key the normal auth gate reads, then loads the demo -transport. A visitor may still choose **Sign out** to inspect the sign-in -surface; reload restores the disposable demo identity. - -No component should branch on demo mode to decide its normal content. The only -permitted UI differences are a small persistent **Demo data** disclosure, a -**Reset demo** action, and copy explaining that writes are private and -temporary. - -### 3.2 Consolidate every transport path - -The seam is incomplete if only `fetchWithRetry` uses it. Inventory and route -all of these through `RuntimeTransport`: - -- authenticated JSON and blob requests in `api.ts`; -- registry-backed `/api/vogt` calls in `vogtApi.ts`; -- install/bootstrap, push, voice and Assistant requests; -- the authorized SSE reader used for server events; -- terminal WebSocket attach; -- any remaining component-local `fetch` call. - -Use a small socket interface containing only what `Terminal.tsx` consumes -(`send`, `close`, `readyState`, `binaryType`, and event listeners) rather than -pretending a simulator is a browser-native `WebSocket` in every detail. - -### 3.3 Per-browser deterministic state - -`DemoStore` owns one versioned scenario. Seed it deterministically on first -load and persist it in `sessionStorage` so: - -- navigation and reload preserve the visitor's changes; -- a different browser/tab starts clean; -- no state crosses users or reaches a server; -- **Reset demo** replaces the store and relevant Vogt UI storage keys with the - canonical scenario. - -Use an explicit logical clock anchored to the scenario timestamp. Relative -ages should advance during a visit, but snapshots and tests must be able to -freeze the clock. IDs, cursors, hashes and timestamps must be stable. - -Writes should implement the real visible semantics where practical: transition -a card, add a comment, archive/snooze/restore Inbox rows, resolve drift, edit a -file, stage/unstage a change, create/rename/kill a simulated session, run a -simulated task, and approve/reject an Assistant action. Every reason field -remains required. Writes append an audit/event record in the demo store so the -Audit view visibly reconciles. - -Unsupported or dangerous actions must return a specific demo refusal, never a -fake success. This includes external forge mutation, push delivery, microphone -upload, GUI process launch and any attempt to execute an arbitrary command. - -### 3.4 Serve the demo without an engine or core - -Build `web/dist/` once from the chosen `dev` SHA. A demo-image stage copies the -same output, verifies the recorded hashes, adds `demo-manifest.json` plus a -static simulated GUI page, and serves it from a minimal static Node origin. -The demo target contains neither the Rust engine nor the Python core, making -the absence of a PTY or shared write path structural rather than configuration. -Run that server with: - -- no token, core URL, workspace or state mount; -- a read-only root filesystem and no Linux capabilities; -- Assistant, voice, FCM, GUI process launch and agent-auth integrations - absent rather than merely unconfigured. - -The browser never calls those real APIs in demo mode. A direct caller receives -a static 404 from every `/api/**` or `/mcp` path. Add an integration smoke that -attempts session creation and proves there is no backend route capable of -creating a PTY. - -The image may be published by CI, but it must not deploy itself. If the demo is -hosted on the estate, use a separate digest-pinned Komodo stack and an ops -repository change, following the same publish-versus-deploy separation as the -other Vogt stacks. - -## 4. Fixture coverage matrix - -The fixture is complete only when each row below has an automated visit and a -non-empty assertion. - -| Surface | Required representative data and states | -| --- | --- | -| Board | At least two projects, two workflows, all active workflow columns, mixed kinds/priorities/assignees/labels/initiatives, long content, one finished item, a movable card and one refused-transition scenario. | -| Backlog | Both Backlog and Bugs tabs, ranked rows with differing scores/trust/freshness, pagination/continuation, selection, ranking explanation and saved lenses. | -| Inbox | GitHub, drift, CI and agent sources; evidence and proposed change; source URL; work-item link; session link; active/archived/snoozed counts; batch actions and clear-all confirmation. | -| Projects | Populated registry, searchable list, project overview, coverage, compliance, dependencies, observations, drift proposals and forge repository import choices. Use fictional neutral names and paths. | -| Audit | Audit records for demo mutations, notifications from more than one reason/source, filters, continuation and freshness/coverage statements. | -| Sessions overview | Idle, running, waiting-for-input and exited sessions; templates; attention ordering; bookmark/menu actions; a pending exact-payload approval. | -| Terminal | Scripted scrollback, incremental output, resize/ping, input echo and canned command responses; two-pane and nested three-pane presets; create, compose existing, drag-to-split, retarget, broadcast, maximise and detach. | -| Editor/Files | Nested tree, recent files, modified markers, text files in several languages, a long Markdown document, a binary-file refusal, save and stale-write conflict. | -| Git | Repository chooser, branch list, staged/unstaged/untracked entries, recent commits, selected diff, and ephemeral stage/unstage/discard/commit/checkout behavior. | -| History | Several archived sessions, search hits, selected result, metadata, replay tail, truncation and pins. | -| Tasks | Scheduled and event-triggered tasks, running/completed/errored runs, work-item binding, findings, conclusion, checkpoint branches, an open gate and steering transcript. | -| GUI stream | A same-origin static simulated desktop/IDE frame with visible content; launch/kill actions explicitly refuse because no real process exists. | -| Assistant | Canned multi-turn transcript, Markdown/code reply, profiles, a pending terminal-input action and a pending reasoned Vogt write; typed follow-up produces deterministic replies. Speech controls explain that capture/playback is unavailable in the public demo. | -| Work item detail | Body Markdown, comments, audit link, observed evidence, branch/PR story, drift, related items, assignee/state edits, and linked current/exited sessions. | -| Setup | Linked-account state, repository picker, first-project choices and completed sweep/coverage data; writes mutate only the demo store. | -| Settings | Operational status, templates, app/terminal themes, auth profiles, push-subscription example, storage preferences, saved workspace layouts and reset/confirmation flows. | -| Global chrome | Non-zero place counts, waiting attention card, Running/Recent/Files sections, session row menus, phone More sheet, full command palette providers, keyboard-shortcut help, dialogs and feedback notifications. | - -### Terminal showcase presets - -Seed these stable session IDs and layouts: - -1. **Build + tests** — `demo-build` beside `demo-tests` in a 50/50 row split. - The build streams compilation progress; tests finish with a mixed but - non-sensitive summary. -2. **Agent review** — `demo-agent` on the left, with `demo-server` above - `demo-logs` on the right. The agent is waiting for input and demonstrates - the attention card and exact-payload response controls. -3. **Incident view** — three already-running sessions suitable for - broadcast/maximise/detach demonstrations. - -Preseed `vogt.tabs.v2`, `vogt.terminalLayouts.v1` and saved workspace-layout -records only when the demo store is new. Existing visitor state wins until -they choose Reset. Publish direct links for the overview and each showcase, -and make those links part of the smoke test. - -The simulated socket must follow the real attach ordering: - -1. `snapshot-start` with byte position; -2. zero or more binary scrollback chunks; -3. `snapshot-done`; -4. timed binary frames and activity events. - -It must also accept auth, resize, ping and input frames. Input is parsed only -enough to echo text and select a canned response. It never reaches a shell, -`eval`, subprocess, forge or network request. - -## 5. Delivery sequence - -### Phase 0 — freeze the parity boundary - -1. Record the target `dev` SHA and the deployed reference asset manifest. -2. Add a build step that emits `web/dist/demo-build.json` containing the source - SHA and hashes of every entry asset. -3. Define the only allowed artifact delta: `demo-manifest.json`, the simulated - GUI document and their static assets. Hashed PWA JS/CSS/font/worker files - must be byte-identical between normal and demo images for the same build. -4. Add a visible demo disclosure design that does not alter the primary - surface grammar or hide first-useful content at phone width. - -**Exit:** a test compares both image roots and rejects any unapproved PWA asset -delta. - -### Phase 1 — introduce the runtime transport seam - -1. Define network-neutral request, event-stream and terminal-socket interfaces. -2. Move existing behavior into `NetworkTransport` without changing request - URLs, auth handling, retry semantics or error classes. -3. Route every API client and direct transport call through the seam. -4. Run the current unit and browser suites against `NetworkTransport`. -5. Add a guard test that scans for unapproved direct `fetch`, `EventSource` and - `new WebSocket` usage outside the transport modules. - -**Exit:** production behavior and visual snapshots are unchanged; the demo -transport can be installed before `App` mounts. - -### Phase 2 — build the typed full-estate fixture - -1. Create small typed fixture factories grouped by domain rather than one - monolithic JSON file. -2. Add the logical clock, deterministic ID/cursor helpers and a state reducer. -3. Cover every engine request used by `api.ts` and every registry route used by - `vogtApi.ts`. -4. Add Python parity tests that compare the demo's Vogt operation list to the - real operation registry. A demo responder may omit a deliberately unused - operation only through a named exclusion with a reason. -5. Reuse fixture factories from Playwright where that reduces duplicate data; - do not make the shipped demo import test runner code. - -**Exit:** a contract test issues every supported request and validates the -response shape used by its surface. - -### Phase 3 — add terminal and live-event simulation - -1. Implement the scripted socket and attach-protocol transcripts. -2. Seed the three showcase layouts and existing saved-layout picker entries. -3. Publish activity, session lifecycle and `vogt-changed` events through the - simulated SSE stream so counts and visible views reconcile normally. -4. Implement safe canned terminal input, split-created simulated sessions and - cleanup behavior. -5. Test replay, incremental output, reconnect, multiple clients to one session, - nested layout reload and duplicate-session focus. - -**Exit:** direct showcase links render their intended split tree, remain usable -after reload, and no test observes a real network WebSocket or process spawn. - -### Phase 4 — populate and exercise every surface - -1. Implement the coverage matrix above, beginning with global shell providers - so rail counts and menus are truthful while individual routes land. -2. Add ephemeral mutation reducers and resulting audit/event records. -3. Add the demo disclosure, reset action and a copyable “start here” set of - links. -4. Test all sub-view controls, row menus, command-palette groups, phone More - links, Settings sections and destructive confirmations. -5. Ensure no fixture contains maintainer estate names, private paths, tokens, - real email addresses or tailnet/ops endpoints. - -**Exit:** the automated route sweep finds no accidental loading, unavailable, -empty or not-found state in the canonical scenario. - -### Phase 5 — visual, accessibility and responsive parity - -Run Chromium at minimum at: - -- 1440 × 900 desktop; -- 1280 × 900 desktop; -- 768 px breakpoint boundary; -- iPhone 13 viewport/touch profile. - -For every route and sub-view: - -1. assert the correct active rail/bottom-nav/tool state and document title; -2. assert meaningful content above the first viewport fold; -3. assert no unexpected horizontal document overflow; -4. open every menu/dialog/disclosure and verify focus return plus accessible - naming; -5. assert no console error, failed request or uncaught rejection; -6. capture the key screenshot set, including both terminal split presets; -7. run selected snapshots under dark and light themes without inventing - demo-only CSS. - -Use screenshots to catch composition regressions, but use asset identity and -shared components as the primary proof of parity. Do not maintain a second set -of screenshots copied from the live site as a design specification. - -**Exit:** the demo browser matrix and the existing PWA browser suite pass from -the same commit. - -### Phase 6 — package, deploy and verify - -1. Add a demo image target/Compose overlay with no workspace, core, integration - or credential mounts. -2. Extend CI on `dev` to build, scan and sign the demo image after the normal - web build; tag it by commit, never as a release. -3. Add a smoke that checks the root document, runtime manifest, source SHA, - asset hashes, canonical routes and direct terminal showcase links. -4. Add a security smoke that calls real session/files/git/GUI/task mutation - endpoints without the hidden engine token and expects refusal. -5. Deploy by digest through the operator-owned stack, not from CI. Record the - URL only after DNS/TLS/exposure and retention decisions are made. -6. Add a scheduled drift check: compare the demo's advertised SHA/asset - manifest with the intended deployed `dev` build and mark the demo stale - rather than claiming parity when it lags. - -**Exit:** a fresh browser can traverse the entire canonical scenario without a -credential, while direct API clients cannot create a process or change shared -state. - -## 6. Expected repository changes - -Names may change during implementation, but ownership should remain clear: - -```text -web/src/runtimeTransport.ts transport selection and shared interfaces -web/src/networkTransport.ts today's real HTTP/SSE/WS behavior -web/src/demo/manifest.ts validated runtime manifest and bootstrap -web/src/demo/fixtures/ typed deterministic scenario factories -web/src/demo/store.ts per-browser state and reducers -web/src/demo/transport.ts engine + Vogt request responder -web/src/demo/socket.ts scripted terminal attach implementation -web/src/demo/gui-stream.html same-origin simulated GUI content -web/src/demo/mobile-showcase.html phone frame around the same responsive PWA -web/tests/browser/demo.spec.ts complete route/menu/split matrix -deploy/demo.overlay.yml isolated demo runtime -deploy/demo.env.example non-secret operator choices -scripts/check_demo_assets.py normal/demo asset identity gate -docs/DEMO_SITE_PLAN.md this plan -``` - -Keep demo response types close to the existing API types. Do not introduce a -second domain model or a handwritten operation that the Python registry does -not know. - -## 7. Acceptance checklist - -### Parity and provenance - -- [x] Implementation is based on `dev` and records the exact source SHA. -- [x] Normal and demo images contain byte-identical hashed PWA assets from one - build. -- [x] The demo shows a persistent, accessible demo-data disclosure. -- [x] A stale or mismatched asset manifest fails verification and cannot claim - current parity. - -### Complete data - -- [x] Every entry in `APP_ROUTES` has a canonical populated scenario. -- [x] Every stable surface sub-view and Sessions tool is populated. -- [x] Desktop rail, session row menus, command palette, keyboard help, Settings - sections, phone bottom nav and phone More sheet have meaningful entries. -- [x] The route sweep reports no unintended empty/loading/unavailable state. - -### Terminals and layouts - -- [x] Four terminal activity/lifecycle states are visible. -- [x] Two-pane and nested three-pane layouts have stable direct links. -- [x] Existing split/retarget/maximise/broadcast/detach/drag controls work. -- [x] Socket replay and incremental frames obey the real protocol ordering. -- [x] Terminal input can produce only canned local responses. - -### Isolation and safety - -- [x] Demo mutations are private to one browser and resettable. -- [x] No real PTY, subprocess, workspace write, forge call, push, voice upload, - GUI process or external Assistant request can occur. -- [x] No engine/core/integration token is shipped to the browser. -- [x] Direct API mutation attempts reach no backend route. -- [x] Fixtures contain only fictional, non-sensitive identities and paths. - -### Quality and operations - -- [x] Existing PWA unit, typecheck, build and browser suites remain green. -- [x] Demo desktop/phone route and screenshot matrices are green. -- [x] The demo image is scanned/signed and deployable only by digest through the - approved deployment path. -- [x] A reset and a complete canonical tour work in a fresh browser. - -## 8. Cut lines - -Do not reduce the first release below complete route/sub-view data, the two -split-layout presets, per-browser isolation or provenance checks; those are the -reason for the demo site. - -Safe later increments are richer terminal transcripts, additional themes, -more fixture volume, narrated tours and optional animations. A marketing -landing page, analytics, lead capture, public shell access, live forge writes -and a standalone fork of the PWA are explicitly outside this plan. diff --git a/docs/DEPLOYMENT.md b/docs/DEPLOYMENT.md index 101a2304..c36a02c4 100644 --- a/docs/DEPLOYMENT.md +++ b/docs/DEPLOYMENT.md @@ -378,7 +378,7 @@ not change production by itself. The desired state a production instance runs — which digests, which overlays, which host specifics — is owned by the operator's own deployment repository, -not this one (#204); this tree ships only the estate-neutral base and overlays, +not this one; this tree ships only the estate-neutral base and overlays, never a turnkey production estate. Two consequences are worth stating plainly. The signed release digests are CLI-free (§3.2), so a production engine that runs agent sessions is one the operator built with diff --git a/docs/DESIGN.md b/docs/DESIGN.md index e4b9b5cb..c5d4bae8 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -1,21 +1,13 @@ # Vogt — Design Outline -Status: **v0.3 (revision r5), built**; **§1.2 reversed at r9** (2026-08-14 — -Vogt runs the work it governs) · design 2026-08-12, reconciled against the -delivered v1 on 2026-08-12, and against the merged product on 2026-08-15. -Scope: standalone product. Cadastre — the maintainer's earlier, private -infrastructure register — is prior art and a lessons source, not a -dependency; see §11. - -**This document describes what Vogt *is*.** That rule was tightened on -2026-08-15 and it changes how to read the sections below. Where the build -decided something differently from an earlier draft, the decision is described -in place and the withdrawn alternative is named so the absence reads as a +**This document describes what Vogt *is*.** Where the build decided +something differently from an earlier draft, the decision is described in +place and the withdrawn alternative is named so the absence reads as a choice. Where something was designed and **never delivered**, it is *not* -described here as though it existed: it is a numbered gap in the -requirements baseline's gap register, with what is missing and what its -absence costs. A design document that describes unbuilt things is the most -expensive kind of wrong, because it reads exactly like one that does not. +described here as though it existed: it belongs in +[`ROADMAP.md`](ROADMAP.md), not here. A design document that describes +unbuilt things is the most expensive kind of wrong, because it reads exactly +like one that does not. The numbered requirements baseline (`FR-*`/`NFR-*` identifiers, revision history, delivery verification and the gap register) is maintained outside @@ -48,8 +40,8 @@ work items, backlog, ranking, contracts, compliance, dependency references, drift, and audit all work against the filesystem and local git alone. Forge-optional is a statement about *dependency*, not about *sequencing*. -Read-only GitHub collectors ship at M2 alongside the local ones (r2 -decision, FR-O5a), because the estate this tool governs keeps most of its +Read-only GitHub collectors ship alongside the local ones (FR-O5a), +because the estate this tool governs keeps most of its real work — issues, PRs, CI results — on GitHub, and an MVP populated only by source markers would be a demo rather than a daily driver. What stays optional is the *dependency*: nothing in the core requires the adapter, the @@ -66,23 +58,22 @@ Consequences: forge adapter is configured. - CI status is modeled as generic `RevisionCheck`-style observations; the GitHub Actions collector is one producer. -- Write-back, historical backfill, and forge-derived drift remain at M5. +- Fuller write-back, historical backfill, and forge-derived drift are + designed but not built ([`ROADMAP.md`](ROADMAP.md)). -### 1.2 Non-goals (v1) *(one reversed at r9)* +### 1.2 Non-goals *(one reversed)* - Multi-forge support (GitHub + GitHub Actions are the only *optional* forge integration in v1). - Multi-node / hosted SaaS (single-node self-hosted only). - Time tracking, sprint ceremonies, burndown charts. - ~~Being an agent runner. Vogt tells agents *what* and *why*; it does - not execute them.~~ **Reversed at r9** — see below. The line stays because - it was true for v1, and because a reader of the delivered v1 will find - nothing in it that runs anything. + not execute them.~~ **Reversed** — see below. The line stays struck + rather than deleted so the reversal reads as a decision, not a drift. - Enforcing anything. Vogt reports; the human or agent acts (§5). -**The agent-runner reversal (r9).** Vogt now runs the work it governs. -A previously separate session-engine codebase is merged in as Vogt's -session engine (`engine/`, `web/`, `mobile/`), and with it come PTY +**The agent-runner reversal.** Vogt runs the work it governs. The +session engine (`engine/`, `web/`, `mobile/`) brings PTY sessions, agent tasks and an assistant, so a work item can *open a coding session* in its project's tree instead of only describing one. The design change is recorded here rather than in a deleted bullet because the original line was not a mistake: @@ -102,9 +93,7 @@ unchanged: an execution surface is not an enforcement surface, and nothing in the merged product consumes compliance, trust or drift status as a precondition for running anything. -The numbered form of all this is requirements revision **r9** (families -FR-E, FR-T, FR-M, and the appended FR-U/FR-S/NFR rows); the engine's own -reference is [`ENGINE.md`](ENGINE.md). +The engine's own reference is [`ENGINE.md`](ENGINE.md). ### 1.3 Alternatives considered @@ -116,7 +105,7 @@ this proposal must answer. | **GitHub Issues/Projects + an MCP shim** | Covers the write plane well and would delete a third of this design. It cannot express the parts that motivate the product: coverage-modelled observation ("has anything even looked at this repo lately?"), declared-vs-observed separation with typed drift, cross-project dependency references, contract compliance, or a ranked global view over repos that are *not* on GitHub. It also makes every answer network-bound and rate-limited, and puts the index of your own estate inside a service you do not run. | | **Jira / Linear / self-hosted alternatives (Plane, Taiga)** | Same write plane, none of the observation layer, and all of them assume work is *entered*. The problem Vogt exists for is the opposite: work already exists in the filesystem and on the forge and nobody has an index of it. Bending one of these into an observation platform is more work than the observation platform. | | **A pile of scripts + a Markdown index** | The honest baseline, and what exists today. It fails on freshness (no answer to "when was this last true"), on provenance, and on being usable by agents without bespoke parsing per script. | -| **Extend cadastre** | Rejected for domain reasons, see §11. | +| **Extend an existing infrastructure register** | Rejected for domain reasons, see §11. | What survives the comparison is the observation, coverage, trust, drift and cross-project layer — the write plane is table stakes that has to exist for @@ -127,7 +116,7 @@ outsourcing it to a forge. ## 2. Design principles (learnings applied) -Carried from cadastre (proven there): +Carried from a prior internal infrastructure register (proven there): | Principle | What it means here | |---|---| @@ -139,12 +128,12 @@ Carried from cadastre (proven there): | Checks return evidence | A contract check returns its result **with the failing rules named**, never a bare boolean. | | Zero-dep self-hosting | SQLite + forward-only migrations + migration lock. Backup/export/import are v1 commands, not v2. | -Inverted from cadastre (its anti-learnings for this domain): +Inverted from that register (its anti-learnings for this domain): -| Cadastre posture | Vogt posture | Why | +| Register posture | Vogt posture | Why | |---|---|---| -| Declaration-first: undeclared work is invisible | **Observed-first**: collected work is visible immediately; adoption upgrades trust | Cadastre's import blocked with 97% of items lacking declared repos; a tracker must never show an empty view of a busy estate | -| Absence in observations can read as drift | **Coverage is modeled explicitly**: "collector X last swept scope Y at T". Absence is only meaningful inside swept scope | Most cadastre "missing" drift was collector-coverage artifact | +| Declaration-first: undeclared work is invisible | **Observed-first**: collected work is visible immediately; adoption upgrades trust | Its import blocked with 97% of items lacking declared repos; a tracker must never show an empty view of a busy estate | +| Absence in observations can read as drift | **Coverage is modeled explicitly**: "collector X last swept scope Y at T". Absence is only meaningful inside swept scope | Most of its "missing" drift was collector-coverage artifact | | Read-only map, no write-back | **Owns the write plane**, incl. opt-in GitHub write-back | It's a tracker; creating/closing/moving work is the product | | No people model | Actors (humans *and* agents) are core entities from day one | Retrofitting assignment/attribution is miserable | | Hand-rolled ASGI | FastAPI | Free OpenAPI, validation, auth middleware; big API surface planned | @@ -348,12 +337,8 @@ Model: a project no sweep has walked (FR-O4). - Update-automation posture (version-updates config / vulnerability alerts / automated security fixes — three independent toggles, never one - boolean) is **forge posture**, not dependency data. It moves to the forge - module at M5. -- Roadmap fit: reference extraction and the cross-project graph land with - M2; `unresolved_dependency` reporting with M3; `mirrored_source` and the - scan record at r15, after an estate onboarding produced eighteen mirrored - crates by hand. + boolean) is **forge posture**, not dependency data, and is owned by the + forge module. ### 3.6 Observed-first without drowning (r2/r3) @@ -401,7 +386,7 @@ provider-matched `repo_url` plus a usable credential and refuses with the missing precondition named. On a **linked** project: - **The work items are the mirrored forge issues.** The observed mirror - (issues synced all-state by the M5 collectors) is the truth for title, + (issues synced all-state by the forge collectors) is the truth for title, body, labels and open/closed; the `work_overlay` table — keyed by the subject, not by a `wrk_*` id — carries the vogt-local half: a workflow state richer than open/closed, priority, effort, assignee, initiative, @@ -477,29 +462,26 @@ adapters/ cli/ thin argparse/typer over application http/ FastAPI: REST + generated OpenAPI; serves GUI API mcp/ MCP server (stdio + streamable HTTP); tool set mirrors REST - github/ OPTIONAL forge adapter plugin: collectors (read, from M2) + - write-back (opt-in write, M5); absent = fully functional + github/ OPTIONAL forge adapter plugin: collectors (read) + + write-back (opt-in write); absent = fully functional collectors/ plugin registry — core (no network): git-local, source-markers, contract-checker, dep-refs; optional (network): gh-issues, gh-prs, gh-actions, gh-releases, session-outcomes (registered only when an engine is configured; FR-E6, FR-E7) -gui/ static ES modules consuming the HTTP adapter only (M6 chose - buildless over React — see `ROADMAP.md` M6; a wheel that - needs npm to build is the cost that decided it) storage/ SQLite x2 (declared.sqlite3, observed.sqlite3), migrations ``` -**The merged tree, from r9.** The repository also carries the session engine +**The merged tree.** The repository also carries the session engine and the front ends it brought, and the layer rules above still describe the Python core rather than the whole product: ``` engine/ the Rust session engine (server + contract crates): PTYs, scrollback, activity, agent tasks, push, the assistant — - and, from M9, the front door: the only listening process, + and the front door: the only listening process, proxying /api/vogt and /mcp to vogt-core on loopback -web/ the Solid PWA, the product's front end from M11; consumes +web/ the Solid PWA, the product's front end; consumes the engine's API and, through /api/vogt, the same public operations the CLI and MCP see. One route model supplies the desktop rail, phone bar and Sessions tools with their current @@ -518,16 +500,15 @@ src/vogt/ unchanged, and still the only definition of an operation The engine's own reference — what it owns, how to run it, its full wire contract, the assistant and the agent-task scheduler — is -[`ENGINE.md`](ENGINE.md). It is one document because it used to be eight, each -describing the engine as a separate product. +[`ENGINE.md`](ENGINE.md). Two properties hold the shape together, and both are asserted rather than described. **The registry is still the single definition**: the PWA's route table resolves against it, and so do the assistant's Vogt tools, which are fetched from the core's own MCP `tools/list` rather than written out again. -**The core is still complete alone**: it serves its own port, its own GUI -and its own MCP when no engine is present, and CI runs the suite with -`engine/`, `web/` and `mobile/` deleted to keep that true (NFR-Q6). +**The core is still complete alone**: it serves its own port and its own +MCP when no engine is present, and CI runs the suite with `engine/`, `web/` +and `mobile/` deleted to keep that true (NFR-Q6). The direction of dependency is the thing to preserve. The engine calls the core, and the core calls the engine only for sessions — four operations @@ -541,7 +522,7 @@ instead of one language argument. `import`, `mcp.stdio` — each acting on the local process or data directory with no meaningful remote semantics, and each carrying its reason as the dictionary's value rather than in a comment. `HTTP_ONLY` is **empty**, and - has been since M4: no operation has yet earned an exception in that + always has been: no operation has yet earned an exception in that direction. It stays declared because a list that only exists once something needs it is a list nobody adds to correctly. - **Write-back** lives only in `adapters/github/` behind a per-project @@ -572,11 +553,11 @@ REST/CLI/GUI are peers over the same operations. - **One transport-neutral operation registry.** Every operation is defined once — name, scope, `mutating` flag, argument schema, HTTP route — and the MCP tool list, FastAPI routes, CLI commands, *and the stdio bridge* - are all generated from that registry. Cadastre's biggest MCP duplication - was the same 20 tool signatures hand-mirrored across its server, its - remote bridge, and its registry; here the bridge is generated, never - hand-written. Parity exclusions are named lists that fail when stale, - never glob matches. + are all generated from that registry. The prior register's biggest MCP + duplication was the same 20 tool signatures hand-mirrored across its + server, its remote bridge, and its registry; here the bridge is + generated, never hand-written. Parity exclusions are named lists that + fail when stale, never glob matches. - **Identity is never a tool argument.** The principal is derived from authentication (token, mTLS, trusted proxy, or `local:`) — a caller-supplied principal would let any token forge provenance. @@ -910,10 +891,10 @@ Deployment: [`DEPLOYMENT.md`](DEPLOYMENT.md). - Transport-parity test matrix with named exclusion lists (§4). - Coverage gate in CI (start ≥80%). - Forward-only migrations, tested against fixture databases. -- A seeded benchmark fixture at the NFR-S1 envelope from M2, with the +- A seeded benchmark fixture at the NFR-S1 envelope, with the interactive-query target asserted in CI — the two-store split means every aggregate query is an application-layer join, and that cost needs a - tripwire rather than a discovery at M6. + tripwire rather than a late discovery. - Every feature lands with CLI + REST + MCP + audit coverage or it doesn't land. - **Own dependencies practice what we preach**: `uv.lock` committed, @@ -953,111 +934,44 @@ Deployment: [`DEPLOYMENT.md`](DEPLOYMENT.md). --- -## 9. Open questions - -Resolved 2026-08-12 (r1): - -- **Licence: MIT**. Developed in a private repository under - `TheDancingDeveloper-org` first; public at a milestone of the owner's - choosing (NFR-O1). -- **Name: Vogt** (final) — the German reeve/bailiff who oversaw an estate - (Vogtei), enforced its rules, and answered for its work; the - counterpart of a cadastre (a land register). Package/CLI `vogt`, env - prefix `VOGT_*`. -- Drift autonomy → low-risk auto-accept defaults (§3.2). -- Notifications → audit-backed `/events` feed, no push in v1 (§4.2). -- M4 auth → **token-only**: scoped bearer tokens bound to actors, - issued/rotated via CLI, one mechanism for GUI/REST/MCP; OIDC may layer - on later. -- Attachments → **deferred past MVP**; schema leaves room, bug evidence - meanwhile lives as paths/URLs in body text. -- **M5 (GitHub write-back) before M6 (GUI)** — build the GUI once against - complete data. This ordering is fixed, not swappable. - -Resolved 2026-08-12 (r2, after whole-proposal review): - -- Gates → **status, not enforcement** (§2.1, §5); `~/WorkingStack` and - `project migrate` withdrawn (§5.1). -- Dependency tracking → **references, not resolved versions** (§3.5). -- MVP line → **M0–M2**, with read-only GitHub collectors pulled into M2 - (§1.1). -- Observed-first → **promotion by convention + suppression + root - exclusions** (§3.6). -- `/events` cursor → **single events table in the declared store** (§4.2). -- Drift evidence → **snapshot at raise time + pinned against retention** - (§3.2). -- Project granularity → **one repo/folder; workspace members are not - projects** (§3.1). - -Resolved 2026-08-12 (r3): - -- **No discovery, no continuous checking** (§2.1, §5, §5.1). Collection - scope is the registered project list; the contract is evaluated on - demand and reported with its age. FR-G5–G8 deferred. -- AI-assisted drift detection and recommendation is recorded as a - **non-committed stretch goal** — the reason the scheduler stays small, - and something no v1 requirement may lean on. - -Resolved 2026-08-12 (the last of the open questions): +## 9. Standing decisions +Positions that shape the product but do not belong to any one section above. +Everything else that was once an open question is decided and described in +place in the section that owns it. + +- **Licence: MIT.** +- **Name: Vogt** — the German reeve/bailiff who oversaw an estate (Vogtei), + enforced its rules, and answered for its work; the counterpart of a + cadastre (a land register). Package/CLI `vogt`, env prefix `VOGT_*`. - **Comment write-back is outbound only** (FR-B5). Comments authored in Vogt post upstream under `comment_only`/`full`; inbound forge comments stay observations against the linked item. Mirroring both ways would need forge-author identity mapping and loop suppression for our own writes, and buys little the observation view doesn't already give. -- **No manual ranking override.** `rank_order` is dropped from - `work_items`; ordering is computed from documented weights and stays - fully explainable, with `priority` and initiative weight as the - hand-set inputs that already feed the score (§3.4). -- **Trust is `disputed`, drift resolution is `contested`** (§6). The - computed vocabulary and the chosen vocabulary no longer collide. -- **Cadastre: accept the duplication for v1** (§11). - -Resolved 2026-08-12 (r4): - -- **Deployment target: a Compose stack running the GHCR image, digest-pinned, - bound to a private-network address** ([`DEPLOYMENT.md`](DEPLOYMENT.md)). - The desired state (compose file plus pinned digest) lives in version - control, not on the host. -- **TLS may terminate in-process** (NFR-D6 revised). A listener that is - only reachable on a private network and already holds a certificate - gains nothing from a reverse proxy in front of it; a proxy remains the - right answer where it is already the host's ingress. -- **"No default port anywhere" was too broad** (NFR-D2 revised). Defaults - that encode exposure or identity (a public hostname, a bind to all - interfaces, an endpoint URL) stay forbidden; defaults that are pure host - allocation (the listen port) are now *required*, because a `${X:?}` gate +- **TLS may terminate in-process** (NFR-D6). A listener that is only + reachable on a private network and already holds a certificate gains + nothing from a reverse proxy in front of it; a proxy remains the right + answer where it is already the host's ingress. +- **Exposure values carry no defaults; allocation values do** (NFR-D2). + Defaults that encode exposure or identity (a public hostname, a bind to + all interfaces, an endpoint URL) are forbidden; defaults that are pure + host allocation (the listen port) are required, because a `${X:?}` gate on an allocation value turns every deploy into a hunt for a number nobody cares about. The distinction is what the value decides, not whether it is a number. - **Publish and deploy are separate acts** (NFR-D10). A tag publishes a signed image; production moves only when a human or agent bumps the - pinned digest and redeploys the stack. Automating that bump is not v1 - scope. - -Nothing is open. New questions get appended here as they arise. + pinned digest and redeploys the stack. --- ## 10. Roadmap -Defined in [`ROADMAP.md`](ROADMAP.md): seven stages (M0 Foundation → M6 -GUI) with per-stage requirement IDs and demo acceptance criteria. -**MVP = M0–M2** (r2): a daily-usable tracker with observed-first views over -the local estate *and* read-only GitHub. M3 adds compliance reporting and -the drift lifecycle, M4 service mode, M5 forge consolidation and -write-back, M6 the GUI. **v1 = M0–M6.** - -Post-v1: M7 (import and the notification inbox) and M8 (`connect` — reaching -an instance from an agent environment). The merge stages **M9–M14** carry the -§1.2 reversal into delivery — foundations, coding sessions, GUI uplift, the -AI layer, mobile, consolidation. **v2 = M9–M14**, M14 being the consolidation -stage rather than a feature one. - -Two of those stages end in a demo that has not been run, because neither can be -run without a browser and a phone: M11's and M13's. `ROADMAP.md` says so at -each stage and the gap register carries them, so that "built" nowhere -quietly means "watched working". +The product described here is built and shipping. What remains designed but +unbuilt, and what is deliberately deferred, is listed in +[`ROADMAP.md`](ROADMAP.md); the live work queue is the GitHub issue +tracker. ### 10.1 Public demo artifact @@ -1082,36 +996,16 @@ browser showcase. The static origin can select that document as `/` through an allowlisted `DEMO_ROOT_DOCUMENT`; the mobile deployment overlay uses this for a separate hostname while retaining `/index.html` as the framed PWA. Both entry points therefore pin one signed image digest and cannot drift apart. -[`DEMO_SITE_PLAN.md`](DEMO_SITE_PLAN.md) records the complete implementation -and acceptance rationale. --- -## 11. Relationship to cadastre - -Cadastre (the maintainer's private land register — infrastructure, hosts, -services) and Vogt (the reeve — product work, backlog, project health) are -separate products with separate domains and separate stores. **Vogt does not -import from cadastre and does not couple to its API in v1**; nothing in -this repository needs it to exist. - -The cost of that decision is real and should be stated rather than -discovered: the declared/observed split, coverage modelling, trust -computation, the audit spine, the operation registry, the transport-parity -harness and the MCP stdio bridge are all being written a second time, and -will drift apart under two maintenance loads. - -**Decision (2026-08-12):** accept the duplication for v1 and re-converge -afterwards. Building Vogt against a shared kernel now would -couple two designs while one of them is still being learned, and the r2 -review of this proposal is itself evidence that Vogt's requirements are -still moving. Instead: keep the shared concepts *named* identically across -both codebases (`declared`/`observed`, `sweep`, `coverage`, `trust_state`, -`drift_proposal`, operation registry), so that extracting a common package -after v1 is a mechanical refactor rather than a redesign. Revisit at v1 -with two working implementations to compare. - -The related open question — whether an infrastructure register eventually -feeds *in* as a collector, adding infrastructure context to a project's -brief — stays open, and would be an optional, read-only external -integration if it happens. It is not a dependency in either direction. +## 11. Relationship to infrastructure registers + +An infrastructure register (hosts, services, credentials — Cadastre is one +such product) and Vogt (product work, backlog, project health) are separate +domains with separate stores. **Vogt does not import from or couple to any +infrastructure register**; nothing in this repository needs one to exist. +Whether such a register eventually feeds *in* as a collector — adding +infrastructure context to a project's brief — stays an open question, and +would be an optional, read-only external integration if it happens. It is +not a dependency in either direction. diff --git a/docs/ENGINE.md b/docs/ENGINE.md index f9dca304..0fe76553 100644 --- a/docs/ENGINE.md +++ b/docs/ENGINE.md @@ -1,13 +1,12 @@ # Vogt — The Session Engine -Status: **built and current as of 2026-08-15** · this document describes what -the engine *is*, not what it was planned to be. Where a capability was designed -and never delivered, it is not described here; outstanding engine work is -tracked in the GitHub issue tracker and [`ROADMAP.md`](ROADMAP.md). - -The session engine is the Rust half of Vogt. It began life as a standalone -product and was merged into this repository to become Vogt's execution -surface. It runs PTYs, streams them over WebSocket, serves the PWA, and is the +This document describes what the engine *is*, not what it was planned to be. +Where a capability was designed and never delivered, it is not described +here; outstanding engine work is tracked in the GitHub issue tracker and +[`ROADMAP.md`](ROADMAP.md). + +The session engine is the Rust half of Vogt — its execution surface. It runs +PTYs, streams them over WebSocket, serves the PWA, and is the merged product's **front door**: the only listening process, proxying `/api/vogt` and `/mcp` to the Python core on loopback. @@ -26,8 +25,7 @@ of the repository; each rule they label is stated in words beside them. Companion documents: [`DESIGN.md`](DESIGN.md) (the product's architecture), [`DEPLOYMENT.md`](DEPLOYMENT.md) (production deployment: images, compose, env, reverse proxy, backups, upgrades), [`USER_GUIDE.md`](USER_GUIDE.md) (how -a person drives it), [`VOICE_DELIVERY.md`](VOICE_DELIVERY.md) (the voice -assistant's delivery status). +a person drives it). --- @@ -83,13 +81,12 @@ The engine's checks run in `.github/workflows/ci.yml` and its image is built by `.github/workflows/build.yml`. Crate names, binary and helper names (`mydevenv2-*`), and the legacy -`MYDEVENV2_*` environment prefix still carry the engine's pre-merge name. -That is deliberate and is not drift: the names divide by *process*, not by -product, and renaming them is a migration on every live deployment. The same -compatibility rule keeps the Android application and notification channel -IDs, browser storage/event keys, and the `MYDEVENV2_NOTIFY:` task hook; none -is presentation copy. Browser/route titles, install labels, login/errors, -notification channel labels and notification content all use **Vogt**. +`MYDEVENV2_*` environment prefix still carry a legacy internal name, as do +the Android notification channel ID, browser storage/event keys, and the +`MYDEVENV2_NOTIFY:` task hook. None is presentation copy, and their removal +is pending housekeeping ([`ROADMAP.md`](ROADMAP.md)). Browser/route titles, +install labels, login/errors, notification channel labels and notification +content all use **Vogt**. Engine settings are read under `ENGINE_*` (see §3); the legacy `MYDEVENV2_*` names are still accepted as aliases and log a warning at startup. @@ -375,9 +372,8 @@ is wrong. ### Contract crate -Rust DTOs live in `engine/contract/` (`vogt-engine-contract`). They were shared -with the archived native client, which the merge left behind, so the server is -now the only consumer in this tree. Those types cover: +Rust DTOs live in `engine/contract/` (`vogt-engine-contract`); the server is +their only consumer in this tree. Those types cover: - session lifecycle payloads - SSE event payloads @@ -387,8 +383,7 @@ now the only consumer in this tree. Those types cover: The browser client still carries TypeScript mirrors in `web/src/api.ts`, but those shapes should follow the shared Rust contract instead of ad hoc -server-local structs. The browser/PWA is the supported client surface; the old -native desktop client remains deprecated legacy code. +server-local structs. The browser/PWA is the supported client surface. Routes whose response shape is named below without a crate to look it up in are server-local: the handler's own struct in `engine/server/src/`, named in the @@ -1606,9 +1601,9 @@ become instructions. One clause of the assistant's requirements is short, and it is not described above as though it existed: FR-T5's spoken validation pass of the recogniser -against domain vocabulary (project slugs, `WI-n` ids) has not been run by a -person — [`VOICE_DELIVERY.md`](VOICE_DELIVERY.md) tracks it. FR-T7's native -Anthropic backend was the other, and was deferred at r12 rather than left +against domain vocabulary (project slugs, `WI-n` ids) is a device-dependent +pass run per release rather than in CI ([`ROADMAP.md`](ROADMAP.md)). FR-T7's +native Anthropic backend was the other, and was deferred rather than left owed — the provider section above. ### Voice @@ -1838,8 +1833,7 @@ Named here so the absences read as decisions rather than omissions. is: read scrollback, read a curated slice of Vogt, and — after an on-screen approval — type into a PTY or make one of four Vogt writes as the approving user. -- **There is no native desktop client.** The engine's pre-merge repository - had one; it was deprecated and not carried across. A reference to `client/` +- **There is no native desktop client.** A reference to `client/` anywhere in this tree names something that is not here. - **Nothing here is a second backlog.** The engine's outstanding work lives in the GitHub issue tracker and `ROADMAP.md`, because an item without a diff --git a/docs/FORGE_FRONTEND_DESIGN.md b/docs/FORGE_FRONTEND_DESIGN.md deleted file mode 100644 index f95ea9bc..00000000 --- a/docs/FORGE_FRONTEND_DESIGN.md +++ /dev/null @@ -1,269 +0,0 @@ -# Forge-frontend implementation design (#181, #182, #183) - -_Execution design for the second half of the #178 pivot. #179 (per-actor PAT -linking) and #180 (repo picker + parity-gated import) land as the additive -foundation; the three issues designed here change the **work model itself** and -carry a schema migration of the live declared store, so they are sequenced as a -deliberate, separately-reviewed step rather than folded into the foundation -deploy. This document is the plan of record for that step; delete it once #183 -closes and the decisions have moved into the requirements register and -`DESIGN.md`. The requirements register (`FR-*`/`NFR-*` IDs) is maintained by -the maintainer outside the repository, in the git-ignored `docs/local/`; IDs -below are stable identifiers, and the rule each one carries is stated in -words beside it._ - -**Predecessors already delivered:** the provider seam (#172), incremental -all-state sync + watermarks + `last_confirmed` (#173), symmetric drift (#174), -the provider write surface — `create/comment/label/close/reopen` (#175), and the -foundation (#179 linking, #180 import). - -**Design source:** issue #178, decisions 1, 2, 7, 8, 9, 10. Read it first; this -document translates those decisions into the concrete code changes, grounded in -the current tree. - ---- - -## 1. What exists today (the starting point) - -Vogt today runs a **dual work model** — this is exactly the thing #181 removes. - -- **Declared work items** live in the declared SQLite store as `WorkItem` - rows (`src/vogt/core/entities.py`), `wrk_*` ids, a human `ref` (`WI-N`), an - `origin` of `"created"` (native) or `"adopted"` (promoted from an - observation). `work.create` (`src/vogt/application/services/work.py:46`) - inserts a native row and writes nothing upstream. -- **Observed subjects** live in the *observed* store (`observed.py`): forge - issues/PRs keyed by subject key `gh:{owner}/{repo}#{n}`, maintained by the - sync collectors with watermarks and `last_confirmed_at`. -- **Adoption** (`observed_first.py:132`) is the one bridge: it materialises a - declared `WorkItem` (`origin="adopted"`) plus a `WorkLink` - (`entities.py:203`) back to the observed `subject_key`. -- **Write-back** (`work.py` → `writeback.attempt`) speaks upstream **only for a - linked subject** (`_linked_subject`, `work.py:377`): a `transition` into a - terminal state closes, out of one reopens, a `comment` posts. Everything else - is local. After #179 the identity is the acting actor's PAT, else the FR-S7 - file token. -- **The Backlog** (`views.py` `_gather`) merges declared + observed → the ~159. - **The Board** (`board.py`) reads declared only → the ~2. #187's interim fix - makes that gap explicit; **#181 is the real convergence.** - -So the machinery the pivot needs already exists in pieces — adoption already -turns an observation into a work item with a subject-key link, and write-back -already speaks to linked subjects. #181 makes that path the *only* path on a -linked project and adds **create-through** (today `work.create` never creates -upstream). - ---- - -## 2. #181 — upstream-truth work items + local overlay (the big one) - -Decisions 1, 2, 9, 10. - -### 2.1 Identity - -On a **forge-linked** project, `vogt id = forge subject key` -(`gh:{owner}/{repo}#{n}`). Native standalone work items cease to exist there. -A project is "linked" when it has a `repo_url` a registered provider matches -**and** a usable credential (per-actor PAT from #179, or the FR-S7 file token) — -reuse `adapters/forge/registry.provider_for` / `has_configured_forge`; do not -invent a second notion of "linked". - -Introduce a **project link state** rather than inferring it every call: -`unlinked | linked`, persisted on the project (new column, migration below). -`publish` (#182) and `import` (#180) set it to `linked`. - -### 2.2 The overlay table (decision 2 — invisible upstream) - -A new declared table `work_overlay`, keyed to the **upstream subject key**, not -to a `wrk_*` id. It carries only vogt-only semantics that must never cross the -boundary: - -``` -work_overlay( - subject_key TEXT NOT NULL, -- gh:{owner}/{repo}#{n} - project_id TEXT NOT NULL REFERENCES projects(id), - rank REAL, -- vogt-local ordering - workflow_state TEXT, -- richer than open/closed - -- relations live in the existing relations table, re-keyed to subject - -- audit stays in the existing audit trail - created_at TEXT NOT NULL, - updated_at TEXT NOT NULL, - PRIMARY KEY (subject_key) -) -``` - -The upstream issue (already mirrored in the observed store) is the source of -truth for title/body/labels/open-closed. The overlay is *additive* local state. -A read of a linked project's work item **joins** the observed mirror (truth) to -the overlay (local semantics). This is the storage stance of decision 9: a -**synced mirror with upstream authority + write-through**, not a live proxy — -the observed store already is that mirror. - -> **Migration discipline.** This is a schema change to the declared store; pin -> the migration id before landing (the migration-identity CI guard). It is the -> reason this work is deferred out of the foundation deploy — a redeploy runs -> declared migrations against the live DB, and an instance's existing native -> work items (few on the maintainer's own instance, but real) must be handled -> by the migration, not stranded. The migration must decide those 2 native -> items' fate the same way #183's link-migration does (publish upstream, then -> re-key) — so **#181's migration and #183 are the same event** for already- -> linked projects and must be designed together. - -### 2.3 Write-through (decision 9) - -`work.create` / `work.comment` / label / open-close on a **linked** project -write through to the forge via the provider (`adapters/forge` write surface -from #175), then reconcile the local overlay + observed mirror: - -- `work.create` → `provider.create_issue(...)` → returns the new issue number → - the vogt id is the subject key → insert the overlay row. The returned - `WorkResult.item.ref` becomes the subject key (or a subject-key-derived ref). -- `work.comment` → `provider.comment(...)` (already wired in `work.py:348`). -- label add/remove → `provider.add_labels` / provider label set. -- `transition` into/out of a terminal state → `close`/`reopen` (already wired, - `work.py:230`). -- **Overlay-only** changes — rank, vogt-only workflow states beyond - open/closed, relations — produce **no** upstream write (decision 2). This is - the invariant the tests must pin: re-ranking or a vogt-only state change must - assert zero provider calls. - -**Failure semantics (decision 9): fail loud.** Write-through failure is a typed -error, no silent queueing. Reuse `WriteBackResult`'s `failed` outcome but for -`create` (the new verb) surface it as a raised typed error so the caller learns -the issue was *not* created — a queued/eventually-consistent success is -forbidden. - -### 2.4 Unlinked projects (decision 10) - -`work.create` and the other write verbs on an **unlinked** project return a -**typed error** telling the caller to link (#180) or publish (#182). Add a -`NotLinked`-style error to `src/vogt/errors.py`. Unlinked projects show **no** -backlog/work surfaces — that surface withdrawal is #183. - -### 2.5 Registry / parity - -Every changed or new operation stays registry-defined -(`registry/operations.py`); CLI/REST/MCP parity is generated. No hand-added -endpoints. The parity test that already exists must stay green with the changed -`work.*` semantics. - -### 2.6 Acceptance (from #181) - -- Creating a work item on a linked project produces a GitHub issue **and** a - keyed overlay row; the returned vogt id is the subject key. -- A vogt-only state change or re-rank produces **no** upstream write. -- A write-through failure surfaces a typed error, not a queued success. -- `work.create` on an unlinked project returns the typed link-or-publish error. - ---- - -## 3. #182 — the publish verb (create repo + push) - -Decision 8. **After** #181's model exists. - -New registry op `forge.publish`: create a remote repo under the linked actor's -PAT (#179) and push the local default branch. - -- **This is the first destructive-capable verb** beyond the deliberately - non-destructive write-back set (FR-B4: `comment/create_issue/add_labels/set_state`, - never force, never delete). It - *creates upstream state and pushes commits*. It therefore needs its own - requirements revision with an FR-ID and a bounded rationale: - - refuses if the remote already exists (typed refusal, never a clobber); - - **never force-pushes, ever** (mirror FR-B4's "no force" invariant into the - push path — `git push` without `--force`, and refuse a non-fast-forward); - - requires a clean/explicit local state (reuse #180's parity-gate helper). -- Provider surface: add `create_repo(name, private, ...)` to the `ForgeProvider` - write protocol and the GitHub implementation; keep it provider-agnostic like - the rest of `adapters/forge/writeback.py`. -- After publish, the project becomes `linked` and upstream-truth (hand to - #181's model); its open native items migrate via #183. -- Parity: CLI/REST/MCP from the registry. - -### Acceptance (from #182) - -- Publish creates the named repo under the linked actor and pushes the default - branch; a naming/existing-repo conflict is a typed refusal, not a clobber. -- No force-push, ever. -- Documented in the requirements register with its FR-ID + bounded - rationale; DESIGN describes it only once built. -- A published project is thereafter linked/upstream-truth. - ---- - -## 4. #183 — native-item migration on link + forge-less guarantee withdrawal - -Decision 7. Shares its migration event with #181 §2.2 for already-linked -projects. - -- **Migration on link/publish.** Enumerate a project's **open** native work - items; publish each as a GitHub issue (`provider.create_issue` + labels to - carry what maps), then re-key to the subject key and fold the vogt-only fields - into the overlay. **No open native item is silently dropped.** Closed/archived - native items: default **leave historical, migrate only open** — record the - choice in the requirements register. -- **Guarantee withdrawal (requirements).** A new revision withdraws the - "forge-less layer stays real" guarantee. The designed-but-now-withdrawn - capability is noted in the requirements register's gap section (§7), - **not** DESIGN.md (which describes only what exists). Cross-reference #178. -- **Surface change.** Unlinked projects show **no** backlog/work surfaces — - instead a **link/publish CTA**. This is the web counterpart to #181 §2.4's - typed error. -- **Test repurposing.** The existing forge-less honesty tests (which assert a - real forge-less work layer) are **repurposed** into link/publish-CTA tests: an - unlinked project asserts the typed link-or-publish error (#181) and the CTA - surface, not a working native backlog. Grep the suite for those honesty tests - before writing new ones — they are the tests to change, not delete. - -### Acceptance (from #183) - -- Linking/publishing a project with N open native items creates N upstream - issues, re-keyed and overlaid; no native open item silently dropped. -- The requirements register carries the withdrawal revision + §7 gap entry; the - former honesty tests now assert the CTA/typed-error behaviour and pass. -- An unlinked project renders the link/publish CTA and no backlog. - ---- - -## 5. Sequencing and the deploy boundary - -``` -#179 linking ─┐ -#180 import ─┼─► (foundation: additive, no work-model migration) ─► deploy A (foundation) -#187 honesty ─┘ - -#181 model + migration ─┐ -#183 native migration ─┼─► (one migration event; live declared DB) ─► deploy B (deliberate) -#182 publish ─┘ -``` - -- **Deploy A** carries the foundation only. `#187` - keeps the Board honest until convergence. -- **Deploy B** is the work-model migration. It must: pin its migration id; back - up the declared store first (`vogt backup` covers it); migrate every live - native item via the #183 path; and be walked against a restore - rehearsal before firing, because it rewrites the live work store. This is why - it is not bundled with A. - -## 6. Risks / watch-items - -- **The migration is the whole risk.** Everything else is additive. Rehearse the - declared-store migration on a copy of the live DB before Deploy B. -- **Write-through latency** on the interactive path (`work.create` now does a - network round-trip). Keep it synchronous and fail-loud (decision 9), but the - PWA must show the round-trip state. -- **Ranking cost.** The overlay's `rank` must not require re-gathering the whole - observed set on every write; key the overlay by subject and update in place. -- **Subject-key as `ref`.** Callers and tests assume `WI-N`-style refs; decide - whether the subject key becomes the ref or a subject-keyed ref is derived, and - do it once, centrally, so the CLI/REST/MCP surfaces agree. - -## 7. Definition of done for the deferred step - -- #181, #182, #183 acceptance boxes all green. -- One declared migration, id-pinned, rehearsed on a live-DB copy. -- Requirements revisions: upstream-truth model; the publish verb + FR-ID; the - forge-less guarantee withdrawal + §7 gap entry. -- DESIGN.md updated to describe only what now exists. -- The Board and Backlog converge (closing #187 for real); its interim banner is - removed. diff --git a/docs/GETTING_STARTED.md b/docs/GETTING_STARTED.md index 1d01c39c..8cb29647 100644 --- a/docs/GETTING_STARTED.md +++ b/docs/GETTING_STARTED.md @@ -362,5 +362,5 @@ This removes the example's named data volume and cannot be undone by Docker. reverse proxy and TLS, digest pinning, backups, upgrades, and running the optional engine next to the core. - [`docs/ENGINE.md`](ENGINE.md) is the optional session engine's reference. -- [`opensource.md`](../opensource.md) states the public boundary: what is - supported, what is optional, and which legacy names remain as aliases. +- [`docs/DESIGN.md`](DESIGN.md) states the architecture and the product + boundary: what the stack is, and which pieces are optional. diff --git a/docs/IMPORT-PLAYBOOK.md b/docs/IMPORT-PLAYBOOK.md deleted file mode 100644 index 953bd8ab..00000000 --- a/docs/IMPORT-PLAYBOOK.md +++ /dev/null @@ -1,247 +0,0 @@ -# Importing a project into Vogt - -A repeatable procedure an AI agent can run, unattended, to review a repository -and bring it under Vogt as the single source of truth for its work. - -It is written as phases. Each phase states what to run, what a good answer -looks like, and what to do when the answer is bad. Phases 0–3 are read-only; -the first write is in phase 4. **Stop and ask a human** wherever a step says -DECISION — those are the points where guessing produces a project record that -is wrong in a way later sweeps will not correct. - -The worked example throughout is `rustnzb`, a Rust workspace whose local -checkout lives at `/srv/work/rustnzbd` and whose GitHub repository was -transferred between organisations shortly before import. Substitute your own -project, path and remote. - ---- - -## 0. Establish the instance you are talking to - -``` -vogt status # or MCP `status` -vogt connect # what a client needs to reach this instance -``` - -Record `instance_id`, `principal`, `data_dir`. If you run more than one Vogt -instance (a development one and a production one, say), importing into the -wrong one is silent and only shows up later as "the project isn't there". - -Then confirm your credential can write projects: - -- `project.register`, `project.import`, `contract.check`, `sweep`, - `forge.onboard` and `forge.writeback` all need scope **`project.write`**. -- A typical agent token carries only `read` + `work.write`. It **cannot** - register a project; `vogt status` shows the scopes of the token in use. -- So the import is run either from the CLI inside the instance - (`docker exec vogt vogt project register ...` against the container, or - `uv run vogt ...` in a source checkout), or with a token minted for the job - (`vogt token issue --scopes read,work.write,project.write ...`, which only - a local process with access to the data directory can do). - -DECISION if no `project.write` path is available: stop; ask for a token or for -the CLI to be run on your behalf. Do not fall back to recording the project as -work items. - -## 1. Establish provenance — which remote is authoritative - -Do this before anything else, because `repo_url` is what every forge collector -keys off. - -``` -git -C remote -v -gh api repos// -i | head -1 # 200 after a redirect is still a redirect -gh api repos// --jq '.full_name, .archived, .default_branch' -gh api orgs//repos --jq '.[].full_name' | grep -i -``` - -`gh api` follows GitHub's transfer redirects and returns `200`, so a URL that -"works" is not evidence the repo still lives there. `full_name` in the body is -the authority; cross-check with the org listing. - -- rustnzb: the remembered URL `github.com//rustnzb` redirects. - `full_name` is `TheDancingDeveloper-org/rustnzb`, and the repo is absent from - the old organisation's listing. The canonical remote is - `https://github.com/TheDancingDeveloper-org/rustnzb`. - -Also check for other copies of the same code that are not the remote: a -self-hosted Forgejo/Gitea mirror, a second working tree, a sibling directory -with a different name. List them in the import note. A single -source of truth is a claim about *all* the copies, not just the one you found -first. - -DECISION when two remotes both look live (e.g. GitHub and Forgejo both take -pushes): which one Vogt tracks, and whether the other becomes a mirror or is -retired. Do not register until this is answered. - -## 2. Decide register vs import - -Two different operations, and the choice is not cosmetic: - -| | `project register` | `project import` | -|---|---|---| -| What it does | records an existing path as a project | clones the named GitHub or configured Forgejo repo into `import_root`, registers *that*, then consolidates | -| Working tree | the one you already have | a new one, server-side | -| Use when | the local tree is the tree people work in | the repo lives on a configured forge and there is no local tree of known ancestry | - -`project import` accepts `owner/name` for GitHub, `host/owner/name` for a -configured Forgejo host, or an explicit HTTPS/SSH repository URL. It -deliberately creates its own checkout so that later divergence -is news rather than ambiguity. That is right for a repo you have never had -locally — and wrong for a repo the user actively develops in, because it -manufactures the second copy you were trying to eliminate. - -- rustnzb has a live working tree at `/srv/work/rustnzbd` - on branch `release/v1.4.5`. → **register the local path**, and set `repo_url` - to the canonical remote from phase 1. Then run `forge onboard` explicitly to - get the consolidation step `import` would have done. - -Confirm the instance can actually *read* `root_path` before registering: the -collectors run inside the Vogt process, so in a container the repository -must be bind-mounted into it (see `docs/CUSTOMISATION.md`) and the registered -path must be the one *inside* the container. If an existing -project under the same parent directory has `git-local` and `source-markers` -sweeps reporting `ok`, that directory is visible to this instance. Check -`coverage` after phase 5 to prove it for yours. - -## 3. Pre-flight review (read-only) - -Run each of these and write down the answer; this is the review the human -reads before approving the write. - -1. **Name and slug.** The slug is derived from the display name. Directory - name, crate/package name and repo name may all differ — pick the one people - say out loud, and note the mismatch. rustnzb: directory `rustnzbd`, repo and - product `rustnzb`. → name `rustnzb`, slug `rustnzb`; the trailing `d` - is a local directory artefact and should not enter Vogt. - Check the slug is free: `project list`, or `project get --slug `. -2. **Contract, dry run.** `vogt contract check --path ` (no `--project`, - so nothing is stored). The default contract v1 wants files - `AGENTS.md`, `README.md`, `LICENSE` and directories `docs/`, `design/`, - `src/`. Record every failing criterion by name. - rustnzb fails three: no root `LICENSE` file (though `Cargo.toml` declares - `license = "MIT"`), no `design/`, no `src/` (it is a Cargo workspace — - code lives in `crates/` and `apps/`). -3. **Lifecycle state.** `incubating` / `active` / `maintenance` / `archived`, - judged from commit recency and releases, not from vibes. - rustnzb: released `v1.4.5` four days before import → `active`. -4. **Existing work to consolidate.** Open issues, PRs, labels, releases: - `gh api repos/ --jq '.open_issues_count'`, `gh pr list`, - `gh release list -L 5`. This sets expectations for phase 5 — an empty - consolidation is only good news if upstream really is empty. -5. **Exclusions.** Defaults are `.venv/ node_modules/ target/ dist/ build/ .git/`. - Add anything large or generated that is specific to this repo. - rustnzb also carries `.ci-output/`, `.ci-artifacts/`, `TestData/`, `demo/`, - `target/` under `benchnzb/` and `desktop/`. -6. **Working tree state.** `git status --porcelain`, current branch. A dirty - tree or a non-default branch is not a blocker, but it is context for the - first sweep and belongs in the reason string. - rustnzb: on `release/v1.4.5`, three untracked `.claude/` directories. - -Failing the contract does **not** block anything: "the contract is a value you -read, not a barrier you pass". Record the failures, do not fix the repo to -please it, and do not skip the import over them. - -DECISION on contract failures that are structural rather than sloppy — a Rust -workspace has no `src/` and never will. Either accept a permanent -`non_compliant` for those criteria, or raise the question of a per-language -contract. Do not invent an empty `src/` to score a pass. - -## 4. Register (first write) - -Every write takes a `reason`, and it is audited. Write a reason a stranger -could act on: what, why, and a reference. - -``` -vogt project register \ - --name rustnzb \ - --root-path /srv/work/rustnzbd \ - --repo-url https://github.com/TheDancingDeveloper-org/rustnzb \ - --lifecycle-state active \ - --reason "Onboard rustnzb; canonical remote is TheDancingDeveloper-org after the org transfer." -``` - -Verify: `project get --slug rustnzb` returns the record, `repo_url` matches -phase 1, `root_path` matches phase 2, `trust_state` is `unverified` (expected — -nothing has corroborated the declaration yet). - -## 5. Collect, consolidate, check - -In this order, each with its own reason: - -``` -vogt sweep --project rustnzb --reason "First sweep after onboarding rustnzb." -vogt forge onboard --project rustnzb --reason "Consolidate existing forge issues, PRs, labels and releases for rustnzb." -vogt contract check --project rustnzb --reason "Record rustnzb's contract status at onboarding." -vogt drift detect --reason "First drift pass after onboarding rustnzb." -``` - -- `sweep` runs the collectors; narrow it to the new project so you are not - paying for every registered project. -- `forge onboard` is read-only upstream. It is what `project import` would have - run for you; registering a local path skips it, so run it explicitly. -- `contract check --project` (not `--path`) is what records the status — - note `contract-checker` shows `never_run` in `coverage` until some project - has been checked this way. -- `drift detect` compares what the repo declares against what was observed. - -Then read it back and confirm the numbers are plausible: - -``` -vogt coverage # every collector ok/partial, and how stale -vogt project brief --slug rustnzb # version, CI status, bugs, backlog, freshness -vogt compliance --project rustnzb # the recorded contract result and its age -vogt drift list --project rustnzb -``` - -A brief that shows `observed_version` matching the repo's actual release, and a -`ci_status` matching what the configured forge says, is the signal the import landed. -`gh-posture` reporting `partial` for one project is a known collector wart, not -an import failure — read its `detail` before treating it as one. If no forge -token is configured for the project's host, `forge onboard` and the forge -collectors report "not collected" rather than an empty upstream; that is the -optional integration being absent, not a finding. Public repositories can -still be cloned by `project import` without a token, but consolidation waits -until the corresponding host credential is configured. - -## 6. Write-back policy — the last decision, made explicitly - -New projects default to `write_back: none`. Vogt only says things upstream once -someone chooses that: - -``` -vogt forge writeback --project --mode none|comment_only|full --reason "..." -``` - -Write-back is additive and forward-only (create, comment, label, close/reopen) -but it is still visible to everyone watching that repo. - -DECISION, always human: leave at `none` for a first import. Move to -`comment_only` or `full` only after a human has watched a full sweep and agrees -with what Vogt believes. `forge actions` is the ledger of what it has said and -what landed. - -## 7. Write the import note - -Append to the project's own `AGENTS.md` (or `docs/`) a short section recording: -the Vogt slug and instance, the canonical remote and any redirect you resolved, -which other copies exist and what their status now is, the contract criteria -that fail and whether that is accepted or open, and the write-back mode. This -is the part that makes "single source of truth" true for the *next* agent, -who will otherwise re-derive all of phase 1 from a stale memory of the URL. - ---- - -## Failure modes this ordering is designed to prevent - -- **Registering a redirected remote.** Phase 1 before phase 4 — collectors key - off `repo_url`, and a stale owner makes every forge sweep quietly wrong. -- **Manufacturing a second working tree.** Phase 2's table. `project import` is - the right verb only when there is no local tree worth keeping. -- **A project registered at a path the instance cannot read.** Phase 2's - visibility check and phase 5's `coverage` read-back. -- **Fixing the repo to satisfy the contract.** Phase 3 records failures; it - does not act on them. A `src/` created to score a pass is a lie in the tree. -- **Silent write-back.** Phase 6 is opt-in, human, and last. -- **An audit trail nobody can use.** Every write carries a reason naming the - driver, not "onboarding". diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 21ee36fe..52e2acba 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,1232 +1,40 @@ -# Vogt — Deliverable Stages (v0.3, revision r15) - -Status: **M0–M6 delivered — v1 is built** (2026-08-12); **M7 and M8 are -post-v1**; **M9–M14 are v2** — the session-engine merge, added by -requirements revision r9 (2026-08-14). **M9–M13 are built** -(2026-08-14); M11 now has automated Chromium GUI evidence, while its live -stack/manual conformance evidence and M13's device evidence remain outstanding. - -**M14 is delivered except for the acts that are somebody's to take.** The -merge is on `main`, both image streams are built and signed, CI is green -across both halves, and the fourth pass of the v2 delivery verification has -taken the "short or absent" column from twenty-five conjuncts to ten. Every -one of those ten is a decision rather than unfinished work: a development -deploy and a production deploy, an APK signing keystore that lives in the -retired forge, a native Anthropic backend deferred -at priority C, a voice pass that needs a device, and four rows where the -argued position is now stated instead of an owed item. What remains of this -stage is retiring the old stacks once the merged one has carried load, and -sunsetting the legacy `MYDEVENV2_*` environment names (still accepted as -aliases of the `ENGINE_*` names) after their transition period. - -Requirement IDs (`FR-*`, `NFR-*`) refer to the numbered requirements -baseline, which is maintained outside this repository. The IDs are stable -and append-only — an ID is never renumbered or reused — so they are quoted -here as plain text; every rule they name is also stated in words where it -matters. - -Each stage below carries an "as built" note recording where the delivery -differed from the sketch. What those notes cannot say is whether the set of -them adds up to the requirements baseline — that is the delivery -verification, written after v1 by checking the build against every ID. It -found seven requirements short of their text (FR-G1, FR-D2, FR-L1, FR-S3, -FR-S6, NFR-I3, NFR-S4) and one CI gate that does not fire on the paths most -likely to trip it (NFR-Q4). -**Nothing in this document should be read as delivered until that -verification agrees.** - -## The cut lines - -- **MVP = M0–M2** (r2): a tracker that is daily-usable by you and your - agents — the write plane, observed-first views over your registered - projects, and read-only GitHub so the views cover work that actually - exists. Everything after is enrichment. -- **M3** adds contract checking and the drift lifecycle; **M4** makes it a - self-hosted service; **M5** adds forge consolidation and write-back; - **M6** adds the GUI. **v1 = M0–M6.** -- M5 precedes M6 and the order is fixed: build the GUI once, against - complete data. -- **v2 = M9–M13** (r9): the session-engine merge, after which Vogt runs the - work it governs. **Merge-MVP = M9–M10** — the first build where a work item - can open a coding session. M14 is consolidation and delivers no new ID. -- M9 precedes M10 for the same reason M5 precedes M6, from the other - direction: the session capability is built once, against a repository and a - stack that are already one. - -Each stage ends with a **demo** — the stage is done when the demo runs, -its requirements' tests pass, and transport parity is green for every -operation the stage added. - -| Stage | Name | One-liner | Requirements delivered | -|---|---|---|---| -| M0 | Foundation | Storage, audit + events spine, operation registry, CI skeleton | FR-L1(part), FR-S1, FR-S2(local), FR-A2(part), FR-A3(harness), FR-N1(store), NFR-Q1–Q5, NFR-C1–C4, NFR-I1, NFR-I3, NFR-S3, NFR-O1, NFR-O3, NFR-PO3 | -| M1 | Core tracker | Local work tracking over CLI + REST + MCP stdio | FR-P1, FR-P2, FR-P4, FR-P5, FR-G11, FR-W1–W3, FR-W6–W9, FR-V1–V3, FR-A1–A4, FR-A5(stdio), FR-S6, FR-N1, FR-N2, NFR-Q3 | -| M2 | Eyes *(MVP)* | Collectors incl. read-only GitHub, observed-first, suppression, trust & freshness | FR-O1–O4, FR-O5a, FR-O6, FR-O7, FR-W4, FR-W5, FR-W10, FR-W11, FR-G12, FR-G15, FR-V4, FR-D1–D4, FR-P3, FR-R4, FR-L3, NFR-I2, NFR-I4, NFR-I5, NFR-S1, NFR-S2, NFR-S4, NFR-PO1, NFR-PO2 | -| M3 | Contract & drift | On-demand contract checks; the drift proposal lifecycle | FR-G1, FR-G3, FR-G4, FR-G13, FR-G14, FR-R1–R3, FR-R5, FR-D5, FR-D8 | -| M4 | Service | A deployed Compose stack, auth, remote MCP, ops | FR-A5(full), FR-A6, FR-A7, FR-S3–S5, FR-S7, FR-L1(full), FR-L2, NFR-D1–D10, NFR-C5, NFR-PO4, NFR-O2 | -| M5 | GitHub module | Consolidation, forge drift, write-back | FR-O5b, FR-B1–B5, FR-D6 | -| M6 | GUI | The visual surface over the same API | FR-U1, FR-U2 | -| M7 | Onboarding & inbox *(post-v1)* | Import a repository from GitHub; collect its notifications | FR-P6, FR-P7, FR-S8, FR-O8, FR-N3, FR-U3 | -| M8 | Reachable by an agent *(post-v1)* | `connect`, and the five deployment prerequisites behind it | FR-A8 | -| M9 | Foundations *(v2)* | One repo, one stack, one published port; both halves' CI green | NFR-D11, NFR-D12, NFR-C6, NFR-Q6, FR-U9 | -| M10 | Coding sessions *(v2, merge-MVP)* | A work item can open a session in its project's tree | FR-E1–E5, FR-E8, FR-E9, FR-S9, FR-S10; *(r18)* FR-E12 | -| M11 | GUI uplift *(v2, restructure)* | The Solid PWA becomes the single front end, then gains places, Inbox, and measured high-volume surfaces | FR-N4–N5, FR-U4–U8, FR-U10–U25, NFR-S5 | -| M12 | AI layer & voice *(v2, restructure)* | The assistant learns the Vogt domain; voice is validated rather than assumed; the existing gate is presented consistently | FR-T1–T4, FR-T6, FR-T8; FR-T5 validated, FR-T7 attempted; *(r16)* FR-T9–T13 and FR-M6 via the POC in `VOICE_POC.md`; *(r18)* FR-T14 | -| M13 | Mobile MVP1 *(v2, restructure)* | The phone is a first-class, sessions-first surface | FR-M1–M5, FR-E6, FR-E7 | -| M14 | Consolidation *(ongoing)* | Old stacks retired, names settled, r9 verified against the build | NFR-I6 (unclaimed by any other stage) | - -Deferred and withdrawn requirement IDs (FR-G2, FR-G5–G10, FR-D7) appear in -no stage by design — they are named non-requirements in the baseline. - -**On the numbering of M9–M14.** The merge design originally drafted these -stages as M8–M13, on the stated grounds that M7 was the last one. M8 was -already taken by *Reachable by an agent* above — the stage during which r8's -protocol-negotiation failure was found — so the merge stages start at M9. -The requirement IDs they deliver are unaffected: §4's append-only rule -governs IDs, and no ID moved. - ---- - -## M0 — Foundation - -**Objective**: the skeleton everything else bolts onto, with the quality -gates and CI shape locked in before any feature exists. - -Deliverables: -- Repo scaffold satisfying its own default contract — including the - `LICENSE`, which makes licence selection an **M0 decision**, not a - pre-publication one (NFR-O1/O3). -- `pyproject.toml`, committed `uv.lock`, Renovate config (all three - toggles), mypy strict + ruff + coverage gate wired. -- GitHub Actions: `docs.yml` / `ci.yml` / tag-only `release.yml` with the - docs-skip path filtering and gate-job pattern. (`build.yml` joins them at - r5, publishing commit images from main — it is not an M0 deliverable.) -- `declared.sqlite3` + `observed.sqlite3` with migration framework, lock, - meta/revision; `audit` and `events` tables and the transactional write - path — entity + audit row + event row + revision bump, atomically - (NFR-I1, FR-N1). -- Actor model with `local:` principal derivation; `reason` - required and non-empty on every write. -- Storage behind an interface that avoids SQLite-only semantics, so a - Postgres backend stays possible without a redesign (NFR-S3). -- The transport-neutral **operation registry** and the parity-test harness - with both exclusion lists (`HTTP_ONLY`, `LOCAL_ONLY`) live. -- CLI: `init`, `status`. - -**Demo**: `vogt init`, register a project record from the CLI with a -reason, `status` shows revision 1, the audit row carries actor + reason, -and `/events` returns exactly one row at `seq=1`. `mypy --strict` and the -parity harness pass in CI; a docs-only commit runs only `docs.yml`. - -### M0 as built — three notes - -Recorded because each one differs slightly from the sketch above, and the -absences and additions should read as decisions. - -1. **Six operations, not two.** The demo requires registering a project and - reading the event and audit rows it produced, so M0 ships - `project.register`, `project.list`, `events.list` and `audit.list` - alongside `init` and `status` — each on all three surfaces. This is the - demo's own surface area, not an M1 pull-forward: FR-P1's lifecycle - transitions, FR-P2's brief, FR-S6's full audit query and FR-N1's complete - feed semantics all remain M1 work. -2. **`LOCAL_ONLY` is not empty; it names `init`.** DESIGN §4 already lists - `init` among the operations with no meaningful remote semantics, so - starting the list empty would have meant either an exclusion that lies or - an `init` route that a running server cannot honour. The list is live and - checked for staleness in both directions, which is what FR-A3 asks for; - `backup`, `restore` and `serve` join it at M4. -3. **`init` is a bootstrap, not a declared write.** It creates the instance - rather than changing anything inside one, so it lands an audit row at - revision 0 and emits **no event** — which is what makes the demo's - "exactly one row at `seq=1`" true, and means a client attaching to - `/events` sees changes it can act on rather than the instance's own - birth. Every other write, including the auto-registration of a - previously unseen principal, lands both rows (`SCHEMA.md` §2.5). - -## M1 — Core tracker (first daily-usable build) - -**Objective**: a working local tracker — the write plane — reachable from -all three surfaces. - -Deliverables: -- `project register` / `project create` (scaffolds a compliant skeleton; - registration is never refused, FR-G11); lifecycle states; the - one-repo-one-project granularity rule (FR-P5). -- Work items (4 kinds, p0–p4, effort, assignee); typed cross-project - relations (`depends_on`, `relates_to`, `duplicate_of`, `parent_of`); - labels (GitHub-aligned); initiatives; comments. -- Cursor-based `/events` feed over the M0 events table. -- Workflow engine with per-kind state machines; rejected transitions name - the violated rule. -- Deterministic ranking with `why` explanations; `brief`, `backlog`, - `bugs` views with filters. -- REST (FastAPI + OpenAPI) and MCP stdio, both generated from the - registry; CLI verbs for everything; audit browsing. -- No manual ranking override: ordering is computed, `why` explains all of - it, and `priority` / initiative weight are the hand-set inputs. - -**Demo**: from Claude Code via stdio MCP: create a bug, block it on -another item, transition it, ask `backlog` and `why` — then show the same -state from the CLI and `curl`, with identical answers, and the audit trail -and event feed of everything the agent just did. - -### M1 as built — four notes - -1. **27 operations, one definition each.** The registry now carries the - whole write plane plus the views, and the parity harness drives every one - of them through CLI, REST and MCP as a single ordered script against three - isolated instances — ordered, because you cannot relate two work items - before creating them, and a `why` that never ran against a real ranked - item proves nothing. -2. **`mcp.stdio` joins `init` in `LOCAL_ONLY`.** A transport that takes over - the process's stdout has no remote semantics; offering it as a REST route - would mean a server hijacking its own framing channel. It is also a bare - `vogt-mcp` console script, because an MCP client config wants a command - rather than an argument list. -3. **Workflows are stored, not hard-coded.** `workflow_defs` holds one - machine per kind, seeded from the shipped defaults by `migrate` — not by - bootstrap, because an instance created before the table existed never - bootstraps again, and not by the migration SQL, because the defaults would - then be spelled twice and drift. `workflow.list` publishes them, so an - agent can pick a legal next state instead of guessing and handling a - rejection. -4. **Ranking rounds before it answers.** An unrounded staleness contribution - carries float noise like `1.157e-05`, which is not information, reads as - spurious precision in `why`, and makes two reads of one item look - different. Four decimal places is far below any ordering the weights can - express. `ci_red_boost` is listed in `why` as an input that *cannot fire - yet* rather than omitted, so the explanation is honest about what it is - not considering. - -## M2 — Eyes (MVP complete) - -**Objective**: the tool sees work you didn't type in — including the work -that lives on GitHub, which is where most of it actually is. - -Deliverables: -- Collector framework: plugin registry, in-process scheduler, on-demand - `sweep`, sweep/coverage records, digest dedup, append-only store + - rebuildable `latest_*` tables. **Scope is always the registered project - list** (FR-G15) — no crawling, no candidates. -- Core collectors: `git-local` (branch, dirty state, tags → version), - `source-markers`, `dep-refs` (path/git/workspace references from - `Cargo.toml`, `package.json`, `pyproject.toml` — manifests only, no - lockfiles, no versions). -- **Read-only GitHub collectors** (FR-O5a): issues, PRs, Actions runs, - releases/tags. No writes, no backfill, no posture — those are M5. -- Noise control before observed-first is switched on: promotion by - convention (`TODO(vogt):`), audited `suppress`, per-project exclusions. -- Observed-first backlog/bugs; `adopt` promotion with maintained links. -- Cross-project dependency references: resolution to registered projects, - reverse lookup, unresolved targets retained. -- Computed trust states; freshness stamps on every aggregated answer; - "not collected" semantics for unswept scope. -- Retention policy over the observed store (NFR-I5): latest observation - per subject kept indefinitely, history pruned on a configurable window. - The drift-proposal exemption arrives with FR-R5 at M3. -- The NFR-S4 benchmark fixture at the ~500-project / ~100k-item envelope, - asserting the sub-second interactive target of NFR-S1 in CI. - -**Demo**: register a dozen real projects; sweep. The global bugs view -shows GitHub issues *and* promoted markers from rustnzb and rustTorrent -with freshness stamps and trust states; suppress a noisy marker and watch -it leave the ranked view but stay in `observations`; `deps --project -nzb-core` lists the projects referencing it; adopt one item into a ranked -work item. Then re-run the whole suite with the network unplugged and the -GitHub adapter disabled — everything except forge observations still works -(NFR-PO1/PO2). **This demo is the MVP acceptance test.** - -### M2 as built — four notes - -1. **Two derived tables, not five.** `SCHEMA.md` §3.2 named five typed - `latest_*` projections. Only the dependency one carries anything the - observation does not already say, so this ships `latest_observations` - (generic, keyed by subject) plus `latest_dep_refs`. The other four - differed in payload shape and not in behaviour, and five rebuild paths - would have been five places for a collector and its projection to drift - apart. Recorded in the migration. -2. **A marker is a leading annotation, not a mention.** The first sweep of - this repository promoted 21 "markers" — every one of them documentation - *about* the promotion pattern: `DESIGN.md` explaining that `TODO(vogt):` - enters the backlog, the generated `config.example.toml` listing the - defaults, a table cell naming them. Vogt read its own description of - markers and filed it as work. The pattern is now anchored to the start of - a line, after comment leaders and list bullets only, which took that - repository from 283 observed markers to 57 and from 21 promoted to 0 — - the right answer, since neither project uses the convention yet. Any - project that documents its conventions would have hit this. -3. **Suppression and adoption live in the declared store.** Both are audited - decisions, not observations, which is exactly why a suppression survives - re-observation of the same subject — a dismissal recorded in the evidence - store could never have worked. Adopted subjects are folded into their - work item rather than listed twice. -4. **The benchmark is a tripwire, not a benchmark suite.** It seeds 500 - projects and asserts the ranked views stay interactive. The threshold is - generous because a wall-clock number on a shared runner is not a metric; - what it actually catches is an accidental per-item query inside a ranked - view, which would otherwise be invisible until M6. - -## M3 — Contract & drift - -**Objective**: the contract is checkable and disagreements between -declared state and observation become resolvable proposals. - -Deliverables: -- The default contract (config, versioned identifier); `contract check` - against any path, returning every rule evaluated and every failing - criterion (FR-G3/G4). -- Recording the result as a registered project's compliance status with a - checked-at timestamp, surfaced with its age on the brief and the global - view (FR-G14). Nothing re-checks on a timer. -- The FR-G13 assertion tested: no code path consumes compliance, trust, or - drift status as a precondition. -- Drift proposal lifecycle: raise with evidence, accept / reject / - contest, all audited, with the default low-risk auto-accept policy; - evidence snapshot at raise time and retention pinning (FR-R5). -- Local drift kinds: `version_mismatch`, `unresolved_dependency`. -- `mirrored_source` reporting (FR-D8) where a path member is also a - registered project. **Not delivered at M3** — see note 4 below; built at - r15, after an estate onboarding produced eighteen of them by hand. - -**Demo**: `contract check` a project missing `AGENTS.md` → the result names -exactly that criterion, and the project's brief shows `non_compliant` -alongside "checked 4 seconds ago"; check nothing for a week and the same -brief says so rather than quietly refreshing. Tag a release without -updating the declared version, sweep → `version_mismatch` proposal with -evidence; accept it, and the audit trail shows who accepted it and why. -Delete the observation's history window and confirm the proposal still -renders its evidence. - -### M3 as built — four notes - -1. **FR-G13 is asserted structurally, not just behaviourally.** One test - registers work against a non-compliant project and transitions it, to - show nothing refuses. A second greps the source: only the modules that - *compute or report* compliance may mention `compliant` / - `non_compliant` at all. A service that starts comparing against those - values fails the suite, which is the only version of this rule that - survives people who have not read the requirement. -2. **`contract-checker` is a collector that opts out of sweeps.** It - carries `on_demand_only`, and `CollectorRegistry.select` skips such - collectors unless they are named. Without that flag, adding the M4 - scheduler would quietly reintroduce the continuous re-checking r3 - deleted — the requirement would have been enforced by nothing but - memory. -3. **Evidence is protected twice, deliberately.** A proposal carries a - self-contained snapshot *and* pins the observation it points at against - retention. Either alone would do in the happy path; both are needed - because the two stores are pruned, backed up and restored - independently. The test deletes `observed.sqlite3` outright and the - proposal still renders its evidence. -4. **`mirrored_source` was listed here and not built** — for eleven stages, - without appearing in the gap register either, because that register was - assembled from gaps documents *stated* and this one was visible only by - grepping for the identifier. Delivered at r15 as its own collector; r15 - records both the delivery and the register's blind spot. - -## M4 — Service (the deployed stack) - -**Objective**: from local tool to always-on service with real identity, -running where it will actually live — a Compose stack on the maintainer's -host, per [`DEPLOYMENT.md`](DEPLOYMENT.md), not a shape to be decided at -this stage. - -Deliverables: -- `serve`: one port, path-routed GUI-placeholder / `/api` / `/mcp` / - health; streamable HTTP MCP with protocol-version negotiation; - `/connection-info`; in-process TLS from a mounted certificate - (`--tls-cert` / `--tls-key`, NFR-D6). -- Token auth bound to actors; scopes; double-gated writes; scope-filtered - `tools/list`; allow+deny audit; token-file config only. -- `vogt-mcp-remote` bridge with startup tool discovery and stderr skew - warning. -- `backup` / `restore` / `export` / `import`; generated example configs; - the NFR-D2 default-policy split enforced (exposure values ungated, - allocation values defaulted). -- `release.yml` completes: buildx image, SBOM and provenance attestations, - keyless cosign sign over the digest, push to - `ghcr.io/thedancingdeveloper-org/vogt`, tag-triggered only, on a - self-hosted runner carrying the `publish` label. *(As built: the - attestations are buildkit's rather than a separate syft run, and cosign - signs without a separate `attest` step. `build.yml` joins it at r5 — - see the r5 note at the end of this document.)* -- The deployed stack: a compose file held in version control, digest-pinned, - bound to a private-network address, hardened per NFR-D9 (read-only root - filesystem, dropped capabilities, non-root user), healthchecked on - `/health/ready`. **Allocate the port here** and verify it free on the - host before committing. - -**Demo**: tag a release → GitHub Actions publishes a signed image and -nothing deploys; pin the digest, redeploy the stack, and it comes up. From -a workstation on the same private network, Claude Code connects via the -bridge with a read-only token — write tools are absent from its tool list; -swap to a `work.write` token — they appear. Probe `/health/ready` with plain -curl on the private address (and confirm nothing answers on any other -interface). Backup, destroy the stack, redeploy, restore — state intact. -Then revert the digest and redeploy to prove rollback. - -### M4 as built — four notes - -1. **The port was allocated and verified free on the host** rather than - inherited from a document. The compose file carries it as a *default*, - not a `${X:?}` gate — gating allocation values turns every deploy into - a hunt for a number nobody cares about, which is the NFR-D2 revision - recorded in `DESIGN.md` §9. -2. **The served context carries the authenticated principal.** The first - version resolved the token correctly and then ran the operation under a - context built from the OS user, so every audited write through the - server would have been stamped with the server operator's name. A test - that asserted attribution caught it. Authentication that resolves the - right identity and then does not use it is worse than none: it looks - correct in the audit trail. -3. **A 401 is answered before an empty body is.** The bridge treated an - empty response as "notification, no reply", so a rejected token left the - client waiting forever instead of learning its credential was refused. -4. **Import is read-only, and says so.** Merging two instances needs an - identity-conflict policy — same slug, different project; same ref, - different item — and inventing one silently is how an import destroys - what it was meant to restore. `restore` is the supported way to move an - instance; `import` reads a file and reports what is in it. - -## M5 — GitHub module (the adapter earns its keep) - -**Objective**: the forge side stops being read-only observation and starts -keeping declared work honest — in both directions, carefully. - -Deliverables: -- **Onboarding = non-destructive consolidation** (FR-B3): a read-only - backfill of all *existing* issues, PRs, labels, releases, and CI history - into observations. Zero GitHub mutations during onboarding; existing - state is incumbent and preserved. -- Forge drift kinds: `forge_state_mismatch`, `vanished_upstream`, - `ci_red_vs_healthy`; per-project/per-kind auto-accept policy. -- Per-toggle update-automation posture (FR-D6): version-updates config, - vulnerability alerts, automated security fixes — three facts, three - answers, never one boolean. `update_automation_gap` drift. -- Write-back module: `none / comment_only / full`, additive/forward-only, - audited, re-observed on the next sweep. Comments go **outbound only** - (FR-B5) — inbound forge comments stay observations against the linked - item. - -**Demo**: enable the module for the rustnzb org and verify — via the -GitHub audit log and `updated_at` timestamps — that onboarding changed -nothing upstream, while every existing issue and PR appears in the global -views with labels intact. Then close an issue on GitHub that a declared -item links → drift proposal; accept → the item closes with provenance. - -### M5 as built — four notes - -1. **"Changed nothing upstream" is asserted, not inspected.** The demo - proposed checking GitHub's audit log after the fact. The test instead - records every HTTP request the adapter makes and fails on any method - that is not a GET — which cannot pass by accident, and fails at the - moment somebody adds a mutation rather than the next time a human reads - a log. -2. **Write-back is a consequence of a declared write, never a separate - act.** There is no "push this upstream" operation: a comment authored - here posts upstream *as part of commenting*, and finishing an item - closes the linked issue *as part of transitioning*. A separate push - command would let the declared change and its upstream half drift - apart, which is the thing this product exists to notice. -3. **FR-B4 is enforced by absence.** There is no deletion, force or - history-rewrite capability — not disabled, not gated, not present. The - client refuses any method other than POST and PATCH, and a test asserts - the union of every policy's permitted actions is exactly - `{create, comment, label, close, reopen}`. -4. **Consolidation does not flood the backlog.** A backfill reads closed - history too — that is the whole point — but only *open* issues are - promoted. Otherwise onboarding a ten-year-old repository would put a - decade of finished work into the ranked view on day one, which is the - observed-first hazard (DESIGN §3.6) arriving through a different door. - -## M6 — GUI - -**Objective**: the visual surface, consuming only the public REST API. - -Deliverables: per-project view; global backlog + bugs; drift inbox; -dependency reference graph; audit browser; trust, freshness and -compliance-age rendered on every aggregate. Served from the same single -port. - -**Demo**: every M2/M3/M5 demo step repeated through the browser, and -nothing the GUI does is absent from the API (parity rule holds). - -### M6 as built — one deviation and four findings - -**Deviation: buildless ES modules, not a React SPA.** `DESIGN.md` §4's -architecture sketch named React (since corrected in place). The reason for not using it is packaging rather than taste: Vogt -installs as a Python wheel, so a framework build means either a Node -toolchain present at wheel-build time — `pip install vogt` requiring npm — -or committed bundler output, which is a generated artefact in version -control that nothing verifies. Neither is worth it for six views over an API -that already exists. What the design actually required is unchanged: the GUI -consumes only the public REST surface, adds no capability of its own, and is -served from the same single port. `test_the_gui_needs_no_build_step` fails if -a build step ever appears, so the deviation has to be re-argued rather than -quietly reversed. - -1. **FR-U2 was unsatisfiable as specified.** "Trust and freshness on every - aggregated view" could not be met for the drift inbox or the dependency - graph, because `DriftListResult` and `DepsResult` carried no freshness. - The GUI could have computed it client-side — which would have broken the - parity rule from the other direction, the GUI doing something the API - cannot. Fixed in the API. The inbox is where it matters most: an empty - inbox is reassuring only if something has looked recently, and without - freshness a collector that stopped running reads as "no drift". -2. **The parity rule is checked against the shipped source, not the - intent.** A test extracts every `/api/...` literal from `app.js` and - resolves it against the operation registry, asserts there is exactly one - `fetch(` in the file, and asserts every operation the GUI names is - non-mutating. A view that grows its own endpoint fails at the moment it - is written. -3. **The comment-reading bug appeared for the fourth time.** The check that - the GUI uses `sessionStorage` rather than `localStorage` failed on the - comment explaining why. Same shape as the marker collector matching prose - about markers and the deploy test reading its own comment: a comment - explaining a rule contains the words the rule forbids. Every - source-reading assertion in `test_gui.py` now goes through one - comment-stripping helper. - -4. **FR-L3 had never been built.** Found by walking every must-have - requirement ID and asking which are cited nowhere in `src/` or `tests/`. - FR-L3 has two halves — collectors run "on an in-process schedule" *and* - are triggerable on demand — and only the second existed. The deployment - document had listed "collector scheduler (in-process background sweeps)" - in the `serve` process diagram since M4, so it described something that - did not run. Built at M6 and covered by - `tests/test_scheduler.py`; the requirement belonged to M2. - -The GUI is read-only. Resolving drift, transitioning work and setting a -write-back policy all require a reason its author typed, and a button cannot -type one — "accepted via GUI" is not a reason (FR-W1). - -The schedule is on by default, at fifteen minutes. That is the one place a -default was chosen rather than required, and the reason is that an instance -which never looks cannot tell stale evidence from none — the failure this -product exists to prevent. `--no-schedule` and `sweep_interval_seconds: 0` -both turn it off. - ---- - -## M7 — Onboarding & inbox (post-v1) - -**Objective**: getting a repository from GitHub or a configured Forgejo host -into Vogt should be one act, and what its forge is trying to tell you about it -should be visible here. - -Deliverables: -- **Import** (FR-P6, FR-P7): `project.import` takes a GitHub or configured - Forgejo repository the caller names, clones it into `import_root`, registers - it with `repo_url` set, and - runs the FR-B3 consolidation — one operation, one reason, one audit trail. - Clone before declared write, so the failure mode is a stray checkout rather - than a project pointing at nothing. -- **Clone credentials out of band** (FR-S8): the token reaches `git` through - an askpass helper. Not in the URL, not in argv, not in the clone's config, - and not in the stored `repo_url`. -- **`gh-notifications`** (FR-O8): a collector over the per-repository - notifications endpoint, unpromoted, degrading to `partial` when the token - lacks the scope. -- **The inbox** (FR-N3, FR-U3): a read-only view and a GUI page, plus the - import form. - -**Demo**: import a repository that exists only on GitHub, from the GUI, with -a reason. It lands on disk at the configured root, appears in the project -list with its issues and PRs already consolidated, and its notifications show -up in the inbox on the next sweep — while `/events` still contains only what -this instance did. - -### Why import rather than register-then-reconcile - -`project.register` assumes the working tree is authoritative for its own -provenance, which is true for a folder and false for a checkout of somebody -else's repository. Registering a local tree and letting collectors discover -the remote means the first sweep compares two sources of unknown -relationship; importing means the local tree is a known derivative of the -remote from the first observation, and every divergence after that is real -news. It is the same argument as consolidation being read-only (FR-B3): the -incumbent is authoritative, and Vogt's job is to notice, not to reconcile -what it could simply have known. - -**The temptation this stage must refuse** is the repository picker. An import -form with a text field is one HTTP call away from an import form with a -dropdown of your repositories, and that dropdown is the registration -candidate listing r3 removed (was FR-G8). The baseline defers it again by -name so that adding it is a decision rather than a Tuesday. - -### M7 as built — one deviation and three notes - -**Deviation: the GUI is no longer read-only.** M6 shipped a rule — every -operation the GUI names is a read — justified by FR-W1: a write needs a -reason its author typed, and a button cannot type one. The import form can, -and does: `reason` is a required field, sent verbatim, and the audit row -records what the user wrote rather than "via GUI". The rule the GUI actually -keeps is therefore narrower than the one M6 wrote down, and it is now stated -that way: a mutating operation may appear in the GUI only through a view -that collects a typed reason. `tests/test_gui.py` holds the permitted set as -`GUI_WRITES` and asserts the reason field exists for each member, so adding -a second write means arguing for it rather than appending to a list. Drift -resolution stays out — resolving from a list *is* a button. - -1. **The cloner is injected through the context, not passed to the - handler.** `AppContext` already carries the clock and the id factory for - exactly this reason, and `project.import` is the only operation that - reaches the internet to change local state. Putting it anywhere else - would have meant the parity harness — which drives every shared operation - over all three transports — needing a network to run. -2. **The token reaches `git` through `GIT_ASKPASS`.** The three obvious - alternatives all leak: a credential in the remote URL is written into - `.git/config` permanently, `-c http.extraHeader=` sits in argv, and - `credential.helper store` writes a file nobody deletes. Two tests assert - the negative — the token appears in no recorded argv and in no resulting - `.git/config`, and the helper does not survive the clone. -3. **An occupied destination is never overwritten.** A clone of the same - remote is reused and reported as such; anything else fails untouched. The - test that matters is the one where the destination holds unrelated files: - an import that deletes somebody's working tree would be the most - destructive thing this product could do, and it is the cheapest possible - mistake to make. - ---- - -## v2 — the merge (M9–M14) - -Added by requirements revision **r9**, which reverses `DESIGN.md` §1.2's -*"being an agent runner"* non-goal: a previously separate session-engine -codebase becomes Vogt's session engine (`engine/`, `web/`, `mobile/` — -see [`ENGINE.md`](ENGINE.md)), and Vogt runs the work it governs. The -surviving boundary is that Vogt never decides to run anything on its own — -every session traces to a person or to a schedule a person created; -autonomous work pickup is a named non-requirement. Sizes are relative -(S < M < L), and as everywhere in this document, "as built" notes come after -the stage is built, not before. - -## M9 — Foundations (M) - -**Objective**: one repository, one stack, one published port — the ground -every later merge stage stands on, and nothing user-visible beyond the fact -that both halves now answer at the same address. - -Delivers NFR-D11, NFR-D12, NFR-C6, NFR-Q6, FR-U9. - -Deliverables: -- **The repo merge, with history**, from the engine's development branch - via `git subtree`: `engine/` (Rust workspace), `web/` (Solid PWA), - `mobile/` (Capacitor shell), and the engine's documents (since folded - into `ENGINE.md`). The archived GPUI desktop client is not carried over. - Provenance is the development branch, not `main`, because that is where - the Vogt MCP registration and the ContextKeeper work already lived, and - merging `main` would mean re-doing them. -- **The front door** (NFR-D11): the Rust engine is the single listening - process — PWA, native APIs, WebSocket attach, `/api/vogt` and `/mcp` - reverse-proxied to vogt-core, and aggregate health. Vogt-core binds - loopback only. Every port that serves MCP still serves plain HTTP health - (FR-A7), which is the invariant the proxy is most likely to quietly lose. -- **The dev/prod split** (NFR-D12), branch-shaped before anything else - lands: `dev` → `:dev` images → the maintainer's development instance; - only `main` reaches production. Mobile, voice and push are verifiable - nowhere else. -- **Merged CI** (NFR-C6): Rust fmt/clippy/test, web typecheck and APK build - join the Python suite, under NFR-C3's build-vs-release discipline. -- **Both absence-modes green** (NFR-Q6): the forge-less run, unchanged, and - a core run with no engine present. -- **The legacy GUI keeps serving** at `/ui-legacy` (FR-U9) so nothing - regresses while M11 is being built. - -**Demo**: one URL serves the PWA, a terminal session, and a Vogt backlog -query. Both suites pass in the merged repository, including the engine-less -core run; a push to `dev` produces a `:dev` image that deploys to the dev -stack, and no push to `dev` moves prod. - -### M9 as built — one deviation and five notes - -**The deviation: the demo ran against two processes, not against the -stack.** vogt-core and the engine were started by hand on loopback and -driven through the engine's port — the backlog answered at `/api/vogt`, -`/ui-legacy` served the vanilla GUI, `POST /api/sessions` opened a PTY, an -MCP `initialize` completed byte-identically to the core's own answer, and a -work item created through the front door landed in the audit log as -`agent:m9demo` with the reason its caller typed. What had *not* run was the -merged image: no Docker daemon was reachable from the environment this was -built in, so `engine/Dockerfile` parsed, every `COPY` resolved, and nothing -more could honestly be claimed. - -**The deviation is closed, on 2026-08-14, and by a runner rather than by -this container.** `build.yml`'s `stack-image` job ran for the first time: -28m53s, `engine/Dockerfile` built with the repository as its context, `vogt ---version` and the engine's `--help` both run *inside* the candidate -before it was pushed, the digest signed, and `dev` plus `dev-ee18adc` -published to `ghcr.io/thedancingdeveloper-org/vogt-stack` — which -`deploy/vogt-stack.compose.yml` now pins in place of the placeholder digest -of zeros it had carried since this note was written. The riskiest unproven -step, copying uv's standalone CPython between build stages, holds: `vogt ---version` cannot run in that image without it. - -What is still unproven is the *stack*, as opposed to the image — the two -processes coming up together under `entrypoint.sh`, the engine finding the -core on loopback, and `/readyz` reporting a core it actually reached. That -needs the compose stack up and `scripts/smoke_merged_stack.sh` pointed at -it, and it is a deploy, which is -a human act (NFR-D10). - -1. **`ReadinessCheck` grew a `fatal` flag, and the core's probe is not - fatal.** Aggregate health was specified; what it should *do* was not. If - an absent core made the container unready, a healthcheck would restart - the engine — which cannot revive the core and would kill every live PTY - doing it, the exact cost FR-E9 says an absent core must never have. So - the four engine-owned checks keep the verdict and the core's outage is - reported beside them. Do not "fix" a red `vogt_core` check by making it - fatal. - -2. **Supervision is the entrypoint, not a framework.** s6-overlay and - supervisord were both weighed and rejected: this container is a - development pod that already sequences optional daemons in an ordered - script, runs as an unprivileged user rather than root, and gets tini as PID 1 from - the compose. A second init system would have displaced that tini, put - PID 1 back to root, and split startup order across two places — for two - long-lived processes. The core is a backoff respawn loop; the engine - stays what `exec` replaces the shell with, so the container's lifetime is - the front door's lifetime. The core's listen address is *derived from* - `VOGT_CORE_URL`, so "loopback only, never published" is enforced rather - than commented, and the two-container fallback (engine and core as separate - services) still works unchanged. - -3. **`vogt init` now runs before every `serve`.** The deployment document - recorded that nothing migrates — `serve` does not, and there is no `vogt - migrate` — leaving a manual "run `vogt init` after a digest bump" step - that a deploy would eventually skip. The supervisor runs it in the retry - loop. The gap in the *product* is still open (FR-L1); what closed is the - gap in this deployment. - -4. **What the merged CI does not carry over from the engine's previous - pipeline**, each for a reason and each written into the workflow: a - shared Rust compile cache backed by a Redis that OOM-crash-looped and - blocked the pipeline (`Swatinem/rust-cache` replaces it), the private - registry pushes and deploy steps (Vogt does not deploy from CI, NFR-D10), - and **APK release signing**, whose keystore lives in the retired forge — the APK builds unsigned, and where - a signed one gets published is an untaken release decision, not an - oversight. - -5. **Three bugs the merge surfaced, two of them older than it.** The CLI - had been dead on Python 3.13 and later since `serve` grew `--no-auth`: - `BooleanOptionalAction` refuses an option name starting with `--no-`, at - parser construction, so *every* command including `--help` raised — and - the suite never saw it because it ran on 3.11, while `requires-python` - says 3.11 *or later*. An engine integration test was flaky about one run - in five. And `/ui-legacy/` — with the trailing slash a browser sends — - fell through to the PWA's catch-all, because a wildcard route segment - matches at least one character; it answered 404 with the engine's "web - bundle not present" placeholder, which reads like a broken front door - rather than a route that never matched. - -**Still open after M9**: the merged image has no TLS (the engine has none; -where the core terminated TLS in-process, the merged port speaks plain HTTP -and expects a reverse proxy or a private network in front of it — the -ingress decision the merge design leaves open), CI does not yet build -`engine/Dockerfile` with a repository-root context, and the merged image's -registry name is a placeholder awaiting that decision. - -## M10 — Coding sessions (M) *(merge-MVP)* - -**Objective**: the capability the reversal exists for — a work item can open -a session in its project's tree, and what the agent does in there comes back -as attributed, audited writes. - -Delivers FR-E1–E5, FR-E8, FR-E9, FR-S9, FR-S10. - -Deliverables: -- **Workspace unification** (FR-E3), the one semantic join: Vogt's import - root and the engine's workspace root become the same tree, and the project - registry — not a path heuristic and not a repo-name match — decides where a - session for a project opens. -- **`session.start` / `session.list` / `session.stop`** in the operation - registry (FR-E8), and therefore on CLI, REST and MCP with parity, like - everything else. -- **Work item ↔ session linkage** (FR-E4): the brief written to a prompt file - through the agent-task mechanism, the session id recorded on the item as an - audited write, the item's views carrying the session's live activity state - (FR-E2). -- **First-party MCP registration** (FR-E5) for agents inside a session, - carrying a **per-session actor-scoped token** minted at start and revoked at - session end (FR-S10). -- **Auth mapping** (FR-S9): the front door's token namespace maps to named - Vogt actors whose paired core tokens the proxy injects. The proxy forwards - and never pre-approves; double-gated writes are unweakened. -- **The engine still boots without the core** (FR-E9) — absence of Vogt costs - Vogt features, never a terminal. - -**Demo**: import a GitHub repository, create a work item on it, start an -agent session from that item, watch the agent update the work item over MCP, -and read the write in the audit log attributed to *that session's* actor — -not to the proxy and not to a shared assistant. Then stop vogt-core and -confirm the terminal is still usable. - -### M10 as built — the demo, and what running it found - -The demo runs as `tests/test_m10_demo.py` and was also run live, against -vogt-core and the engine as two processes: a work item, a real PTY opened in -the registry's tree, the brief on disk where the engine put it, an agent -inside that terminal posting a comment with the credential its session was -given, and the audit reading -`agent:session:ses_01KZZ1XPW5GMG6NMKG6BV7Q7RE` beside a `session.start` from -`local:`. - -**Running it live is what found the requirement failing.** Three -independent places overwrote a session's token with the pod's shared one, -and every one of them failed silently — the agent authenticated, wrote, -and got a 200, while the audit log recorded the wrong actor: - -1. The engine's agent-auth helper fetched the pod's Vogt token - unconditionally, and with automatic agent auth enabled — the deployed - setting — it is what launches every session's shell. -2. The MCP wrapper Claude Code and OpenCode are registered with re-brokers - through that same helper before running the bridge. -3. The stdio bridge read only `VOGT_TOKEN_FILE`, which the broker rewrites. - -All three now defer to a session's own token when `VOGT_SESSION_ID` is set, -and the bridge states the precedence in one place. Codex was already correct -by accident: it registers the URL natively with `--bearer-token-env-var -VOGT_HTTP_TOKEN` and reads what the session set. - -The class of bug is worth naming, because M11–M13 can reintroduce it: **a -credential that is silently replaced produces working writes and a false -audit trail.** Nothing errors, no test that stubs the transport can see it, -and the only signal is an actor name nobody reads until they need to. Two -smaller notes from the same session: - -- The engine has **no environment override for `workspace_root`**; it comes - from the config file, else `$HOME/Working`. An hour went to a session - refusing to open in a path that was, as far as the operator was - concerned, configured. -- Vogt sends the registry's **absolute** path as the session's `cwd`, which - the engine accepts via `resolve_existing_allow_absolute` — the path must - still resolve inside the engine's workspace root after symlinks. Estates - mounted somewhere other than that root will be refused, correctly, and - §6.3's "the import root and the engine's workspace root shall be the same - tree" is what prevents it. - -## M11 — GUI uplift (L) - -**Objective**: the Solid PWA becomes the single front end, specified to -interaction depth rather than to a list of views — M6's lesson, applied -before the fact this time. - -Delivers FR-N4–N5, FR-U4–U8, FR-U10–U25, NFR-S5, and retires FR-U9's -legacy surface at parity. The reviewed restructure is staged inside M11 so -the normalized Inbox contract and the places shell land before their surfaces. - -### M11 restructure delivery order - -The implementation plan in `RESTRUCTURE.md` is the acceptance boundary for -this addition to M11: - -1. Stage 0 records FR-N4–N5, FR-U23–U25, FR-T8 and FR-M5 in the requirements - history and gap register; no product code is justified before that gate. -2. Stage 1 delivers the server-owned Inbox projection and audited triage. -3. Stage 3 migrates the product shell to stable places and Sessions panes; - Stage 4 consumes the single Inbox operation. -4. Stage 5 proves measured content-sized windowing and bounded reads before - Stages 6–7 change Board and Backlog. -5. Stages 8–9 complete Sessions presentation and the sessions-first phone; - Stage 10 records automated, browser, live-stack and device evidence. - -An unfinished stage remains in the gap register even when an earlier stage's -code has landed. Source inspection or jsdom does not close the browser, -live-stack or device demonstrations. - -Build order, which is part of the deliverable: -1. Board **with its interaction contract** — FR-U10–U12 land *with* FR-U4, - not after it. A board that renders before it reconciles is a board that - will be shipped that way. -2. Work item detail (FR-U5, U17, U20). -3. Backlog and bugs (FR-U6, U14, U15). -4. Project pages and the drift inbox (FR-U7, U18). -5. Global surfaces — audit browser (FR-U19), notification inbox, admin. -6. Palette and keyboard pass (FR-U16, U22). -7. Absent-state pass (FR-U21). - -The parity rule carries over and gets stronger (FR-U8): every URL in the -shipped bundle resolves against the operation registry *and* the engine's -API contract. r6's rule binds the new surfaces unchanged — a mutating -operation appears only through a view that collects a reason the user typed, -which is why quick-create (FR-U15) will not submit without one and why the -command palette opens that view rather than executing the write. - -**Demo**: every operation the legacy GUI exposed is reachable in the PWA; a -board drag round-trips `work.transition`, *including* a rejected transition -rolling back visibly with the server's stated reason; a board URL carrying -filters restores its exact view after reload; killing the engine mid-demo -disables session controls with a named reason while every Vogt view keeps -answering. - -### M11 as built — automated GUI evidence and remaining acceptance work - -The PWA now has a Playwright Chromium suite in `web/tests/browser/gui.spec.ts`. -It covers a real desktop Board drag/drop gesture, filter URL reload, Inbox -evidence and reasoned actions, labelled phone navigation and Go to reachability; -the jsdom suite also covers migration, keyboard/batch Inbox decisions and the -Sessions exact-payload approval review. What remains is the live-server -refusal round-trip, a large-estate browser fixture, the full route sweep and -the live-stack observation recorded in the gap register. - -That is also why the legacy GUI is still here. FR-U9's condition — every -operation it exposes is rendered by a PWA surface — was reached, and a test -now asserts it stays reached. Removing a verified front end in favour of an -unverified one is not what the requirement was asking for. `test_pwa.py` -carries the reminder and the order of operations: run the demo, then delete -`src/vogt/gui/`, the `/ui-legacy` routes and both tests together. - -1. **Parity was nearly declared by a spelling.** The first parity check - compared route *tables*: the client listed `notifications` and no surface - asked for it, and the check said parity was met. Parity now means a - surface calls the operation. The general form is worth keeping: a - requirement about what a user can reach, tested against what a client - could theoretically call, is satisfied by writing a constant. - -2. **The client's types were wrong in three ways, and every surface had - quietly routed around them** — a `swept_at` the server never sends, edge - pairs where the server sends adjacency, `Record` where - the server has always been precise. Each surface had written a defensive - parser, and each parser hid the mismatch instead of surviving it. The - types now come from `models.py` and the parsers are gone. - -3. **Deep links did not work at all** until the surfaces were integrated: - routes and tab kinds existed with no branch in the URL→tabs effect, so a - cold load opened nothing and the tabs only appeared because the shell - persists them. Activating a tab also dropped its query string, which on - these surfaces *is* the view. Two surfaces had independently grown a - workaround for that. - -4. **Six columns in the legacy GUI had been em dashes on every row**, found - while building their replacements: field names that were never right, and - two columns (`ecosystem`, `constraint`) removed from the product by r2 - along with lockfiles. Nothing failed, because an em dash is also how that - GUI renders "not collected" — so a typo and an honest absence looked - identical in the one product whose argument is that you can tell those - apart. `tests/test_gui.py` now reads the accessors off the models. - -5. **Deliberate readings, recorded so they are decisions.** A board drop - pre-fills the reason with the last one the server accepted; r6 permits it - — "a form with a required field" — and the alternative considered, one - armed reason per session, is how fifty audit rows end up saying "triage". - The board caps rendered cards per column with an explicit "+N more" - rather than virtualizing, and pages its reads with a truncation banner; - NFR-S5's "does not fetch the estate to render a page" holds, true - virtualization does not. Bulk drift accept does not exist, by §3. - -**Open after M11** — the live/manual browser conformance evidence the gap -register describes, and `backlog`/`bugs` having no `offset`, so there is no -way to page past the top 200 of a ranked view (**FR-V5**). - -The third said comments were audited against the comment rather than the work -item, so a per-item audit filter would miss them. **That was fixed and this -paragraph was not.** `declared.py`'s trail query is a semi-join over the item's -comment ids, and `tests/test_audit_query.py` asserts both directions — a -comment appears in its own item's trail, and stays out of another's. The stale -note survived long enough to have a requirement minted from it in r11, which is -the cost of a roadmap paragraph that outlives the thing it describes. - -## M12 — AI layer & voice validation (M–L) - -**Objective**: the assistant learns the Vogt domain — and voice, adopted -unproven, is put through its paces rather than assumed. - -Delivers FR-T1–T4, FR-T6 and the existing pending-action presentation in -FR-T8; FR-T5 is validated and FR-T7 attempted. The approval extension stays -inside the engine's one in-memory gate and does not add a core approval -ledger. - -Deliverables: -- **Registry-derived read tools** (FR-T1): the curated read slice generated - from the operation registry, not hand-written, so a new operation does not - mean a new hand-maintained schema. -- **The gated write set** (FR-T2): every mutating tool through the - pending-action gate — one at a time, exact payload shown, expiring - unapproved, approved only on screen. Voice never approves — a stated - non-requirement, not an omission. -- **Honest attribution** (FR-T3): an assistant-initiated write is audited to - the approving user's actor with a `why` from the conversation. -- **Threat-model extension** (FR-T4): Vogt reads are external content by the - assistant's own rule — issue titles and imported forge text get the same - untrusted-data delimiting terminal output already gets. -- **Provider cleanup** (FR-T7): a native Anthropic path, or the documented - `claude-*` hang resolved, or the route refused with a named reason. -- **The voice shakedown** (FR-T5): a deliberate validation pass against - domain vocabulary — project names, "backlog" — because "it has a mic" is - not evidence. - -**Demo**, by voice on the APK: ask for the top bug, hear the answer, start a -session on it, approve by on-screen tap — and confirm that saying "approve" -does not. - -### M12 as built — three notes - -1. **The tool schemas are fetched, not written.** The core already generates - MCP tool schemas from its operation registry, so the assistant asks it — - `tools/list` over JSON-RPC, `inputSchema` forwarded verbatim — instead of - keeping a second copy that drifts. Curation is an intersection against a - named read set, and a curated name the core does not serve is logged and - skipped rather than invented. What is deliberately *not* taken from the - core is which tools mutate: that comes from a local write set, so the - approval gate never depends on a remote answer. - -2. **The credential is the approver's, not the proposer's.** FR-T3 says an - assistant write is audited to the approving user's actor, and the only way - that is true by construction is to take the front-door pairing from the - request that pressed approve. A write has no shared fallback at all: an - unpaired approver is refused by token name, because falling back to a - deployment-wide actor is exactly the "shared assistant actor" the - requirement forbids. - -3. **Voice is still unproven, and is now written down as such.** FR-T5 asks - for a validation pass against domain vocabulary before v2 ships; it needs - a device and a microphone, and neither exists here. What was done instead - is smaller and honest: the prompt states that items are `WI-7` and - projects are slugs, which is what a recognizer's output has to survive. - -4. **FR-T7 was half attempted, and this note used to say it was not** - *(corrected r11, settled r12)*. The requirement offered two ways out of the - `claude-*` hang — a native Anthropic backend, or the route refused with a - named reason — and the second is built: `assistant.rs` refuses a `claude-*` - model id on the OpenAI-compatible transport with a sentence naming the - model, the transport and the setting that overrides it, and - `assistant_allow_claude_proxy` lets a deployment whose proxy serves those - routes correctly say so and own the result. "Not attempted" understated the - work by exactly the amount that would have caused somebody to do it again. - - **The other half is now deferred rather than outstanding (r12.)** The hang - was the failure that mattered and it is closed; a second transport buys a - choice of vendor rather than a capability, and nothing in v2 depends on - Anthropic specifically; the assistant speaks to any OpenAI-compatible - chat endpoint (`ENGINE_ASSISTANT_BASE_URL`, `ENGINE_ASSISTANT_API_KEY`, - `ENGINE_ASSISTANT_MODEL`). r12 carries the argument. So M12 delivered FR-T7 as it now stands, and this stage has - nothing left open against it. - -## M13 — Mobile MVP1 (S–M) - -**Objective**: the phone becomes a real surface — the one where an agent -waiting for input actually reaches you. - -Delivers FR-M1–M5, FR-E6 and FR-E7. The four primary places are Sessions, -Inbox, Board and Backlog; secondary routes remain reachable through the -labelled navigation control described by `RESTRUCTURE.md` Stage 9. - -Deliverables: -- The Capacitor shell repointed at the merged PWA (FR-M1), APK CI on the dev - stream, phone-width pass with the board rendering as a list below the - narrow breakpoint (FR-M3). -- **Push routing worth an interruption** (FR-M2): `waiting-for-input`, - `errored`, new drift, and the agent-task notify hook — and nothing else by - default, because a notification channel that cries wolf is uninstalled. -- **Session outcomes as observations** (FR-E6): exit code, duration, and the - working-tree delta the session left behind, carrying freshness and trust - like all other evidence. -- **Bound agent tasks** (FR-E7): a scheduled task may name a project or work - item and file its findings as observations rather than only as a push. - -**Demo**, from the phone: receive a push that a session is waiting for input, -open it, unblock it. Then, **not from the phone**: `vogt observations list ---kind session.outcome` (or `--kind agent_task.run`), which is where the -outcome lands and the only place it is readable. The original sentence said -"see the session's outcome land as an observation against the work item", and -no surface can show that: `vogtApi.ts` has no `observations` binding, so no -Vogt surface in the PWA reads the observed store at all, and the item page's -"Collected evidence" panel is the ranking's contributions rather than -evidence. The evidence is real, tested and CLI-only — the delivery -verification carries the row (FR-U17's provisional clause) and this line is -not allowed to imply otherwise. - -### M13 as built — three notes and a thing that was already wrong - -**The APK assembles; nothing was ever installed.** The first version of this -note said there was no Android SDK in the build environment. That was wrong — -the SDK, Gradle 9.7 and Java 21 were present, and `cap sync` plus -`./gradlew assembleDebug` produce a 5.5M debug APK whose manifest reads -`application-label:'Vogt'`, pointing at the configured server URL with -cleartext off. So the build is proven and the -configuration's branches are proven against a real Gradle run rather than -only under node. - -The build is now also proven where it counts: `the Android shell assembles` -passed on a self-hosted runner on 2026-08-14, in the job gated behind the -engine job, so the first green engine run was also the first APK this -pipeline has ever produced. Local Gradle was validation; the runner is the -build (that is the standing rule for everything here). - -What is still unproven is everything after the build: **no APK has been -installed on a device, and no notification has been delivered by either -transport**. The drift path has never run against a live core. Release -signing is also still absent — the keystore lives in the retired forge — so -what CI produces is an unsigned debug artefact. - -1. **The shell was pointing at the wrong stack, and it looked like it - worked.** `capacitor.config.ts` hardcoded the hostname of the - standalone engine, with no vogt-core behind it. An APK built on that - default reaches a front door where `/api/vogt` answers 503, `vogt. - configured` is false, and the four Vogt tabs are hidden or in their outage - state, while terminals and the assistant work perfectly. The build now - asks for the URL and fails without one, because the merged stack has no - settled name and NFR-D2 forbids inventing one. `cleartext` follows the - scheme rather than being hardcoded `false`, which a plain-HTTP deployment - on a private network had turned into a runtime network error. - -2. **FR-M2's "and for nothing else by default" was already false.** Idle-stall - and agent-task-started notifications both defaulted on, and neither is in - the set the requirement names. Both default off now; the watchers still - run and the toggles still exist, and a device that already made a choice - keeps it. This is the requirement being read as a *list* rather than as a - direction, which is what it says. - -3. **One reader of the core's event cursor, and drift push hangs off it.** - The follower (M11, FR-U10) already polls `events.list` and republishes each - change onto the server's bus; the drift watcher subscribes to that bus the - way the session watcher subscribes to activity, and adds only the filter - and the fan-out. It fires on `drift.raised` — a named kind, not a `drift.` - prefix, because "and for nothing else by default" has to survive the core - growing kinds — and coalesces a ten-second window, so a sweep raising - thirty proposals is one notification rather than thirty. - - This was built twice. The first attempt was a second poller with its own - persisted cursor, which is strictly more machinery for the same - notification and was replaced. What the surviving design costs is stated - rather than hidden: the follower's cursor is in memory, so drift raised - while the engine was down is never notified. A redeploy is a hole in the - stream. The proposal is not lost — it stays open in the inbox until - somebody rules on it — so what is missed is the interruption, not the work, - and the alternative buys that back at the price of a phone that can replay - an estate's history after a restart. A missed buzz is recoverable by - opening the app; a channel someone switched off is not. - -**The breakpoint the surfaces had all missed**: the shell goes to phone -layout at 768px and each of the five Vogt surfaces had picked 900, leaving a -band where the shell was a phone and the surface inside it was not. Phone -work is at 768 now, and a test fails if a sixth breakpoint narrower than -that appears. - -## M14 — Consolidation (S, ongoing) - -**Objective**: finish the merge rather than leave it half-standing. Delivers -no new requirement ID, which is the point. - -Deliverables: -- The standalone stacks (the old engine-only and core-only deployments) - retired once the merged stack has carried the load. -- The name and domain decision, and the sunset of the legacy `MYDEVENV2_*` - config aliases (still accepted as aliases for the `ENGINE_*` names) after - their transition period. -- **The r9 as-built reconciliation**: the requirements baseline and - `DESIGN.md` brought back against what was actually built, in the usual - style — including a delivery verification of every ID r9 added, with - each conjunct of a requirement's text judged separately. *(Done: the v2 - delivery verification, and `DESIGN.md` §4's merged-tree section.)* -- **NFR-I6** — backup and restore covering the whole product as one act: the - core's SQLite, the engine's `state_dir`, and enough metadata to - re-establish FR-E3's path agreement after a restore. *(Built. It had - appeared in no milestone's `Delivers` line at all, which is why nobody had - built it — a §4 traceability failure the per-conjunct audit found. It - lands here because retiring the standalone stacks is exactly when somebody - needs to restore one.)* - -**Demo**: there is none, and that is honest — the acceptance test for this -stage is the delivery verification agreeing with the build, which is the -same bar v1 was held to. - -### M14 as built, so far — what the pipeline settled - -**The merge reached `main` on 2026-08-14**, 328 commits fast-forwarded after -a green `dev`, which is NFR-D12's route taken rather than described. Both -image streams exist and are signed: `dev`/`dev-` on the dev branch and -`sha-` from main, in `vogt-stack` beside the core-only `vogt`. - -Three things worth recording, because none of them was visible before CI ran -for the first time and all three had been asserted in prose: - -1. **Two first-run failures, both checks written against this machine.** - `docs` failed on a link that resolved here because the file exists on this - filesystem; `check_docs.py` now rejects absolute paths outright. The engine - job failed on `sudo apt-get` with *root is not in the sudoers file* — the - runner is root, so there was nothing to elevate from. Neither was a code - defect and neither was findable by reading. -2. **"Needs hardware" was wrong, and is retired from the documents.** It - described this container. The runners have the Docker daemon and the - Android SDK it lacks, and on the same day they produced the merged image - and an assembled APK. What genuinely remains needs a phone, a speaker, a - browser with a layout engine, and somebody choosing to deploy. -3. **The APK assembles in CI**, in the job gated behind the engine job that - was failing — so the first green engine run was also the first APK the - pipeline has ever built. It still points at `127.0.0.1:8910` and is still - signed with Gradle's debug key. -4. **The image jobs are rate-limited by Docker Hub, and that is now the only - thing standing between this branch and a fully green pipeline.** Every base - image in both Dockerfiles is pulled anonymously from Docker Hub; a - self-hosted runner is one IP for every build this repository makes; an - afternoon of merged-image builds exhausts the per-IP allowance, and the - next build fails at `FROM` with `429 Too Many Requests` before a line is - compiled. It failed twice, including a deliberate retry. - - `build.yml` and `release.yml` now log in when a `DOCKERHUB_USERNAME` / - `DOCKERHUB_TOKEN` secret exists and emit a notice when it does not, so the - failure names itself. **Adding the secret is a decision for the operator, - and so is the alternative**: mirroring the bases into GHCR, which every job - already authenticates to, and which would remove the dependency entirely - at the cost of moving the digests this repository pins. - - *Settled since, the other way round (#33).* The secret was added and the - build 429'd anyway: the credential is estate-wide, so its 200/hour is - shared with every other project's CI and is empty exactly when the estate - is busy, while this repository's own per-IP bucket was at 99 of 100 - remaining. Authenticating is a *worse* position, not a better one, and - both logins have been removed rather than left inert for somebody to - helpfully re-enable. The alternative was taken instead: both Dockerfiles - build `FROM` `ghcr.io/thedancingdeveloper-org/vogt-base/*`, which - `.github/workflows/mirror-base-images.yml` fills by copying the manifest - lists verbatim. The digests this repository pins did not move — a copied - manifest keeps its digest, and the mirror job fails if it does not. - -5. **A cancelled run was a check that never ran.** Path gating classifies a - push by `before..sha`, so each commit is examined by exactly one run — and - `cancel-in-progress` was true for every trigger, so pushing twice within a - few minutes cancelled the first run and the next run's range began after - it. A lint error reached `dev` and sat there green; it was found by - running `ruff` by hand. Cancellation is now pull-requests-only, whose runs - classify against the merge base. - -Still open, and each is somebody's decision rather than unfinished work: the -standalone stacks are retired only after the merged stack carries load; -the `MYDEVENV2_*` aliases sunset after their transition period; and -the APK keystore lives in the retired forge, so where a signed APK is -published is an untaken decision. - ---- - -## Revision r5 — a build is not a release - -NFR-C3 said a push to main shall never publish an image, so the only way to -obtain a deployable artefact was to tag a version. Within one afternoon that -produced v0.1.0, v0.1.1 and v0.1.2 — none of which marked a release. Two were -repairs to an image that had never been executed, and the third was a uid -change that concerns one deployment and no user of this software. - -Version numbers had become a build counter, which is the failure mode where a -number that should mean "this is what changed for you" comes to mean "the -pipeline ran again". - -The requirement was conflating two acts that only look alike: - -| | Release | Build | -|---|---|---| -| Trigger | `v*` tag | push to main | -| Image tags | `0.1.2`, `0.1`, `latest` | `sha-` | -| Wheel, SBOM attestation | yes | image SBOM only | -| Signed | yes | yes | -| Says "use this" | yes | no | - -What NFR-C3 was protecting is intact. Merging still cannot cut a release: no -semver tag, no `latest`, no wheel. Merging still cannot deploy: production -moves only when an operator pins a new digest and redeploys the stack -(NFR-D10). What it loses is the accidental coupling that made a version bump -the price of a hotfix. - -Commit builds are signed like releases. An unsigned artefact that can reach -production would be a wider hole than the one this closes. +# Roadmap + +The product described in [`DESIGN.md`](DESIGN.md) is built and shipping: the +Python core, the Rust engine, the Solid PWA, and the Capacitor mobile shell, +released as versioned images from this repository. + +The authoritative queue of live work is the GitHub issue tracker. This file +holds only the standing items that are *designed but not built*, or +deliberately deferred — the things an issue alone would not explain. When a +change defers something that was designed, record the deferral here; when a +deferral becomes an issue with a plan, link it and keep the one-line entry +until it ships. + +## Designed but not built + +- **Forge write-back beyond comments.** The core posts comments and performs + opt-in write-back for the linked forge, but fuller bidirectional sync + (historical backfill, forge-derived drift acceptance) is designed and + deferred. Tracked in the issue tracker as it is picked up. +- **Native Anthropic assistant backend.** The assistant speaks to any + OpenAI-compatible chat endpoint; a native Anthropic backend is deferred. + +## Pending cleanup + +- **Legacy `MYDEVENV2_*` names.** The engine still reads `MYDEVENV2_*` + environment aliases, and a handful of internal identifiers (helper binary + names, browser-storage keys, an Android notification channel) carry the + pre-merge name. No released installation depends on them, so removing them + is ordinary housekeeping — new configuration and examples use `ENGINE_*` / + `VOGT_*` names only. + +## Deliberate deferrals + +- **Fork-runnable CI (#207).** Every workflow job names a self-hosted runner + by policy; a fork cannot run hosted CI as-is. The fork-safe local checks in + [`CONTRIBUTING.md`](CONTRIBUTING.md) are the supported path until #207 + lands. +- **Device-dependent conformance.** The voice and mobile-push passes that + need a physical phone and speaker are run per release by the maintainer + rather than in CI. diff --git a/docs/SCHEMA.md b/docs/SCHEMA.md index 930eb788..f138143b 100644 --- a/docs/SCHEMA.md +++ b/docs/SCHEMA.md @@ -1,34 +1,9 @@ -# Vogt — Data Schema & Topology (v0.3, revision r5) - -Status: **built** (reconciled against the delivered v1 on 2026-08-12; the -as-built shape of §3.2 is the note at the end of that section, and the -requirement-by-requirement verification lives with the requirements -baseline, outside this repository; `FR-*`/`NFR-*` IDs are quoted as plain -text). -Types are indicative; DDL is written -at M0. Companion to `DESIGN.md` §3 (domain model) and §7 (storage). - -r2 changes: `contracts` and `packages` tables removed; `events` and -`suppressions` tables added; dependency tables reduced to references; -drift proposals carry an evidence snapshot; retention rules tightened. - -r3 changes: no sweep roots and no discovery — collector scope is always a -set of registered project ids; `projects.exclusions` replaces per-root -exclusion patterns; contract results are written by on-demand checks only. - -r4 changes: `coding_sessions` added (§2.6) for the merge with the session -engine — the declared link between a work item or project and a terminal the -engine runs for it. No other table changed; the engine keeps its own state in -its own `state_dir` and nothing about it is mirrored here. - -r5 changes: **no schema change at all.** Session outcomes and bound -agent-task runs (FR-E6, FR-E7) arrive as observations of two new kinds, -`session.outcome` and `agent_task.run`, against two new subject-key shapes -(§3.1) — which is what r4's §2.6 said would happen and what nothing had -built. This revision is recorded because the previous one described that -mechanism as existing: a document that reasons from an unbuilt thing to a -schema decision is the failure the v1 delivery verification found in this -file, and r5 is the correction as well as the build. +# Vogt — Data Schema & Topology + +Companion to `DESIGN.md` §3 (domain model) and §7 (storage). Types are +indicative; the migrations under `src/vogt/storage/` are authoritative. +`FR-*`/`NFR-*` IDs refer to the numbered requirements baseline, which is +maintained outside this repository and quoted here as plain text. ## 1. Storage topology @@ -193,7 +168,7 @@ not yet an input to the phase. `writeback_actions` records the *attempt*, not only the success — including `skipped`, which is what a policy refusal looks like. Write-back is never a -separate operation (`ROADMAP.md` M5): a comment authored here posts upstream +separate operation: a comment authored here posts upstream as part of commenting, so this table is the only place the upstream half of a declared write is visible before the next sweep re-observes it. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index b14f8655..e6170b12 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -17,14 +17,11 @@ are covered in [`GETTING_STARTED.md`](GETTING_STARTED.md) and [`DEPLOYMENT.md`](DEPLOYMENT.md); every environment variable is listed in [`CONFIG.md`](CONFIG.md). -The engine was once a separate product, and a few deliberately invisible -compatibility identifiers keep the historical name: browser storage keys, some -Rust crate and config names, the Android package and notification-channel IDs, -and the agent-task notify phrase (§2.7). Legacy `MYDEVENV2_*` environment -names are still accepted as aliases for `ENGINE_*`. Renaming those would -discard local preferences, break push delivery, create a second Android app or -invalidate existing task definitions, so they stay; every user-facing label -says Vogt. +A few internal identifiers — some browser storage keys, Rust crate and +config names, the Android notification-channel ID, the agent-task notify +phrase (§2.7), and the `MYDEVENV2_*` environment aliases for `ENGINE_*` — +still carry a legacy internal name. Their removal is pending housekeeping +([`ROADMAP.md`](ROADMAP.md)); every user-facing label says Vogt. Status: **current as of 2026-08-30**, and describing one product. @@ -32,8 +29,9 @@ Status: **current as of 2026-08-30**, and describing one product. implemented and covered by tests. The Solid surfaces have jsdom coverage against a fake front door and automated Chromium coverage for representative desktop and phone routes, including Sessions composition, terminal splitting, -zoom-event ownership and dirty-editor lifecycle. A human/device M13 demo is -still outstanding and `ROADMAP.md` says so. +zoom-event ownership and dirty-editor lifecycle. The device-dependent voice +and push passes run per release on real hardware rather than in CI +([`ROADMAP.md`](ROADMAP.md)). --- diff --git a/docs/VOICE_DELIVERY.md b/docs/VOICE_DELIVERY.md deleted file mode 100644 index fab560b5..00000000 --- a/docs/VOICE_DELIVERY.md +++ /dev/null @@ -1,105 +0,0 @@ -# Voice assistant — delivery status and remaining scope - -*2026-08-26. Companion to [`VOICE_POC.md`](VOICE_POC.md) (the POC's design -and findings) and [`ENGINE.md`](ENGINE.md) §6 (the assistant's reference, -including **Configuring the assistant provider**). The `FR-Txx` / `FR-Mxx` -ids are stable requirement identifiers; each rule they label is stated in -words. This file records what is delivered, how to enable it, and what -remains — each remainder tracked as a GitHub issue under the tracker -[#188](https://github.com/TheDancingDeveloper-org/vogt/issues/188).* - -## 1. What is delivered - -**Checkpoint A of the POC is built and tested** (`VOICE_POC.md` §6 is the -evidence, with per-piece test counts): - -- Provider profiles (FR-T9): named OpenAI-compatible `{base_url, key, model, - effort}` sets — `engine/server/src/{config,assistant,assistant_api,gui}.rs`. -- `notifications` tool over the curated `inbox.list` projection (FR-T10), with - coverage spoken as part of the answer. -- `model` / `effort` on `session.start` with registry parity, the scratch - project for subject-less requests, and argv validation of model ids - (FR-T11) — core `models.py`/`sessions.py`, migration `0010`, - `engine/server/src/agent_cli.rs`. -- The vocabulary repair pass (FR-T13's middle): `web/src/voiceRepair.ts`, - single-word fuzzy slug match + `WI-\d+` repair, shown before send. -- The Assistant surface: `web/src/Assistant.tsx`, a tab the shell shows only - when `/api/config` says `assistant_enabled` (FR-T6). -- The FR-T2 gate holds against spoken approval; both mutating journeys cost - exactly one tap. - -**Delivered since** (the implementation work is complete; the remaining -external gate is the real-device coexistence test tracked by #191): - -- Desktop microphone via Web Speech where the browser has it (#189). -- Server-side STT/TTS (FR-T12, #190): `POST /api/assistant/stt` / `tts` - proxied to OpenAI-compatible audio endpoints — - `engine/server/src/assistant_speech.rs`. -- The durable, attributable interaction log (FR-T14, #193) — - `engine/server/src/assistant_log.rs`. -- The phone's foreground service and speak-the-push (FR-M6, #192). The - implementation is delivered; the remaining device validation is the - dev/prod FCM coexistence test tracked by #191. - -**Voice input without any of that is mobile-only.** The on-device microphone -is the Capacitor `SpeechRecognition` plugin; a desktop browser without Web -Speech and without a configured server STT gets typed input plus spoken -replies (Web Speech synthesis). - -**[voicemode](https://github.com/mbailey/voicemode) is a pattern, not a -dependency.** FR-T12 adopts its architecture — STT/TTS behind -OpenAI-compatible audio endpoints so cloud and local Whisper.cpp + Kokoro -are interchangeable by configuration. - -## 2. Enabling it - -The assistant is off until a chat provider is configured, and the PWA hides -its tab until then. Any OpenAI-compatible chat endpoint works; the three -settings that turn it on are: - -| Variable | Example | -|---|---| -| `ENGINE_ASSISTANT_BASE_URL` | `https://api.openai.com/v1`, or a local server such as `http://127.0.0.1:11434/v1` | -| `ENGINE_ASSISTANT_API_KEY` | the provider's key (any non-empty value for a local server that needs none) | -| `ENGINE_ASSISTANT_MODEL` | a model id the endpoint serves, e.g. `gpt-5.4-mini` | - -Server-side speech is separate and also off by default: -`ENGINE_ASSISTANT_STT_BASE_URLS` / `ENGINE_ASSISTANT_TTS_BASE_URLS` name -ordered lists of OpenAI-compatible audio endpoints (e.g. a local whisper/TTS -server first, a hosted one as fallback), with `_MODEL`, `_VOICE` and `_API_KEY` -beside them. The full table — every variable, its TOML key, default and -semantics, plus named provider profiles — is -[`ENGINE.md` §6, "Configuring the assistant provider"](ENGINE.md#configuring-the-assistant-provider). -Legacy `MYDEVENV2_*` names are accepted as aliases. - -Rollback is removing the variables and restarting. Two caveats a deployment -should expect, both carried by -[#194](https://github.com/TheDancingDeveloper-org/vogt/issues/194): a probe -with the *engine's* primary token can `401` on `/api/assistant/*` — that is a -capability question (the `assistant` capability), not proof of breakage — and -a shared provider key with a spend limit should be replaced with a dedicated -one before anyone relies on it. - -## 3. What remains - -Only [#191](https://github.com/TheDancingDeveloper-org/vogt/issues/191) is -still open in this delivery tracker. It requires an operator to install the -`com.sprooty.vogt.dev` APK beside `com.sprooty.vogt`, confirm both FCM -registrations, and verify that environment-specific pushes reach the intended -installation. The implementation issues #189, #190, #192, #193 and #194 are -closed; their code and validation records remain linked below for context. - -| Issue | Scope | Design source | Status | -|---|---|---|---| -| [#189](https://github.com/TheDancingDeveloper-org/vogt/issues/189) | Checkpoint B: desktop mic via Web Speech `webkitSpeechRecognition`, then the spoken five-utterance pass with findings into `VOICE_POC.md` §6 | `VOICE_POC.md` §3.4, §4–5; FR-T13 | Closed | -| [#190](https://github.com/TheDancingDeveloper-org/vogt/issues/190) | Checkpoint C: engine `POST /api/assistant/stt` / `tts` proxying OpenAI-compatible audio endpoints; MediaRecorder capture and `