feat: Frame templates

AGENTS.md

@@ -25,6 +25,7 @@ make dev           # backend :8080 (dev mode, fixture-seeded) + Vite :5173 with
 make dev-auth      # Keycloak in Docker :8081 + backend on :5173 serving the built SPA
 make dev-clean     # kill orphan dev servers, drop dev DB + WAL/SHM, tear down Keycloak
 make image         # docker build (linux/amd64)
+make e2e           # blackbox RPC suite against a deployed frames (needs FRAMES_E2E_BASE_URL + FRAMES_E2E_TOKEN)
 
 # One Go test / package
 go test ./backend/internal/frames -run TestResolve -race
@@ -45,8 +46,11 @@ CI (`.github/workflows/ci.yml`) gates on five jobs: `proto` (buf lint plus a sta
 `gen/`), `go` (golangci-lint pinned to v2.12 plus race tests), `web` (lint, typecheck, vitest),
 `chart` (helm lint, template renders, kubeconform pinned to v0.7.0), and `e2e-sandbox` (deploys the
 built image onto a kind Nebari sandbox via ArgoCD and exercises real Keycloak auth through the
-gateway). `docs.yml` also fails if the generated CLI reference is stale. Run the local
-equivalents before pushing.
+gateway). `e2e-sandbox` also provisions a dedicated Keycloak client and user, then runs the
+build-tagged blackbox RPC suite in `e2e/` (`make e2e`) against the deployed pod through the real
+gateway; that suite skips locally whenever `FRAMES_E2E_BASE_URL` and `FRAMES_E2E_TOKEN` are unset, so
+`make test` is unaffected. `docs.yml` also fails if the generated CLI reference is stale. Run the
+local equivalents before pushing.
 
 **Generated code is checked in.** After touching `proto/frames/v1/*.proto`, run `make proto` and
 commit `gen/`. After touching `cli/cmd/*`, run `go run ./tools/docs-gen` and commit
@@ -85,6 +89,22 @@ store.Repository**.
   - `slots.go` - `SlotTable` is the single source of truth for slot keys, markdown headings, and
     content shape (terms / list / prose). Add or rename a slot here only; the `.frame.md` codec and
     the MCP composer both read it.
+  - `templates.go` / `builtins.go` - Frame templates: an authoring affordance, not Frames and not
+    a Frame Spec concept. A template carries prefill content plus a per-slot rule (required /
+    recommended / optional, with a note). Built-ins are embedded YAML parsed and validated at
+    package init, so a malformed starter panics at startup rather than failing at a user's first
+    click. `Check` reports required slots a document leaves empty; `publish` merges those with the
+    schema's own violations so an author sees everything at once. Checked on create only: nothing
+    is recorded on the Frame, so `required` is an authoring aid rather than ongoing governance.
+    See `docs/adr/0001-frame-templates-are-not-frames.md`. A prefill is held to the same content
+    rules a Frame is (`contentErrors`), but at the write boundary (`validateTemplateInput`), never
+    in `ParsePrefill`: that decoder is also the read path (`rowToTemplate` runs every stored row
+    through it), so a rule enforced there would make an already-stored row unreadable rather than
+    merely unpublishable. Starters arrive through two doors - the write RPCs and `loadBuiltins` at
+    package init. Built-ins carry rules and notes but no content at all: `builtinFile` has no
+    `prefill` key, so there is nothing for a content rule to check there.
+    `TestBuiltinFileCarriesNoPrefill` is a tripwire over that premise - add such a field and it
+    fails, telling you to run `contentErrors` in `loadBuiltins` and to replace the tripwire.
   - `framemd.go` - the Frame Spec v0.2 `.frame.md` codec (YAML frontmatter plus one `##` section per
     slot). Round-trip fidelity matters: `examples/*.yaml` and `examples/*.frame.md` are checked-in
     conformance fixtures asserted by `examples_test.go`.
@@ -102,7 +122,10 @@ store.Repository**.
   cannot be shadowed by a case variant. The in-memory fake enforces the same unique constraints;
   where it cannot, tests reach for real SQLite and say why. Publishes go
   through `CreateFrameVersion`, which inserts the frame row, version, inheritance edges, and grants
-  atomically. **SQLite is single-writer, so the deployment is pinned to one replica.**
+  atomically.
+  `frame_templates` holds org-authored templates, with prefill and field rules as opaque blobs so
+  a change to `SlotTable` never touches the schema; built-ins are not rows.
+  **SQLite is single-writer, so the deployment is pinned to one replica.**
 - `backend/internal/mcp` is a thin protocol adapter over `frames.Service`, exposing frames as MCP
   resources under `nebari-frame://<org>/<name>[@<version>]` plus RFC 9728 metadata at
   `/.well-known/oauth-protected-resource`. It is also a write surface: `create_frame` and
@@ -122,6 +145,33 @@ store.Repository**.
   `//go:embed` still compiles on a clean checkout. Auth guards live in `web/src/app/`
   (`RequireAuth`, `RequireMembership`, `RequireAdmin`); pages in `web/src/pages/`; RPC transport and
   domain helpers in `web/src/lib/`.
+  - Seeding a form from server data comes in two shapes. A form that may re-seed keys its `reset`
+    on an identity that changes when the data does (the authoring page's edit path keys on
+    `version.digest`). A form that seeds once and then belongs to the author - a template prefill,
+    the template edit dialog - must pin `refetchOnMount: "always"` on the query and take its data
+    from `freshData()` (`web/src/lib/query-freshness.ts`), which is where the reasoning lives:
+    React Query returns a cached row synchronously and refetches behind it, counts a *failed*
+    fetch in `isFetchedAfterMount`, and never clears cached data on error. Where the latch lives
+    depends on where the form's identity comes from: the template edit dialog is keyed and mounted
+    per target, so a boolean is its identity, while the authoring page's template comes from
+    `?template=` and can change while the page stays mounted, so it latches on the id it seeded
+    from.
+  - A form seeded from a query does not render editable before the seed lands. Both template
+    surfaces withhold it (a skeleton, or the dialog's), because `reset` replaces form state
+    wholesale and `refetchOnMount: "always"` guarantees the seed arrives a round trip after the
+    first render - so anything typed in between would vanish.
+  - `react-hooks/set-state-in-effect` reports at most one violation per effect, so a second
+    `setState` in the same effect can look clean while being the same construct. Do not read the
+    linter's silence as a constraint, and do not add a `setTimeout` claiming to satisfy a rule
+    that never fired - an unused `eslint-disable` is itself reported.
+  - Two schemas cover the same slot keys on purpose. `frame-yaml.ts` decodes whatever is stored
+    (lenient, because rows predate rules); `contentSlotsSchema` in `authoring-schema.ts` is what a
+    human may submit, and every form that edits slot content resolves against it. The backend
+    splits the same two jobs the same way (`Parse` vs `contentErrors`).
+  - Error text splits by direction. A failed read states a fixed sentence - the server's message
+    there is storage or wiring detail and retrying is all the reader can do. A failed write shows
+    `ConnectError.rawMessage`, because it names what the caller must change; when it carries
+    `FieldViolations`, it belongs on the input instead.
 - `cli/` is the `frames` binary (Cobra plus Viper; config at `~/.config/frames/config.yaml`, env
   prefix `FRAMES_`, device-flow login).
 - `chart/` deploys onto a Nebari cluster; the nebari-operator provisions routing, TLS, and the OIDC
@@ -148,5 +198,5 @@ store.Repository**.
   and keep new comments in the same register.
 - Always stop a dev loop with a single Ctrl-C. Killing it leaves the SQLite lock held and the next
   start fails with `disk I/O error` / `database is locked`; `make dev-clean` recovers.
-- Design docs in `docs/design/`, client connection guides in `docs/connect/`, manual QA scripts in
-  `docs/qa/`.
+- Design docs in `docs/design/`, architecture decision records in `docs/adr/`, client connection
+  guides in `docs/connect/`, manual QA scripts in `docs/qa/`.

Makefile

@@ -1,4 +1,4 @@
-.PHONY: proto lint test build build-web run-dev dev dev-auth dev-clean clean image
+.PHONY: proto lint test e2e build build-web run-dev dev dev-auth dev-clean clean image
 
 # Local dev runner: builds the SPA into the binary, then starts the server in
 # dev mode (no OIDC). The auth interceptor injects a fixed dev-user identity;
@@ -16,10 +16,20 @@ proto:
 
 lint:
 	golangci-lint run ./...
+	# e2e is built with a `//go:build e2e` tag so it stays out of `go build ./...`
+	# and out of a plain lint run; without this second invocation nothing ever
+	# lints it and it can silently rot.
+	golangci-lint run --build-tags e2e ./e2e/...
 
 test:
 	go test ./... -race -coverprofile=coverage.out
 
+# Blackbox suite against a deployed frames. Needs FRAMES_E2E_BASE_URL and
+# FRAMES_E2E_TOKEN; skips loudly without them. Not part of `make test`, which
+# must stay runnable with no cluster.
+e2e:
+	go test -tags e2e -count=1 -v ./e2e/...
+
 build:
 	CGO_ENABLED=0 go build -o nebari-frames-server ./backend/cmd/server
 

backend/internal/frames/builtins.go

@@ -0,0 +1,130 @@
+package frames
+
+import (
+	"embed"
+	"fmt"
+	"sort"
+	"strings"
+)
+
+// Built-in templates are compiled in rather than seeded into the database.
+// Seeding them per org would mean N rows per org and reconciliation on every
+// upgrade, and a shared "system org" is unreadable by design: rbac.Can opens
+// with a cross-org deny, so no caller could see one. See ADR 0001.
+//
+// The definition files are our own format, deliberately not .frame.md: that is
+// the external interchange shape framemd.go exists to translate, and a
+// frame-spec revision moving a heading must not be able to break a starter
+// document that has no reason to care.
+
+//go:embed builtins/*.yaml
+var builtinFS embed.FS
+
+// builtinFile is the on-disk shape of one definition. There is no `prefill`
+// key: built-ins carry rules and notes only, because shipping example prose
+// that every author has to delete makes the blank page worse rather than
+// better. Prefill is for org templates. Add the key when a built-in genuinely
+// needs it.
+type builtinFile struct {
+	ID          string               `yaml:"id"`
+	Title       string               `yaml:"title"`
+	Description string               `yaml:"description"`
+	FieldRules  map[string]FieldRule `yaml:"field_rules,omitempty"`
+}
+
+var builtins []Template
+
+func init() {
+	loaded, err := loadBuiltins()
+	if err != nil {
+		// A malformed starter must fail at startup, not at a user's first click
+		// on the picker. The files are compiled in, so a failure here is a build
+		// defect with no runtime recovery - the same reasoning as the
+		// package-level regexp.MustCompile calls in validate.go.
+		panic("frames: loading built-in templates: " + err.Error())
+	}
+	builtins = loaded
+}
+
+func loadBuiltins() ([]Template, error) {
+	entries, err := builtinFS.ReadDir("builtins")
+	if err != nil {
+		return nil, err
+	}
+	out := make([]Template, 0, len(entries))
+	seen := map[string]bool{}
+	for _, entry := range entries {
+		name := entry.Name()
+		b, err := builtinFS.ReadFile("builtins/" + name)
+		if err != nil {
+			return nil, err
+		}
+		var f builtinFile
+		if err := unmarshalStrict(b, &f); err != nil {
+			return nil, fmt.Errorf("%s: %w", name, err)
+		}
+		if !IsBuiltin(f.ID) {
+			return nil, fmt.Errorf("%s: id %q must start with %q", name, f.ID, BuiltinPrefix)
+		}
+		if seen[f.ID] {
+			return nil, fmt.Errorf("%s: duplicate id %q", name, f.ID)
+		}
+		seen[f.ID] = true
+		if strings.TrimSpace(f.Title) == "" {
+			return nil, fmt.Errorf("%s: title must not be empty", name)
+		}
+		if strings.TrimSpace(f.Description) == "" {
+			return nil, fmt.Errorf("%s: description must not be empty", name)
+		}
+		for key, rule := range f.FieldRules {
+			if _, ok := slotByKey(key); !ok {
+				return nil, fmt.Errorf("%s: field_rules names unknown slot %q", name, key)
+			}
+			if strings.TrimSpace(rule.Note) == "" {
+				return nil, fmt.Errorf("%s: field_rules[%s] has no note", name, key)
+			}
+		}
+		out = append(out, Template{
+			ID: f.ID, Title: f.Title, Description: f.Description, FieldRules: f.FieldRules,
+		})
+	}
+	// Blank leads: it is the "no template" escape hatch and must be easy to
+	// find. Everything else keeps filename order, which is alphabetical by
+	// title. SliceStable so that order is preserved rather than reshuffled.
+	sort.SliceStable(out, func(i, j int) bool {
+		return out[i].ID == BlankTemplateID && out[j].ID != BlankTemplateID
+	})
+	return out, nil
+}
+
+// BuiltinTemplates returns the templates compiled into the binary, Blank first.
+// Each call gets its own copy, rules map included: every request reads these, so
+// a caller must not be able to mutate the package's copy.
+func BuiltinTemplates() []Template {
+	out := make([]Template, 0, len(builtins))
+	for _, tmpl := range builtins {
+		out = append(out, copyTemplate(tmpl))
+	}
+	return out
+}
+
+// BuiltinTemplate looks one up by ID, reporting whether it exists.
+func BuiltinTemplate(id string) (Template, bool) {
+	for _, tmpl := range builtins {
+		if tmpl.ID == id {
+			return copyTemplate(tmpl), true
+		}
+	}
+	return Template{}, false
+}
+
+func copyTemplate(in Template) Template {
+	out := in
+	if in.FieldRules != nil {
+		out.FieldRules = make(map[string]FieldRule, len(in.FieldRules))
+		for key, rule := range in.FieldRules {
+			out.FieldRules[key] = rule
+		}
+	}
+	return out
+}

backend/internal/frames/builtins/blank.yaml

@@ -0,0 +1,5 @@
+# Blank is a real template rather than a special case in the picker, so that
+# starting from nothing takes the same code path as starting from a starter.
+id: builtin:blank
+title: Blank
+description: Start from an empty document and add only the sections you need.

backend/internal/frames/builtins/brand-voice.yaml

@@ -0,0 +1,13 @@
+id: builtin:brand-voice
+title: Brand Voice and Style
+description: How your organization sounds in writing, and the words it does and does not use.
+field_rules:
+  style:
+    level: required
+    note: Voice, tone, and formatting. Say what to do, not only what to avoid, and give an example of each.
+  terminology:
+    level: recommended
+    note: Product and brand names with their correct spelling and casing, plus any word you deliberately avoid.
+  rules:
+    level: recommended
+    note: Hard constraints, one per entry, that an assistant must never break when writing as your organization.

backend/internal/frames/builtins/business-process.yaml

@@ -0,0 +1,13 @@
+id: builtin:business-process
+title: Business Process
+description: How a piece of work moves through your organization, stage by stage.
+field_rules:
+  business_process:
+    level: required
+    note: The stages work passes through, who owns each, and what has to be true before it moves on.
+  terminology:
+    level: recommended
+    note: Names for stages, artifacts, and roles, so an assistant uses your words rather than generic ones.
+  goals:
+    level: recommended
+    note: What a good outcome looks like at the end of this process.

backend/internal/frames/builtins/domain-vocabulary.yaml

@@ -0,0 +1,10 @@
+id: builtin:domain-vocabulary
+title: Domain Vocabulary
+description: The terms of art your organization uses and what they actually mean here.
+field_rules:
+  terminology:
+    level: required
+    note: One entry per term. Define it the way an insider would explain it to a new colleague, not the way a dictionary would.
+  rules:
+    level: recommended
+    note: Usage constraints worth stating, such as a term that is often confused with another.

backend/internal/frames/builtins/engineering-norms.yaml

@@ -0,0 +1,13 @@
+id: builtin:engineering-norms
+title: Engineering Norms
+description: How your team expects software to be built, reviewed, and shipped.
+field_rules:
+  norms:
+    level: required
+    note: "Ways of working: review expectations, testing standards, what \"done\" means, and how work gets committed."
+  rules:
+    level: recommended
+    note: "Hard constraints, one per entry, that must hold regardless of context."
+  architecture:
+    level: recommended
+    note: "The structure worth knowing before changing anything: services, boundaries, and what depends on what."

backend/internal/frames/builtins/product-context.yaml

@@ -0,0 +1,16 @@
+id: builtin:product-context
+title: Product Context
+description: What you are building, who it is for, and what success looks like.
+field_rules:
+  goals:
+    level: required
+    note: What good outcomes look like for work done in this product, and how you would know you got one.
+  architecture:
+    level: recommended
+    note: The system's shape at the level a newcomer needs before touching it.
+  terminology:
+    level: recommended
+    note: Product-specific names for features, objects, and user types.
+  business_process:
+    level: recommended
+    note: How a change reaches users, if that is not obvious.