diff --git a/.claude/commands/generate-feature-docs.md b/.claude/commands/generate-feature-docs.md new file mode 100644 index 000000000..17073181a --- /dev/null +++ b/.claude/commands/generate-feature-docs.md @@ -0,0 +1,7 @@ +Generate publisher-facing documentation from an implemented engineering spec. + +Spec path: $ARGUMENTS + +Use the `generate-feature-docs` skill at `.claude/skills/generate-feature-docs/SKILL.md` to perform this task. The skill runs in two interactive stages (extraction pass for outline review, generation pass for prose and reference-doc updates) and commits the result on user approval. + +If `$ARGUMENTS` is empty, ask the user which spec to document, defaulting to the most recently modified file under `docs/superpowers/specs/implemented/`. diff --git a/.claude/skills/generate-feature-docs/SKILL.md b/.claude/skills/generate-feature-docs/SKILL.md new file mode 100644 index 000000000..9682051c0 --- /dev/null +++ b/.claude/skills/generate-feature-docs/SKILL.md @@ -0,0 +1,408 @@ +--- +name: generate-feature-docs +description: "Use when generating, writing, or updating publisher-facing documentation from an implemented engineering spec. Activates on requests like \"generate docs for spec X\", \"write a guide page for the RSL spec\", \"update docs for the EC KV extension\". Operates on specs under docs/superpowers/specs/implemented/ with status implemented frontmatter." +--- + +# Generate Feature Docs + +You convert implemented engineering specs into publisher-facing documentation pages on the Trusted Server VitePress site. You run in two interactive stages: an extraction pass that produces a structured outline for the user, and a generation pass that writes prose, updates reference docs, and commits on user approval. + +## Output contract + +You write only to: + +- One file under `docs/guide/.md` (created or augmented). +- Up to three additive updates to `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`. + +Writes are confined to the four files listed above. You never write any other file, never open PRs, never push, never deploy, never modify code under `crates/`, and never modify the spec you are reading. + +## Spec readiness check (run first, before anything else) + +Before doing anything else, parse the spec's YAML frontmatter and check the `status` field. + +- `status: implemented`: proceed to the extraction pass. +- Any other value, or missing `status`: stop. Print: + > "This spec has `status: ` (or no status). The skill operates on `status: implemented` specs. Continue without status: implemented? Reply `y` to proceed." + + Wait for the user's reply. Treat any reply other than a single `y` (case-insensitive) as abort. On `y`, print this warning once before continuing: + > "Proceeding without `status: implemented`. The generated docs may drift from product." + +You never add frontmatter on the user's behalf. If the file has no frontmatter, the user must add it before re-running. + +Optional frontmatter fields the skill recognizes but does not enforce: + +- `implemented_in`: PR number where the implementation landed. +- `last_reviewed`: date of the most recent engineering review, in `YYYY-MM-DD` format. +- `verified_against_commit`: commit SHA the engineer asserts the spec was verified against at promotion time. Audit trail only. + +If `verified_against_commit` is present, surface its value in the Stage 1 outline header (alongside the other metadata) so the user can compare it against the current branch state if drift is suspected. + +## Style rules (apply to ALL output, both chat messages and written files) + +- No em-dashes. Use commas, colons, or semicolons. +- No emojis, no decorative characters, no exclamation marks. +- No marketing words: "powerful", "seamless", "robust", "efficiently", "appropriately", "leveraging". +- Status indicators in tables use text (`verified`, `NOT FOUND`), not symbols. +- Direct, present-tense, second-person voice when speaking to the reader. +- Match the register of `docs/guide/edge-cookies.md` and `docs/guide/integration-guide.md`. + +Before writing any file or chat message, scan your draft for em-dashes, emojis, exclamation marks, and the forbidden words above. If any are present, rewrite. + +## Slash command invocation + +Invoked as `/generate-feature-docs `. The argument is a path to a spec file under `docs/superpowers/specs/implemented/`. If the argument is empty, resolve to the most recently modified file in that directory and confirm with the user. + +If the spec file does not exist, abort with a clear error. If the spec file lives outside `docs/superpowers/specs/implemented/`, warn once and ask the user to confirm before proceeding. + + +## Stage 1: Extraction pass + +Read-only. Produces a structured outline shown to the user in chat. Do not write any files during stage 1. + +### Step 1.1: Parse the spec + +Read the spec file. Extract: +- The H1 title (treat as the feature name). +- The intro paragraph (treat as the description). +- All H2 and H3 section headings. +- All fenced code blocks. Note the language tag of each block. + +### Step 1.2: Detect spec kind + +Heuristic on section names: +- A spec with sections like "Configuration", "Public API", or "Endpoints" is a **feature spec**. Proceed normally. +- A spec with sections like "Migration phases" or "Rollout plan" is a **migration spec**. +- A spec with sections like "Pre-prod checklist" or "Production readiness" is a **readiness report**. +- Anything else with no clear kind is **unknown**. + +For non-feature specs and unknown specs, ask: +> "This looks like a `` spec, not a feature spec. Continue anyway, or abort?" + +Do not proceed without explicit confirmation. + +### Step 1.3: Resolve the target page path + +Slug the feature name to kebab-case (e.g., "RSL AI Crawler Licensing" becomes `ai-crawler-licensing`). The target page is `docs/guide/.md`. + +Check if the target page already exists: +- If exists: this is an augmentation case. Note the existing file's section structure (H2/H3 walk). +- If not: this is a greenfield case. + +If a near-match exists (e.g., the slug differs only by a word), surface it as a candidate before proceeding: +> "I will write to `docs/guide/.md`. A similar page exists at `docs/guide/.md`. Augment the existing page, or create a new one?" + +### Step 1.4: Detect Sequence-section need + +Heuristic: scan the spec for numbered request-flow steps, or language like "first ... then ... finally", or sequence diagrams. If present, mark `needs_sequence_section: yes` for stage 2. + +### Step 1.5: Detect multi-feature specs + +If the spec has 2 or more top-level "Feature: X" sections, or the H1 is ambiguous (covers multiple distinct features), list candidate features and ask: +> "This spec covers multiple features: , , . Generate one page per feature, one combined page, or a subset?" + +No default. The user must pick. + +### Step 1.6: Extract handles + +Walk the spec body for: + +- **Config keys**: TOML keys (`section.key` or `key` inside a `[section]` block), and any inline references like `the X config key`. +- **Endpoint paths**: URL strings starting with `/`, often inside code blocks or backtick-delimited. +- **HTTP headers**: names matching `X-...` or shown as `Header: value`. +- **Error variants**: Rust enum variants matching `SomethingError::Variant`, and any plain-text references to error codes. + +Record each handle with its surface form and any context the spec gave (purpose, valid values, defaults). + +### Step 1.7: Verify each handle against code + +For each handle, search the code: + +- Config keys: grep `crates/**/*.rs` and `trusted-server.toml` for the key name. Capture the file and line number when found. +- Endpoint paths: grep `crates/**/*.rs` for the path string (try both quoted and unquoted forms). +- HTTP headers: grep for the header name as a string literal, plus any const declarations. +- Error variants: grep for the variant name, and locate its enum definition. + +Mark each handle as `verified` (with `file:line`) or `NOT FOUND`. + +### Step 1.7a: Compute verification rate and gate on threshold + +After all handles have been verified in Step 1.7: + +1. Compute `verified_count` (handles marked `verified`) and `total_extracted_count` (all extracted handles, regardless of type). +2. Compute `verification_rate = verified_count / total_extracted_count`. +3. If `total_extracted_count` is zero, this is the "no shipped code" edge case (Section "Edge cases and failure modes"). Use the existing handling for that case. +4. If `verification_rate < 0.50`, add a hard prompt to the Stage 1 "Issues" subsection: + + > "Stage 1 verified `` of `` handles in this codebase (`%`). Below 50% suggests this spec may not be fully implemented in this branch. Options: (A) generate stubs for unverified handles, (B) abort and check status, (C) override and proceed normally." + + The user must pick A, B, or C before the skill proceeds. The threshold of 50% is initial; tune based on real usage. + +5. The threshold gate is in addition to per-handle Issues from Step 1.7. Both get surfaced in the same Stage 1 outline. + +### Step 1.7b: Branch-state heuristic + +In addition to handle verification, check whether the current branch has touched code relevant to the feature: + +```bash +git log --name-only $(git merge-base HEAD main 2>/dev/null || git merge-base HEAD master 2>/dev/null)..HEAD -- crates/ trusted-server.toml +``` + +If the result is empty (the current branch has no commits touching `crates/` or `trusted-server.toml`), surface this as an additional informational note in the Stage 1 outline header: + +> "Note: this branch has no commits touching `crates/` or `trusted-server.toml`. If you expect the implementation to be on this branch, you may be on the wrong branch." + +This is informational, not a hard stop. Pair it with Step 1.7a's verification rate to give a fuller picture. + +If the merge-base lookup fails (no `main` or `master` branch found), skip this check silently and proceed. + +### Step 1.8: Detect spec inconsistencies + +Look for: +- Same config key spelled two ways across the spec (e.g., `rsl.enabled` and `rsl_enabled`). +- Two endpoints with the same path but different descriptions. +- Two error variants with conflicting trigger descriptions. + +Surface any findings under an "Inconsistencies" subsection in the outline. + +### Step 1.9: Render the outline + +Render a single chat message in this format. Use it verbatim, filling in the values you extracted: + +```markdown +## Extraction summary for `` + +**Feature:** +**Target page:** `docs/guide/.md` (NEW or EXISTING) +**Spec kind:** +**Sequence section:** +**Verified against commit:** + + +> Note: this branch has no commits touching `crates/` or `trusted-server.toml`. If you expect the implementation to be on this branch, you may be on the wrong branch. + +### Config keys +| Key | Status | Location | +| --------------- | ------------------- | --------------------- | +| `` | verified or NOT FOUND | `` or "spec only" | + +### Endpoints +| Path | Methods | Status | Location | +| --------------- | ------- | ------------------- | --------------------- | +| `` | | verified or NOT FOUND | `` or "spec only" | + +### Headers +| Name | Direction | Status | Location | +| --------------- | ------------------- | ------------------- | --------------------- | +| `` | request or response | verified or NOT FOUND | `` or "spec only" | + +### Error variants +| Variant | Status | Location | +| --------------- | ------------------- | --------------------- | +| `` | verified or NOT FOUND | `` or "spec only" | + +### Inconsistencies (if any) +- + +### Issues +For each handle marked `NOT FOUND` or each inconsistency, list options: +- (A) Mark inline as "planned, not yet shipped" +- (B) Drop the row from the relevant reference doc +- (C) Pause and let me fix the spec or the code first + +Reply `proceed`, redirect specific fields (e.g. "use slug `rsl-licensing`"), or pick A/B/C for each issue. +``` + +Omit empty subsections (e.g., if no headers were extracted, omit the Headers table). Always include at least one of: Config keys, Endpoints, or Error variants. If none of these exist, the spec may not be a feature spec. + +### Step 1.10: Wait for user response + +Do not proceed to stage 2 until the user replies with `proceed` or equivalent affirmative ("yes, go ahead", "ok proceed", etc.). Substantial redirects (different slug, different target, new feature scope) regenerate the outline; minor redirects (drop a handle, override a heuristic) are noted and the skill proceeds. + +## Stage 2: Generation pass + +Runs only after the user types `proceed`. Inputs: the spec, the approved outline from stage 1, and the existing docs. Output: files written to disk; nothing is committed until the user approves the diff. + +### Step 2.1: Branch check (before any writes) + +Detect the current git branch: + +```bash +git branch --show-current +``` + +If the result is `main` or `master`: +- Stop. Do not write any files. +- Propose a branch name in the form `docs/` (e.g., `docs/ai-crawler-licensing`). Ask: + > "You are on ``. Create branch `docs/` and switch to it?" +- The user can specify a different branch name. +- The skill refuses to proceed on `main` or `master` under any circumstance, including override attempts. +- After confirmation, run `git checkout -b `. + +Check the working tree for uncommitted changes outside the planned doc files: + +```bash +git status --short +``` + +If there are unrelated changes (anything not under `docs/guide/` or otherwise unrelated to this skill's output), stop with: +> "Uncommitted changes detected outside the planned doc files. Commit, stash, or revert them before running this skill, since the doc commit must contain only doc files." + +This is a hard stop. No override. + +### Step 2.2: Choose template structure + +Based on stage 1 outputs, plan the page sections. Standard order, omit empty sections: + +1. **Overview**: what the feature is, who it is for. One to three short paragraphs. +2. **How it works**: mechanism, key concepts, behavior an operator needs. +3. **Sequence** (optional): numbered list, only if `needs_sequence_section: yes` from stage 1. +4. **Configuration**: one to two paragraphs naming the config keys, with a link to `/guide/configuration` for the full reference. +5. **API contract**: endpoints, headers, request and response shapes. Code blocks for each. +6. **Error handling**: error variants, what triggers them, what the response looks like. +7. **Privacy and consent considerations**: only if the feature has consent or PII implications. +8. **Related docs**: internal links to adjacent feature pages. + +A feature with no errors has no Error handling section. A feature with no consent implications has no Privacy section. The template is a maximum, not a minimum. + +### Step 2.3: Write or augment the feature page + +**If greenfield (page does not exist):** +- Write `docs/guide/.md` from scratch using the template above. +- Every concrete reference (config key, file path, endpoint, header, error variant) must be one of the verified handles from stage 1, or an explicit `` for items the user opted into during the issues prompt. +- Empty sections drop entirely; do not write a heading with no content. +- If the spec does not say enough to write a section, write one sentence, not a paragraph of speculation. + +**If augmenting (page exists):** + +1. Walk the existing page's H2 and H3 structure. +2. For each template section that already exists in the page: leave existing prose alone. Add new items only (e.g., a new row in a config table, a new bullet in a list). Never rewrite human-authored prose for stylistic reasons. +3. For sections in the template that do not exist in the page: insert them in template order. +4. For prose that *contradicts* the new spec or current code (e.g., a sentence mentioning a config key that no longer exists, or a behavioral claim that the spec has revised): show the existing text and the proposed replacement, and ask the user to approve, skip, or edit per item: + > "Existing prose says: ``. Spec says: ``. Replace, skip, or edit?" + + This is the only path by which you rewrite existing prose. + +The default posture is conservative. Under-augmenting is recoverable; destroying a teammate's hand-edits is not. + +### Step 2.4: Apply mechanical reference-doc updates + +For each of `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`: + +1. Read the file first to learn its existing structural pattern: column layout in tables, section ordering, code-block formatting. +2. Determine which entries (if any) the spec contributes: + - `configuration.md`: new config keys. + - `api-reference.md`: new endpoints or headers. + - `error-reference.md`: new error variants. +3. Append or insert each entry following the existing pattern. +4. If an entry already exists for the same key (config key, endpoint path, header, error variant) and the spec defines it differently, prompt: + > "Configuration.md already has a row for `` that says ``. Spec says ``. Overwrite, keep existing, or pause?" + + Only overwrite on explicit user approval. +5. Updates are otherwise additive and idempotent. Running the skill twice on the same spec produces no second diff. + +If the spec contributes nothing to a given reference doc, do not modify that file. + +### Step 2.5: Diff review + +After all files are written, post a chat message in this format: + +```markdown +Generated files: + - [docs/guide/.md](docs/guide/.md) (, ) + - [docs/guide/configuration.md](docs/guide/configuration.md) (+ lines, ) + - [docs/guide/api-reference.md](docs/guide/api-reference.md) (+ lines, ) + - [docs/guide/error-reference.md](docs/guide/error-reference.md) (+ lines, ) + +Inline TODOs: () + +Reply `commit`, `show diff`, or redirect a section. +``` + +File paths in the message use markdown link syntax with relative paths so the user can click to open each file in their editor. Omit lines for files that were not modified. + +### Step 2.6: Handle user response + +- `commit`: proceed to step 2.7. +- `show diff`: run `git diff` against the modified files, paste the output inline, then re-prompt: "Reply `commit` or redirect a section." +- A redirect ("Overview is too long, cut it in half" or "rewrite the Configuration section to use the new key"): apply the redirect to the named section only, re-show the diff for the affected file, then re-prompt. + +Do not proceed to commit without explicit `commit`. + +### Step 2.7: Commit + +Stage explicitly via path: + +```bash +git add docs/guide/.md +git add docs/guide/configuration.md docs/guide/api-reference.md docs/guide/error-reference.md +``` + +Only include the paths of files actually modified. + +Commit message format: +- New page: `Add docs for ` +- Augmentation: `Update docs for ` + +Body lists each file touched and the inline TODO count, if any. Sentence case, imperative mood, no semantic prefixes (no `feat:`, `chore:`, etc.), matching the existing CONTRIBUTING.md style. + +```bash +git commit -m "$(cat <<'COMMIT_MSG' +Add docs for + +- docs/guide/.md (new feature page) +- docs/guide/configuration.md (added config keys) +- docs/guide/api-reference.md (added endpoints) + +Inline TODOs: +COMMIT_MSG +)" +``` + +After the commit, post a final message: +> "Committed as ``. Run `git log -1` to inspect, or push when ready." + +Do not push. + +## Edge cases and failure modes + +| Case | Behavior | +| ------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Spec lacks `status: implemented` frontmatter | Prompt `Continue without status: implemented? Reply y to proceed.` Treat any reply other than literal `y` as abort. | +| Spec covers multiple features | List candidates, ask user to pick: one page per feature, combined page, or subset. No default. | +| Non-feature spec (migration, readiness, tech-spec) | Prompt: "This looks like a `` spec, continue anyway?". No automatic fallback. | +| No shipped code (zero handles verify against `crates/`) | Prompt: "No shipped code found. Generate stub page with sections marked 'planned, not yet shipped', or abort?". Behavior verification is for skill #2. | +| Spec is internally contradictory | Surface in stage 1 outline under "Inconsistencies". Ask user to resolve before proceeding. | +| Target page name cannot be determined | Ask user for target path explicitly. | +| Spec file not found | Hard error, abort with message naming the path that was looked up. | +| Spec file outside `docs/superpowers/specs/implemented/` | Warn once: "This file is outside `implemented/`. Is this really an implemented spec?". Proceed only on confirmation. | +| Current branch is `main` or `master` | Hard stop, no override. Propose `docs/` branch name. Switch via `git checkout -b` only on explicit confirmation. | +| Working tree has unrelated uncommitted changes | Hard stop, no override. User must clean up first. | +| Re-run on a spec that has already produced docs | Supported. Stage 1 finds existing page. Stage 2 augments per the augment-in-place rules. A clean re-run with no spec or code changes produces zero diff (idempotency requirement). | + +## Idempotency + +Re-running the skill on the same spec, with no intervening spec or code changes, must produce zero diff. This is a verification target; before posting the diff-review message, check whether `git diff` is empty for all files the skill would have modified, and if so, post: + +> "Re-run produced no changes. The docs are already up to date for this spec." + +Do not produce an empty commit. + +## Self-check before each user message + +Before sending any chat message or writing any file, scan your output for: +- Em-dashes (`—` or `–`) +- Emojis or decorative characters +- Exclamation marks +- The words: "powerful", "seamless", "robust", "efficiently", "appropriately", "leveraging" + +If any are present, rewrite. This includes prompts, status updates, summaries, the extraction outline, and the final diff-review message. + +## Out of scope + +You do not: +- Detect drift between spec and code *behavior*. You verify handle existence only. Behavioral verification is skill #2's job. +- Update narrative docs (`getting-started.md`, `gdpr-compliance.md`, `architecture.md`, etc.). Those are humans' responsibility. +- Generate Mermaid diagrams. Sequence sections use numbered lists. +- Touch code under `crates/`. The codebase is read-only. +- Open PRs, push, or deploy. You commit to the current branch only. +- Modify the spec file you are reading. diff --git a/.gitignore b/.gitignore index af70c452a..a10fde2a0 100644 --- a/.gitignore +++ b/.gitignore @@ -24,6 +24,7 @@ src/*.html !.claude/settings.json !.claude/commands/ !.claude/agents/ +!.claude/skills/ # SSL certificates *.pem diff --git a/CLAUDE.md b/CLAUDE.md index ec76ee46e..83de39b02 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -375,6 +375,32 @@ both runtime behavior and build/tooling changes. --- +## Engineering Spec Conventions + +Implementation specs live under `docs/superpowers/specs/`, split by lifecycle: + +- `docs/superpowers/specs/drafts/`: brainstorm output. The Superpowers brainstorming skill writes new specs here. Specs are not yet ready for documentation. Frontmatter: `status: draft`. +- `docs/superpowers/specs/implemented/`: post-implementation truth. Specs are promoted here once code has shipped and the spec body has been updated to reflect what shipped. Frontmatter: `status: implemented`. + +**Required frontmatter** (every spec): + +```yaml +--- +status: draft | in-progress | implemented +implemented_in: PR#123 # optional, set on promotion +last_reviewed: 2026-04-15 # optional, YYYY-MM-DD +verified_against_commit: a1b2c3d4 # optional, audit trail (skill records, does not validate) +--- +``` + +**For agents writing new specs:** when invoked from this project, the brainstorming skill must write to `docs/superpowers/specs/drafts/`, not the parent directory. Add `status: draft` frontmatter at write time. + +**For agents generating user-facing docs:** the `generate-feature-docs` skill operates only on specs under `implemented/` with `status: implemented`. See `.claude/skills/generate-feature-docs/SKILL.md`. + +Promoting a spec from draft to implemented is a `git mv` plus a frontmatter edit, ideally landed in the same PR that updates the spec body to match shipped code. + +--- + ## What NOT to Do - Do not add unnecessary dependencies without justification. diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index d66a820c0..dff51192b 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -38,6 +38,7 @@ export default withMermaid( description: 'Privacy-preserving edge computing for ad serving and edge cookie (EC) generation', base: '/trusted-server', + srcExclude: ['superpowers/**', '**/node_modules/**'], // Replace version placeholders like {{NODEJS_VERSION}} with values from .tool-versions markdown: { diff --git a/docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md b/docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md new file mode 100644 index 000000000..164b48eae --- /dev/null +++ b/docs/superpowers/plans/2026-04-28-generate-feature-docs-skill.md @@ -0,0 +1,1298 @@ +# Generate Feature Docs Skill Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a Claude Code skill that converts implemented engineering specs in `docs/superpowers/specs/implemented/` into publisher-facing documentation pages in `docs/guide/`, with handle verification against code, augment-in-place behavior for existing pages, and mechanical updates to `configuration.md`, `api-reference.md`, and `error-reference.md`. + +**Architecture:** Two files compose the skill: a thin slash command at `.claude/commands/generate-feature-docs.md` and the actual skill instructions at `.claude/skills/generate-feature-docs/SKILL.md`. The skill runs in two interactive stages: stage 1 reads the spec, verifies handles against code, and produces a structured outline for user review; stage 2 writes prose, applies mechanical reference-doc updates, shows a diff, and commits on user approval. Built entirely on Claude Code's existing skill and slash-command primitives. No new runtime infrastructure. Skill content is plain markdown, versioned in git, deployed via `git pull`. + +**Tech Stack:** Markdown for skill instructions and slash command, VitePress for the docs site (already configured), Git for versioning, Bash for one-time migrations and validation grep checks. No language runtime required to build the skill itself. + +**Source spec:** [docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md](../specs/drafts/2026-04-28-generate-feature-docs-skill-design.md) + +--- + +## Phase 1: Prerequisites + +Setup work that must complete before the skill itself is built. None of these tasks change runtime behavior; they establish conventions and clean up the docs site. + +### Task 1: Exclude internal specs from the public VitePress build + +**Why:** The docs site at iabtechlab.github.io/trusted-server is currently rendering internal engineering specs from `docs/superpowers/specs/` because the VitePress config does not exclude them. This is a pre-existing privacy issue that the skill design depends on. Must be fixed before the skill is used in normal operation. + +**Files:** + +- Modify: `docs/.vitepress/config.mts` + +- [ ] **Step 1: Read the existing config** + +Open `docs/.vitepress/config.mts` and locate the `defineConfig({...})` call (currently around line 36, wrapped in `withMermaid(...)`). + +- [ ] **Step 2: Add srcExclude option to defineConfig** + +Add the following property inside the `defineConfig({...})` object, near `base:` and before `markdown:`: + +```ts +srcExclude: ['superpowers/**', '**/node_modules/**'], +``` + +After the change, the relevant region of the file should look like: + +```ts +export default withMermaid( + defineConfig({ + title: 'Trusted Server', + description: + 'Privacy-preserving edge computing for ad serving and edge cookie (EC) generation', + base: '/trusted-server', + srcExclude: ['superpowers/**', '**/node_modules/**'], + + // Replace version placeholders like {{NODEJS_VERSION}} with values from .tool-versions + markdown: { + // ...unchanged +``` + +- [ ] **Step 3: Build the docs locally** + +Run: + +```bash +cd docs && npm run build +``` + +Expected: build completes without errors. Output goes to `docs/.vitepress/dist/`. + +- [ ] **Step 4: Verify excluded pages are absent from the build output** + +Run: + +```bash +find docs/.vitepress/dist -path '*/superpowers/*' | head +``` + +Expected: zero output. If anything prints, the exclusion did not work; revisit Step 2. + +- [ ] **Step 5: Commit** + +```bash +git add docs/.vitepress/config.mts +git commit -m "Exclude internal specs from VitePress build" +``` + +--- + +### Task 2: Create the implemented/ subdirectory + +**Why:** The directory split (drafts/ vs implemented/) is the structural signal for the spec lifecycle. The `drafts/` directory already exists (created when the design spec was written). The `implemented/` directory does not exist yet; create it now with a `.gitkeep` so the convention is in place before any specs are promoted. + +**Files:** + +- Create: `docs/superpowers/specs/implemented/.gitkeep` + +- [ ] **Step 1: Create the directory and the keepfile** + +```bash +mkdir -p docs/superpowers/specs/implemented +touch docs/superpowers/specs/implemented/.gitkeep +``` + +- [ ] **Step 2: Verify both lifecycle directories exist** + +```bash +ls -d docs/superpowers/specs/drafts docs/superpowers/specs/implemented +``` + +Expected: both paths print without errors. + +- [ ] **Step 3: Commit** + +```bash +git add docs/superpowers/specs/implemented/.gitkeep +git commit -m "Add implemented/ directory for promoted specs" +``` + +--- + +### Task 3: Move existing specs to drafts/ with status frontmatter + +**Why:** All 12 existing specs at `docs/superpowers/specs/*.md` are brainstorm output, not finalized. They must live under `drafts/` and carry `status: draft` frontmatter to match the convention. The new design spec already lives in `drafts/`; this task handles the other 12. + +**Files:** + +- Move and modify: all 12 files matching `docs/superpowers/specs/*.md` + +The 12 files: + +``` +2026-01-15-attestation-design.md +2026-03-11-production-readiness-report-design.md +2026-03-19-auction-orchestration-flow-design.md +2026-03-19-edgezero-migration-design.md +2026-03-24-ssc-prd-design.md +2026-03-24-ssc-technical-spec-design.md +2026-03-25-streaming-response-design.md +2026-03-30-pr7-geo-client-info-design.md +2026-04-02-ec-kv-schema-extensions-design.md +2026-04-02-ec-kv-seeding-design.md +2026-04-18-microsoft-monetize-server-side-ad-templates-codex-reviewed-design.md +2026-04-22-rsl-ai-crawler-licensing-design.md +``` + +Note: 3 of these are currently untracked in git (per the repo's working state). The script below handles tracked and untracked files identically by working at the filesystem level and letting `git add` figure out the rest. + +- [ ] **Step 1: Confirm exactly 12 files at the top level of specs/** + +```bash +ls docs/superpowers/specs/*.md | wc -l +``` + +Expected: `12`. If more or fewer, stop and investigate; the file list above may be stale. + +- [ ] **Step 2: Verify none of the 12 files already has frontmatter** + +```bash +for f in docs/superpowers/specs/*.md; do + head -1 "$f" | grep -q '^---' && echo "ALREADY HAS FRONTMATTER: $f" +done +``` + +Expected: zero output. If any file already has frontmatter, the prepend would corrupt it; that file must be handled by hand. + +- [ ] **Step 3: Move each file to drafts/ and prepend status frontmatter** + +```bash +for f in docs/superpowers/specs/*.md; do + base=$(basename "$f") + newpath="docs/superpowers/specs/drafts/$base" + { printf -- '---\nstatus: draft\n---\n\n'; cat "$f"; } > "$newpath" + rm "$f" +done +``` + +After this, the top level of `docs/superpowers/specs/` should contain only `drafts/` and `implemented/` subdirectories. + +- [ ] **Step 4: Verify all 12 files now live in drafts/ with frontmatter** + +```bash +ls docs/superpowers/specs/drafts/*.md | wc -l +``` + +Expected: `13` (the 12 moved files plus the design spec written earlier). + +```bash +for f in docs/superpowers/specs/drafts/*.md; do + head -3 "$f" | grep -q 'status: draft' || echo "MISSING FRONTMATTER: $f" +done +``` + +Expected: zero output. + +- [ ] **Step 5: Verify nothing remains at the top level of specs/** + +```bash +ls docs/superpowers/specs/*.md 2>/dev/null | wc -l +``` + +Expected: `0`. The shell may print `ls: cannot access` to stderr; that is fine. + +- [ ] **Step 6: Build the docs site to confirm nothing broke** + +```bash +cd docs && npm run build +``` + +Expected: build completes cleanly. + +- [ ] **Step 7: Stage and commit** + +```bash +git add docs/superpowers/specs/ +git status --short docs/superpowers/specs/ +``` + +Expected: a mix of `R` (rename) and `A` (added) entries, depending on whether each source file was tracked. No untracked files should remain under `docs/superpowers/specs/`. + +```bash +git commit -m "Move existing specs to drafts/ with status: draft frontmatter" +``` + +--- + +### Task 4: Document the spec-readiness convention in CLAUDE.md + +**Why:** The directory layout and `status:` frontmatter are conventions Claude must follow when invoked from this repo. CLAUDE.md is the right place because the harness rules give CLAUDE.md priority over default skill behavior. This is also where future contributors will look. + +**Files:** + +- Modify: `CLAUDE.md` + +- [ ] **Step 1: Read CLAUDE.md and locate a suitable insertion point** + +A reasonable location is after the existing `## Project Overview` section and before `## Workspace Layout`. Or as a new top-level section near the end, before `## What NOT to Do`. + +- [ ] **Step 2: Add a new section to CLAUDE.md** + +Insert the following block at the chosen location: + +```markdown +## Engineering Spec Conventions + +Implementation specs live under `docs/superpowers/specs/`, split by lifecycle: + +- `docs/superpowers/specs/drafts/`: brainstorm output. The Superpowers brainstorming skill writes new specs here. Specs are not yet ready for documentation. Frontmatter: `status: draft`. +- `docs/superpowers/specs/implemented/`: post-implementation truth. Specs are promoted here once code has shipped and the spec body has been updated to reflect what shipped. Frontmatter: `status: implemented`. + +**Required frontmatter** (every spec): + +## \`\`\`yaml + +status: draft | in-progress | implemented +implemented_in: PR#123 # optional, set on promotion +last_reviewed: 2026-04-15 # optional, YYYY-MM-DD + +--- + +\`\`\` + +**For agents writing new specs:** when invoked from this project, the brainstorming skill must write to `docs/superpowers/specs/drafts/`, not the parent directory. Add `status: draft` frontmatter at write time. + +**For agents generating user-facing docs:** the `generate-feature-docs` skill operates only on specs under `implemented/` with `status: implemented`. See `.claude/skills/generate-feature-docs/SKILL.md`. + +Promoting a spec from draft to implemented is a `git mv` plus a frontmatter edit, ideally landed in the same PR that updates the spec body to match shipped code. +``` + +(The triple-backticks above use `\`\`\`` to escape; replace those with literal backticks when writing into CLAUDE.md.) + +- [ ] **Step 3: Verify the markdown renders correctly** + +```bash +grep -A 30 'Engineering Spec Conventions' CLAUDE.md +``` + +Expected: the new section is present and readable. + +- [ ] **Step 4: Commit** + +```bash +git add CLAUDE.md +git commit -m "Document spec-readiness convention in CLAUDE.md" +``` + +--- + +## Phase 2: Skill Implementation + +Build the slash command and the skill itself. After Phase 2, the skill is invokable but has not yet been validated against real specs. + +### Task 5: Create the slash command file + +**Why:** The slash command is the user-facing entry point. Following the existing convention (`check-ci.md`, `verify.md`, etc.), it is a thin file that invokes the skill with `$ARGUMENTS`. + +**Files:** + +- Create: `.claude/commands/generate-feature-docs.md` + +- [ ] **Step 1: Read an existing slash command to match the pattern** + +```bash +cat .claude/commands/check-ci.md +``` + +The existing pattern is plain prose instructions. Slash commands in this repo do not currently use frontmatter or any special directives. + +- [ ] **Step 2: Create the slash command file** + +Write the following content to `.claude/commands/generate-feature-docs.md`: + +```markdown +Generate publisher-facing documentation from an implemented engineering spec. + +Spec path: $ARGUMENTS + +Use the `generate-feature-docs` skill at `.claude/skills/generate-feature-docs/SKILL.md` to perform this task. The skill runs in two interactive stages (extraction pass for outline review, generation pass for prose and reference-doc updates) and commits the result on user approval. + +If `$ARGUMENTS` is empty, ask the user which spec to document, defaulting to the most recently modified file under `docs/superpowers/specs/implemented/`. +``` + +- [ ] **Step 3: Verify the file was written correctly** + +```bash +cat .claude/commands/generate-feature-docs.md +``` + +Expected: the content above. + +- [ ] **Step 4: Commit** + +```bash +git add .claude/commands/generate-feature-docs.md +git commit -m "Add /generate-feature-docs slash command" +``` + +--- + +### Task 6: Create SKILL.md with identity, invocation, and readiness rules + +**Why:** SKILL.md is the heart of the skill. It is loaded into Claude's context whenever the skill is invoked. The file is built up across Tasks 6, 7, 8, and 9, one logical section per task, so each commit is reviewable in isolation. This task lays the foundation: the skill's identity, when it activates, what it operates on, and the spec-readiness convention. + +**Files:** + +- Create: `.claude/skills/generate-feature-docs/SKILL.md` + +- [ ] **Step 1: Create the directory** + +```bash +mkdir -p .claude/skills/generate-feature-docs +``` + +- [ ] **Step 2: Write SKILL.md with the identity and readiness sections** + +Write the following content to `.claude/skills/generate-feature-docs/SKILL.md`. This is the initial file; subsequent tasks extend it. + +```markdown +--- +name: generate-feature-docs +description: 'Use when generating, writing, or updating publisher-facing documentation from an implemented engineering spec. Activates on requests like "generate docs for spec X", "write a guide page for the RSL spec", "update docs for the EC KV extension". Operates on specs under docs/superpowers/specs/implemented/ with status implemented frontmatter.' +--- + +# Generate Feature Docs + +You convert implemented engineering specs into publisher-facing documentation pages on the Trusted Server VitePress site. You run in two interactive stages: an extraction pass that produces a structured outline for the user, and a generation pass that writes prose, updates reference docs, and commits on user approval. + +## Output contract + +You write only to: + +- One file under `docs/guide/.md` (created or augmented). +- Up to three additive updates to `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`. + +Writes are confined to the four files listed above. You never write any other file, never open PRs, never push, never deploy, never modify code under `crates/`, and never modify the spec you are reading. + +## Spec readiness check (run first, before anything else) + +Before doing anything else, parse the spec's YAML frontmatter and check the `status` field. + +- `status: implemented`: proceed to the extraction pass. +- Any other value, or missing `status`: stop. Print: + + > "This spec has `status: ` (or no status). The skill operates on `status: implemented` specs. Continue without status: implemented? Reply `y` to proceed." + + Wait for the user's reply. Treat any reply other than a single `y` (case-insensitive) as abort. On `y`, print this warning once before continuing: + + > "Proceeding without `status: implemented`. The generated docs may drift from product." + +You never add frontmatter on the user's behalf. If the file has no frontmatter, the user must add it before re-running. + +## Style rules (apply to ALL output, both chat messages and written files) + +- No em-dashes. Use commas, colons, or semicolons. +- No emojis, no decorative characters, no exclamation marks. +- No marketing words: "powerful", "seamless", "robust", "efficiently", "appropriately", "leveraging". +- Status indicators in tables use text (`verified`, `NOT FOUND`), not symbols. +- Direct, present-tense, second-person voice when speaking to the reader. +- Match the register of `docs/guide/edge-cookies.md` and `docs/guide/integration-guide.md`. + +Before writing any file or chat message, scan your draft for em-dashes, emojis, exclamation marks, and the forbidden words above. If any are present, rewrite. + +## Slash command invocation + +Invoked as `/generate-feature-docs `. The argument is a path to a spec file under `docs/superpowers/specs/implemented/`. If the argument is empty, resolve to the most recently modified file in that directory and confirm with the user. + +If the spec file does not exist, abort with a clear error. If the spec file lives outside `docs/superpowers/specs/implemented/`, warn once and ask the user to confirm before proceeding. + + +``` + +- [ ] **Step 3: Verify the file** + +```bash +head -20 .claude/skills/generate-feature-docs/SKILL.md +``` + +Expected: shows the YAML frontmatter and the first part of the content. + +- [ ] **Step 4: Commit** + +```bash +git add .claude/skills/generate-feature-docs/SKILL.md +git commit -m "Add SKILL.md with identity and readiness rules for generate-feature-docs" +``` + +--- + +### Task 7: Add stage 1 (extraction pass) instructions to SKILL.md + +**Why:** Stage 1 is read-only and produces the structured outline. This is the key checkpoint where the user redirects the skill before any prose is written. Mistakes here are cheap; mistakes in stage 2 are expensive. + +**Files:** + +- Modify: `.claude/skills/generate-feature-docs/SKILL.md` (append) + +- [ ] **Step 1: Append the stage 1 section to SKILL.md** + +Append the following content to the end of the file: + +````markdown +## Stage 1: Extraction pass + +Read-only. Produces a structured outline shown to the user in chat. Do not write any files during stage 1. + +### Step 1.1: Parse the spec + +Read the spec file. Extract: + +- The H1 title (treat as the feature name). +- The intro paragraph (treat as the description). +- All H2 and H3 section headings. +- All fenced code blocks. Note the language tag of each block. + +### Step 1.2: Detect spec kind + +Heuristic on section names: + +- A spec with sections like "Configuration", "Public API", or "Endpoints" is a **feature spec**. Proceed normally. +- A spec with sections like "Migration phases" or "Rollout plan" is a **migration spec**. +- A spec with sections like "Pre-prod checklist" or "Production readiness" is a **readiness report**. +- Anything else with no clear kind is **unknown**. + +For non-feature specs and unknown specs, ask: + +> "This looks like a `` spec, not a feature spec. Continue anyway, or abort?" + +Do not proceed without explicit confirmation. + +### Step 1.3: Resolve the target page path + +Slug the feature name to kebab-case (e.g., "RSL AI Crawler Licensing" becomes `ai-crawler-licensing`). The target page is `docs/guide/.md`. + +Check if the target page already exists: + +- If exists: this is an augmentation case. Note the existing file's section structure (H2/H3 walk). +- If not: this is a greenfield case. + +If a near-match exists (e.g., the slug differs only by a word), surface it as a candidate before proceeding: + +> "I will write to `docs/guide/.md`. A similar page exists at `docs/guide/.md`. Augment the existing page, or create a new one?" + +### Step 1.4: Detect Sequence-section need + +Heuristic: scan the spec for numbered request-flow steps, or language like "first ... then ... finally", or sequence diagrams. If present, mark `needs_sequence_section: yes` for stage 2. + +### Step 1.5: Detect multi-feature specs + +If the spec has 2 or more top-level "Feature: X" sections, or the H1 is ambiguous (covers multiple distinct features), list candidate features and ask: + +> "This spec covers multiple features: , , . Generate one page per feature, one combined page, or a subset?" + +No default. The user must pick. + +### Step 1.6: Extract handles + +Walk the spec body for: + +- **Config keys**: TOML keys (`section.key` or `key` inside a `[section]` block), and any inline references like `the X config key`. +- **Endpoint paths**: URL strings starting with `/`, often inside code blocks or backtick-delimited. +- **HTTP headers**: names matching `X-...` or shown as `Header: value`. +- **Error variants**: Rust enum variants matching `SomethingError::Variant`, and any plain-text references to error codes. + +Record each handle with its surface form and any context the spec gave (purpose, valid values, defaults). + +### Step 1.7: Verify each handle against code + +For each handle, search the code: + +- Config keys: grep `crates/**/*.rs` and `trusted-server.toml` for the key name. Capture the file and line number when found. +- Endpoint paths: grep `crates/**/*.rs` for the path string (try both quoted and unquoted forms). +- HTTP headers: grep for the header name as a string literal, plus any const declarations. +- Error variants: grep for the variant name, and locate its enum definition. + +Mark each handle as `verified` (with `file:line`) or `NOT FOUND`. + +### Step 1.8: Detect spec inconsistencies + +Look for: + +- Same config key spelled two ways across the spec (e.g., `rsl.enabled` and `rsl_enabled`). +- Two endpoints with the same path but different descriptions. +- Two error variants with conflicting trigger descriptions. + +Surface any findings under an "Inconsistencies" subsection in the outline. + +### Step 1.9: Render the outline + +Render a single chat message in this format. Use it verbatim, filling in the values you extracted: + +```markdown +## Extraction summary for `` + +**Feature:** +**Target page:** `docs/guide/.md` (NEW or EXISTING) +**Spec kind:** +**Sequence section:** + +### Config keys + +| Key | Status | Location | +| ------- | --------------------- | ---------------------------- | +| `` | verified or NOT FOUND | `` or "spec only" | + +### Endpoints + +| Path | Methods | Status | Location | +| -------- | ------- | --------------------- | ---------------------------- | +| `` | | verified or NOT FOUND | `` or "spec only" | + +### Headers + +| Name | Direction | Status | Location | +| -------- | ------------------- | --------------------- | ---------------------------- | +| `` | request or response | verified or NOT FOUND | `` or "spec only" | + +### Error variants + +| Variant | Status | Location | +| ----------- | --------------------- | ---------------------------- | +| `` | verified or NOT FOUND | `` or "spec only" | + +### Inconsistencies (if any) + +- + +### Issues + +For each handle marked `NOT FOUND` or each inconsistency, list options: + +- (A) Mark inline as "planned, not yet shipped" +- (B) Drop the row from the relevant reference doc +- (C) Pause and let me fix the spec or the code first + +Reply `proceed`, redirect specific fields (e.g. "use slug `rsl-licensing`"), or pick A/B/C for each issue. +``` + +Omit empty subsections (e.g., if no headers were extracted, omit the Headers table). Always include at least one of: Config keys, Endpoints, or Error variants. If none of these exist, the spec may not be a feature spec. + +### Step 1.10: Wait for user response + +Do not proceed to stage 2 until the user replies with `proceed` or equivalent affirmative ("yes, go ahead", "ok proceed", etc.). Substantial redirects (different slug, different target, new feature scope) regenerate the outline; minor redirects (drop a handle, override a heuristic) are noted and the skill proceeds. +```` + +- [ ] **Step 2: Verify the file size grew as expected** + +```bash +wc -l .claude/skills/generate-feature-docs/SKILL.md +``` + +Expected: substantially more than the previous step (somewhere around 130-180 lines depending on formatting). + +- [ ] **Step 3: Commit** + +```bash +git add .claude/skills/generate-feature-docs/SKILL.md +git commit -m "Add stage 1 extraction pass to generate-feature-docs skill" +``` + +--- + +### Task 8: Add stage 2 (generation pass) instructions to SKILL.md + +**Why:** Stage 2 is where prose gets written, reference docs get updated, and commits are produced. This is the largest section of the skill. + +**Files:** + +- Modify: `.claude/skills/generate-feature-docs/SKILL.md` (append) + +- [ ] **Step 1: Append the stage 2 section to SKILL.md** + +Append the following content to the end of the file: + +````markdown +## Stage 2: Generation pass + +Runs only after the user types `proceed`. Inputs: the spec, the approved outline from stage 1, and the existing docs. Output: files written to disk; nothing is committed until the user approves the diff. + +### Step 2.1: Branch check (before any writes) + +Detect the current git branch: + +```bash +git branch --show-current +``` + +If the result is `main` or `master`: + +- Stop. Do not write any files. +- Propose a branch name in the form `docs/` (e.g., `docs/ai-crawler-licensing`). Ask: + > "You are on ``. Create branch `docs/` and switch to it?" +- The user can specify a different branch name. +- The skill refuses to proceed on `main` or `master` under any circumstance, including override attempts. +- After confirmation, run `git checkout -b `. + +Check the working tree for uncommitted changes outside the planned doc files: + +```bash +git status --short +``` + +If there are unrelated changes (anything not under `docs/guide/` or otherwise unrelated to this skill's output), stop with: + +> "Uncommitted changes detected outside the planned doc files. Commit, stash, or revert them before running this skill, since the doc commit must contain only doc files." + +This is a hard stop. No override. + +### Step 2.2: Choose template structure + +Based on stage 1 outputs, plan the page sections. Standard order, omit empty sections: + +1. **Overview**: what the feature is, who it is for. One to three short paragraphs. +2. **How it works**: mechanism, key concepts, behavior an operator needs. +3. **Sequence** (optional): numbered list, only if `needs_sequence_section: yes` from stage 1. +4. **Configuration**: one to two paragraphs naming the config keys, with a link to `/guide/configuration` for the full reference. +5. **API contract**: endpoints, headers, request and response shapes. Code blocks for each. +6. **Error handling**: error variants, what triggers them, what the response looks like. +7. **Privacy and consent considerations**: only if the feature has consent or PII implications. +8. **Related docs**: internal links to adjacent feature pages. + +A feature with no errors has no Error handling section. A feature with no consent implications has no Privacy section. The template is a maximum, not a minimum. + +### Step 2.3: Write or augment the feature page + +**If greenfield (page does not exist):** + +- Write `docs/guide/.md` from scratch using the template above. +- Every concrete reference (config key, file path, endpoint, header, error variant) must be one of the verified handles from stage 1, or an explicit `` for items the user opted into during the issues prompt. +- Empty sections drop entirely; do not write a heading with no content. +- If the spec does not say enough to write a section, write one sentence, not a paragraph of speculation. + +**If augmenting (page exists):** + +1. Walk the existing page's H2 and H3 structure. +2. For each template section that already exists in the page: leave existing prose alone. Add new items only (e.g., a new row in a config table, a new bullet in a list). Never rewrite human-authored prose for stylistic reasons. +3. For sections in the template that do not exist in the page: insert them in template order. +4. For prose that _contradicts_ the new spec or current code (e.g., a sentence mentioning a config key that no longer exists, or a behavioral claim that the spec has revised): show the existing text and the proposed replacement, and ask the user to approve, skip, or edit per item: + + > "Existing prose says: ``. Spec says: ``. Replace, skip, or edit?" + + This is the only path by which you rewrite existing prose. + +The default posture is conservative. Under-augmenting is recoverable; destroying a teammate's hand-edits is not. + +### Step 2.4: Apply mechanical reference-doc updates + +For each of `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`: + +1. Read the file first to learn its existing structural pattern: column layout in tables, section ordering, code-block formatting. +2. Determine which entries (if any) the spec contributes: + - `configuration.md`: new config keys. + - `api-reference.md`: new endpoints or headers. + - `error-reference.md`: new error variants. +3. Append or insert each entry following the existing pattern. +4. If an entry already exists for the same key (config key, endpoint path, header, error variant) and the spec defines it differently, prompt: + + > "Configuration.md already has a row for `` that says ``. Spec says ``. Overwrite, keep existing, or pause?" + + Only overwrite on explicit user approval. + +5. Updates are otherwise additive and idempotent. Running the skill twice on the same spec produces no second diff. + +If the spec contributes nothing to a given reference doc, do not modify that file. + +### Step 2.5: Diff review + +After all files are written, post a chat message in this format: + +```markdown +Generated files: + +- [docs/guide/.md](docs/guide/.md) ( lines>, ) +- [docs/guide/configuration.md](docs/guide/configuration.md) (+ lines, ) +- [docs/guide/api-reference.md](docs/guide/api-reference.md) (+ lines, ) +- [docs/guide/error-reference.md](docs/guide/error-reference.md) (+ lines, ) + +Inline TODOs: () + +Reply `commit`, `show diff`, or redirect a section. +``` + +File paths in the message use markdown link syntax with relative paths so the user can click to open each file in their editor. Omit lines for files that were not modified. + +### Step 2.6: Handle user response + +- `commit`: proceed to step 2.7. +- `show diff`: run `git diff` against the modified files, paste the output inline, then re-prompt: "Reply `commit` or redirect a section." +- A redirect ("Overview is too long, cut it in half" or "rewrite the Configuration section to use the new key"): apply the redirect to the named section only, re-show the diff for the affected file, then re-prompt. + +Do not proceed to commit without explicit `commit`. + +### Step 2.7: Commit + +Stage explicitly via path: + +```bash +git add docs/guide/.md +git add docs/guide/configuration.md docs/guide/api-reference.md docs/guide/error-reference.md +``` + +Only include the paths of files actually modified. + +Commit message format: + +- New page: `Add docs for ` +- Augmentation: `Update docs for ` + +Body lists each file touched and the inline TODO count, if any. Sentence case, imperative mood, no semantic prefixes (no `feat:`, `chore:`, etc.), matching the existing CONTRIBUTING.md style. + +```bash +git commit -m "$(cat <<'COMMIT_MSG' +Add docs for + +- docs/guide/.md (new feature page) +- docs/guide/configuration.md (added config keys) +- docs/guide/api-reference.md (added endpoints) + +Inline TODOs: +COMMIT_MSG +)" +``` + +After the commit, post a final message: + +> "Committed as ``. Run `git log -1` to inspect, or push when ready." + +Do not push. +```` + +- [ ] **Step 2: Verify the file grew** + +```bash +wc -l .claude/skills/generate-feature-docs/SKILL.md +``` + +Expected: now in the range of 280-360 lines. + +- [ ] **Step 3: Commit** + +```bash +git add .claude/skills/generate-feature-docs/SKILL.md +git commit -m "Add stage 2 generation pass to generate-feature-docs skill" +``` + +--- + +### Task 9: Add edge cases section to SKILL.md + +**Why:** Edge cases capture the "what if" scenarios from Section 10 of the spec. Without explicit guidance, Claude will default to forging ahead in cases where it should pause or abort. This task closes those holes. + +**Files:** + +- Modify: `.claude/skills/generate-feature-docs/SKILL.md` (append) + +- [ ] **Step 1: Append the edge cases section** + +Append the following content to the end of the file: + +```markdown +## Edge cases and failure modes + +| Case | Behavior | +| ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Spec lacks `status: implemented` frontmatter | Prompt `Continue without status: implemented? (y/N)`. Default N. Abort unless explicit `y`. | +| Spec covers multiple features | List candidates, ask user to pick: one page per feature, combined page, or subset. No default. | +| Non-feature spec (migration, readiness, tech-spec) | Prompt: "This looks like a `` spec, continue anyway?". No automatic fallback. | +| No shipped code (zero handles verify against `crates/`) | Prompt: "No shipped code found. Generate stub page with sections marked 'planned, not yet shipped', or abort?". Behavior verification is for skill #2. | +| Spec is internally contradictory | Surface in stage 1 outline under "Inconsistencies". Ask user to resolve before proceeding. | +| Target page name cannot be determined | Ask user for target path explicitly. | +| Spec file not found | Hard error, abort with message naming the path that was looked up. | +| Spec file outside `docs/superpowers/specs/implemented/` | Warn once: "This file is outside `implemented/`. Is this really an implemented spec?". Proceed only on confirmation. | +| Current branch is `main` or `master` | Hard stop, no override. Propose `docs/` branch name. Switch via `git checkout -b` only on explicit confirmation. | +| Working tree has unrelated uncommitted changes | Hard stop, no override. User must clean up first. | +| Re-run on a spec that has already produced docs | Supported. Stage 1 finds existing page. Stage 2 augments per the augment-in-place rules. A clean re-run with no spec or code changes produces zero diff (idempotency requirement). | + +## Idempotency + +Re-running the skill on the same spec, with no intervening spec or code changes, must produce zero diff. This is a verification target; before posting the diff-review message, check whether `git diff` is empty for all files the skill would have modified, and if so, post: + +> "Re-run produced no changes. The docs are already up to date for this spec." + +Do not produce an empty commit. + +## Self-check before each user message + +Before sending any chat message or writing any file, scan your output for: + +- Em-dashes (`—` or `–`) +- Emojis or decorative characters +- Exclamation marks +- The words: "powerful", "seamless", "robust", "efficiently", "appropriately", "leveraging" + +If any are present, rewrite. This includes prompts, status updates, summaries, the extraction outline, and the final diff-review message. + +## Out of scope + +You do not: + +- Detect drift between spec and code _behavior_. You verify handle existence only. Behavioral verification is skill #2's job. +- Update narrative docs (`getting-started.md`, `gdpr-compliance.md`, `architecture.md`, etc.). Those are humans' responsibility. +- Generate Mermaid diagrams. Sequence sections use numbered lists. +- Touch code under `crates/`. The codebase is read-only. +- Open PRs, push, or deploy. You commit to the current branch only. +- Modify the spec file you are reading. +``` + +- [ ] **Step 2: Verify the file is complete** + +```bash +wc -l .claude/skills/generate-feature-docs/SKILL.md +``` + +Expected: roughly 360-440 lines total. + +```bash +grep -c '^## ' .claude/skills/generate-feature-docs/SKILL.md +``` + +Expected: at least 7 top-level sections (Output contract, Spec readiness check, Style rules, Slash command invocation, Stage 1, Stage 2, Edge cases, Idempotency, Self-check, Out of scope). + +- [ ] **Step 3: Run a style self-check on the SKILL.md itself** + +```bash +grep -nE "—|–" .claude/skills/generate-feature-docs/SKILL.md +grep -niE "powerful|seamless|robust|efficiently|leveraging" .claude/skills/generate-feature-docs/SKILL.md | grep -v "Forbidden words" +``` + +Expected: zero output for em-dashes. The marketing-words grep may show lines where the words are listed AS forbidden (those are correct uses); any other hits are bugs. + +- [ ] **Step 4: Commit** + +```bash +git add .claude/skills/generate-feature-docs/SKILL.md +git commit -m "Add edge cases and self-check rules to generate-feature-docs skill" +``` + +--- + +## Phase 3: Validation + +The skill is now built but has not been run. Phase 3 invokes it against real specs and checks the output against the success criteria from Section 11 of the design spec. If validation reveals issues, edit `SKILL.md` and re-run; treat each tweak as a small commit. + +### Task 10: Promote the RSL spec to implemented/ for greenfield validation + +**Why:** Validation case 1 (greenfield) requires at least one spec to live in `implemented/`. The RSL AI crawler licensing spec is a good candidate because it is recent, has no corresponding guide page yet, and represents a complete feature. + +**Files:** + +- Move: `docs/superpowers/specs/drafts/2026-04-22-rsl-ai-crawler-licensing-design.md` to `docs/superpowers/specs/implemented/` +- Modify: the same file's frontmatter + +- [ ] **Step 1: Find the implementation PR** + +The spec was added in commit `8d081287 Add RSL AI crawler licensing design spec`. Identify the PR that landed the actual implementation (if any). If implementation has not yet shipped, the spec is not yet truly `implemented`, and this validation task should wait. Run: + +```bash +git log --all --oneline | grep -i "rsl\|crawler\|licensing" | head -10 +``` + +Inspect the output. If only the spec commit appears, implementation has not landed; pick a different spec for validation, such as one of the older specs whose features are clearly shipped (e.g., the Edge Cookie work). + +For the rest of this task, assume the implementation has shipped. Substitute the actual PR number where the placeholder `` appears. + +- [ ] **Step 2: Move the spec file** + +```bash +git mv docs/superpowers/specs/drafts/2026-04-22-rsl-ai-crawler-licensing-design.md \ + docs/superpowers/specs/implemented/2026-04-22-rsl-ai-crawler-licensing-design.md +``` + +- [ ] **Step 3: Update the frontmatter** + +Edit `docs/superpowers/specs/implemented/2026-04-22-rsl-ai-crawler-licensing-design.md`. Replace the existing frontmatter: + +```yaml +--- +status: draft +--- +``` + +with: + +```yaml +--- +status: implemented +implemented_in: PR# +last_reviewed: 2026-04-28 +--- +``` + +- [ ] **Step 4: Verify** + +```bash +head -6 docs/superpowers/specs/implemented/2026-04-22-rsl-ai-crawler-licensing-design.md +``` + +Expected: shows `status: implemented` and the optional fields. + +- [ ] **Step 5: Commit** + +```bash +git add docs/superpowers/specs/ +git commit -m "Promote RSL AI crawler licensing spec to implemented" +``` + +--- + +### Task 11: Run greenfield validation case + +**Why:** Validates the most common path: spec lands, code ships, docs do not exist, skill produces a publishable page. + +**Files:** + +- Will be created by the skill: `docs/guide/ai-crawler-licensing.md` (or similar slug; the skill resolves it) +- Will be modified by the skill: `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and/or `docs/guide/error-reference.md` + +- [ ] **Step 1: Confirm preconditions** + +```bash +git branch --show-current +``` + +Expected: not `main` or `master`. If on main, switch first: `git checkout -b docs/ai-crawler-licensing`. + +```bash +git status --short +``` + +Expected: empty (no uncommitted changes). If not, commit or stash unrelated work first. + +- [ ] **Step 2: Invoke the skill** + +In Claude Code: + +``` +/generate-feature-docs docs/superpowers/specs/implemented/2026-04-22-rsl-ai-crawler-licensing-design.md +``` + +The skill should run stage 1 and produce an extraction outline. + +- [ ] **Step 3: Review the extraction outline** + +Verify in the chat output: + +- The feature name is reasonable. +- The target page slug is reasonable. +- All extracted handles (config keys, endpoints, headers, errors) are listed. +- Each handle has a status (verified or NOT FOUND). +- Issues, if any, are surfaced with A/B/C options. +- No em-dashes, no emojis, no marketing words appear in the outline. + +If anything is wrong, redirect the skill or capture the issue and edit `SKILL.md`. Common issues to fix in SKILL.md: outline format incorrect, missing sections, handle types not extracted. + +- [ ] **Step 4: Approve and proceed to stage 2** + +Reply `proceed`. The skill should: + +- Check the current branch (already verified in Step 1). +- Write the new feature page. +- Apply mechanical updates to the relevant reference docs. +- Post the diff-review message. + +- [ ] **Step 5: Inspect the generated docs** + +For each file in the diff-review message: + +- Open the file. Read it as if you were a publisher integrating the feature. +- Verify the page follows the template (Overview, How it works, optional Sequence, Configuration, API contract, Error handling, Privacy, Related docs). +- Verify every concrete reference (config key, file path, endpoint, etc.) matches a verified handle from stage 1. +- Verify the prose is direct, present-tense, second-person, and contains no em-dashes, emojis, exclamation marks, or marketing words. + +If the page reads as a draft you would merge with light editing: success. If it reads as something you would rewrite from scratch: capture what is wrong, edit SKILL.md, do not commit, re-run. + +- [ ] **Step 6: Build the docs site to confirm no broken links** + +```bash +cd docs && npm run build +``` + +Expected: build completes cleanly. Any errors point to broken VitePress link references. + +- [ ] **Step 7: Style verification** + +```bash +grep -nE "—|–" docs/guide/ai-crawler-licensing.md +grep -niE "powerful|seamless|robust|efficiently|leveraging" docs/guide/ai-crawler-licensing.md +grep -nE "[\xF0-\xF7][\x80-\xBF]+|✅|❌|⚠️|🔥" docs/guide/ai-crawler-licensing.md +``` + +Expected: zero output from all three. Any hits are skill bugs; fix SKILL.md and re-run. + +- [ ] **Step 8: Approve commit** + +If all checks pass, reply `commit` to the skill. The skill creates one commit on the current branch. + +If any check fails, do not commit. Edit SKILL.md to fix the issue, then re-run from Step 2. + +--- + +### Task 12: Run idempotency case + +**Why:** Validates that re-running the skill produces no diff when neither spec nor code has changed. This is a hard requirement: a non-idempotent skill produces noise on every run. + +**Files:** + +- None modified (this is the assertion) + +- [ ] **Step 1: Confirm clean working tree after Task 11** + +```bash +git status --short +``` + +Expected: empty. + +- [ ] **Step 2: Re-run the skill on the same spec** + +``` +/generate-feature-docs docs/superpowers/specs/implemented/2026-04-22-rsl-ai-crawler-licensing-design.md +``` + +- [ ] **Step 3: Approve through the outline and reach diff review** + +Stage 1 should produce the same outline as Task 11. Reply `proceed`. + +- [ ] **Step 4: Verify the skill detects zero changes** + +The skill should post: + +> "Re-run produced no changes. The docs are already up to date for this spec." + +It must NOT produce an empty commit. It must NOT prompt for commit if nothing changed. + +If the skill produces a non-empty diff on the second run, that is a bug. Inspect the diff to find which step is non-deterministic, edit SKILL.md, and re-run. + +--- + +### Task 13: Run augmentation validation case + +**Why:** Validates the augment-in-place behavior on an existing page. Picks a feature with an existing guide page and a spec that extends the feature. + +**Files:** + +- Will be modified by the skill: `docs/guide/edge-cookies.md` (or another existing page) +- Will be modified by the skill: `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and/or `docs/guide/error-reference.md` if applicable + +- [ ] **Step 1: Choose a candidate spec** + +Pick one of the EC-related drafts (e.g., `2026-04-02-ec-kv-schema-extensions-design.md`) and promote it to `implemented/` if and only if its implementation has shipped. Use the same promotion procedure as Task 10. If implementation has not shipped, choose a different spec. + +- [ ] **Step 2: Invoke the skill on the promoted spec** + +``` +/generate-feature-docs docs/superpowers/specs/implemented/.md +``` + +- [ ] **Step 3: Verify the extraction outline shows EXISTING for the target page** + +The outline must show the target page (e.g., `docs/guide/edge-cookies.md`) with `(EXISTING)` tag. If it shows NEW, the slug resolution is wrong; either the spec's H1 produces a different slug than the existing page name, or the slug logic in SKILL.md is wrong. + +- [ ] **Step 4: Approve and proceed** + +Reply `proceed`. + +- [ ] **Step 5: Verify augment-in-place behavior** + +Open the modified existing page and compare to its prior content (use `git diff`). Confirm: + +- Existing prose is intact except where contradiction-detection prompted you per item. +- New content was added in the right sections (Configuration table got new rows, etc.). +- No human-authored content was destroyed. + +If the skill rewrote prose without prompting, that is a bug. Edit SKILL.md to enforce the conservative augment rule, undo the changes (`git checkout docs/guide/`), and re-run. + +- [ ] **Step 6: Approve commit if checks pass** + +Reply `commit` to the skill. + +--- + +### Task 14: Run non-feature validation case + +**Why:** Validates that the skill correctly detects a non-feature spec and prompts before proceeding rather than silently producing nonsense. + +**Files:** + +- None expected to be modified (the skill should refuse to proceed without confirmation) + +- [ ] **Step 1: Promote the EdgeZero migration spec to implemented/** + +Use the procedure from Task 10: + +```bash +git mv docs/superpowers/specs/drafts/2026-03-19-edgezero-migration-design.md \ + docs/superpowers/specs/implemented/2026-03-19-edgezero-migration-design.md +``` + +Update its frontmatter to `status: implemented`, then commit. + +- [ ] **Step 2: Invoke the skill on the migration spec** + +``` +/generate-feature-docs docs/superpowers/specs/implemented/2026-03-19-edgezero-migration-design.md +``` + +- [ ] **Step 3: Verify the skill detects spec kind and prompts** + +Expected behavior: stage 1 detects `spec_kind: migration` and emits: + +> "This looks like a `migration` spec, not a feature spec. Continue anyway, or abort?" + +If the skill silently proceeds, that is a bug; the spec-kind detection in SKILL.md needs strengthening. + +- [ ] **Step 4: Reply with abort** + +Reply something like "abort" or "no, this is a migration spec, do not proceed". Confirm the skill exits cleanly without writing any files. + +- [ ] **Step 5: Verify no files were written** + +```bash +git status --short +``` + +Expected: empty (or whatever was there before, unchanged). + +--- + +### Task 15: Run drift validation case + +**Why:** Validates handle verification. Tests that a spec with a config key, endpoint, header, or error variant that does not exist in code is flagged in stage 1 with a NOT FOUND status, and the skill correctly handles the user's choice (mark as TODO, drop, or pause). + +**Files:** + +- A test spec with a deliberately-broken handle + +- [ ] **Step 1: Create a test spec with a broken handle** + +Make a minimal feature spec under `docs/superpowers/specs/implemented/` named `2026-04-28-test-drift-validation.md` with the following content: + +````markdown +--- +status: implemented +last_reviewed: 2026-04-28 +--- + +# Test Drift Validation + +A synthetic feature for validating handle drift detection. Not a real feature. + +## Configuration + +```toml +[test_drift] +enabled = true +nonexistent_key = "this key does not exist in code" +``` +```` + +## API contract + +GET `/test-drift/nonexistent-endpoint` returns drift validation data. + +``` + +- [ ] **Step 2: Invoke the skill on the test spec** + +``` + +/generate-feature-docs docs/superpowers/specs/implemented/2026-04-28-test-drift-validation.md + +```` + +- [ ] **Step 3: Verify the extraction outline flags the drift** + +The outline must show: +- Config key `test_drift.enabled`: NOT FOUND, location "spec only" +- Config key `test_drift.nonexistent_key`: NOT FOUND, location "spec only" +- Endpoint `/test-drift/nonexistent-endpoint`: NOT FOUND, location "spec only" + +The Issues section must list these and offer A/B/C options for each. + +If the skill marks any of these as `verified` (since they do not exist in code), that is a bug; the verification logic in SKILL.md is wrong. + +- [ ] **Step 4: Reply with abort to clean up** + +Reply: "abort, this was just a drift test". + +- [ ] **Step 5: Delete the test spec** + +```bash +git rm docs/superpowers/specs/implemented/2026-04-28-test-drift-validation.md +git commit -m "Remove drift validation test spec" +```` + +--- + +### Task 16: Final verification and handoff + +**Why:** Confirms all moving parts are in place and the team can use the skill from this point forward. + +- [ ] **Step 1: Verify the slash command is recognized** + +In Claude Code, type `/generate-feature-docs` (no arguments) and confirm Claude offers to use the skill. The skill should ask which spec to document, defaulting to the most recently modified file under `docs/superpowers/specs/implemented/`. + +- [ ] **Step 2: Run the full test suite to confirm nothing else regressed** + +```bash +cd /Users/jevans/trusted-server +cargo fmt --all -- --check +cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo test --workspace +cd crates/js/lib && npx vitest run +cd ../../../docs && npm run format && npm run build +``` + +Expected: all pass. Any failure indicates a side effect from this work. + +- [ ] **Step 3: Verify the public docs site no longer indexes specs** + +```bash +find docs/.vitepress/dist -path '*/superpowers/*' +``` + +Expected: empty. + +- [ ] **Step 4: Update the team** + +Post a short note in the team channel describing: the skill is available, where to find the source (`.claude/skills/generate-feature-docs/`), how to invoke it (`/generate-feature-docs `), and the spec lifecycle convention (drafts/ vs implemented/, status frontmatter). + +- [ ] **Step 5: Open a follow-up issue for skill #2** + +Create an issue titled "Build spec-vs-reality gap-analysis skill (skill #2)" referencing this skill's design spec section 13 (Out of scope, deferred to skill #2). Skill #2 closes the behavioral-drift gap that this skill cannot detect. + +- [ ] **Step 6: Open a follow-up issue for the upstream Superpowers PR** + +Create an issue titled "Propose drafts/implemented split to Superpowers brainstorming skill" referencing this skill's design spec section 14 (Related work). The PR is to update the brainstorming skill to write to `/drafts/` by default and add `status: draft` frontmatter automatically. + +--- + +## Self-Review + +After writing this plan, the following coverage check confirms each spec section maps to a task: + +- Spec section 1 (Overview): implicit in plan goals. +- Spec section 2 (Audience): encoded in SKILL.md style rules (Task 6). +- Spec section 3 (Goals): drives validation criteria (Tasks 11-15). +- Spec section 4 (Non-goals): encoded in SKILL.md "Out of scope" section (Task 9). +- Spec section 5 (Skill identity and invocation): Tasks 5, 6. +- Spec section 6 (Spec readiness convention): Task 4 (CLAUDE.md), Task 6 (SKILL.md readiness check). +- Spec section 7 (Directory layout): Tasks 2, 3. +- Spec section 8 (Stage 1 extraction): Task 7. +- Spec section 9 (Stage 2 generation): Task 8. +- Spec section 10 (Edge cases): Task 9. +- Spec section 11 (Verification and validation): Tasks 11-15. +- Spec section 12 (Prerequisites): Tasks 1-4. +- Spec section 13 (Out of scope, deferred to skill #2): Task 16 step 5 (open follow-up issue). +- Spec section 14 (Related work, upstream PR): Task 16 step 6 (open follow-up issue). +- Spec section 15 (Implementation summary): meta, no task needed. + +**Placeholder scan:** the plan uses literal placeholders only inside code blocks that demonstrate format (e.g., ``, ``). These are documentation, not unfilled gaps. The plan itself contains no TBDs or TODOs. + +**Type consistency:** the slash command name (`/generate-feature-docs`), the skill path (`.claude/skills/generate-feature-docs/SKILL.md`), the slash command path (`.claude/commands/generate-feature-docs.md`), the directory paths (`docs/superpowers/specs/drafts/`, `docs/superpowers/specs/implemented/`), and the frontmatter field name (`status`) are used consistently throughout. + +The plan is ready for execution. diff --git a/docs/superpowers/specs/2026-01-15-attestation-design.md b/docs/superpowers/specs/drafts/2026-01-15-attestation-design.md similarity index 99% rename from docs/superpowers/specs/2026-01-15-attestation-design.md rename to docs/superpowers/specs/drafts/2026-01-15-attestation-design.md index 9d8d8e14a..910145e7f 100644 --- a/docs/superpowers/specs/2026-01-15-attestation-design.md +++ b/docs/superpowers/specs/drafts/2026-01-15-attestation-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # Runtime Integration Attestation Architecture > **Status:** Proposal diff --git a/docs/superpowers/specs/2026-03-11-production-readiness-report-design.md b/docs/superpowers/specs/drafts/2026-03-11-production-readiness-report-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-11-production-readiness-report-design.md rename to docs/superpowers/specs/drafts/2026-03-11-production-readiness-report-design.md index 5203e5ff7..a1fef1374 100644 --- a/docs/superpowers/specs/2026-03-11-production-readiness-report-design.md +++ b/docs/superpowers/specs/drafts/2026-03-11-production-readiness-report-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # Production Readiness Report **Date:** 2026-03-04 diff --git a/docs/superpowers/specs/2026-03-19-auction-orchestration-flow-design.md b/docs/superpowers/specs/drafts/2026-03-19-auction-orchestration-flow-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-19-auction-orchestration-flow-design.md rename to docs/superpowers/specs/drafts/2026-03-19-auction-orchestration-flow-design.md index 02a86d629..88905a181 100644 --- a/docs/superpowers/specs/2026-03-19-auction-orchestration-flow-design.md +++ b/docs/superpowers/specs/drafts/2026-03-19-auction-orchestration-flow-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # 🎯 Auction Orchestration Flow ## 🔄 System Flow Diagram diff --git a/docs/superpowers/specs/2026-03-19-edgezero-migration-design.md b/docs/superpowers/specs/drafts/2026-03-19-edgezero-migration-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-19-edgezero-migration-design.md rename to docs/superpowers/specs/drafts/2026-03-19-edgezero-migration-design.md index 0ae8c906b..52bc2d355 100644 --- a/docs/superpowers/specs/2026-03-19-edgezero-migration-design.md +++ b/docs/superpowers/specs/drafts/2026-03-19-edgezero-migration-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # EdgeZero Migration Plan > Migration plan for moving trusted-server from direct Fastly Compute SDK usage diff --git a/docs/superpowers/specs/2026-03-24-ssc-prd-design.md b/docs/superpowers/specs/drafts/2026-03-24-ssc-prd-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-24-ssc-prd-design.md rename to docs/superpowers/specs/drafts/2026-03-24-ssc-prd-design.md index 7f88ef15a..b3fe183d3 100644 --- a/docs/superpowers/specs/2026-03-24-ssc-prd-design.md +++ b/docs/superpowers/specs/drafts/2026-03-24-ssc-prd-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # Product Requirements: Edge Cookie (EC) **Status:** Draft diff --git a/docs/superpowers/specs/2026-03-24-ssc-technical-spec-design.md b/docs/superpowers/specs/drafts/2026-03-24-ssc-technical-spec-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-24-ssc-technical-spec-design.md rename to docs/superpowers/specs/drafts/2026-03-24-ssc-technical-spec-design.md index ad47c65d3..6ce9cb7f2 100644 --- a/docs/superpowers/specs/2026-03-24-ssc-technical-spec-design.md +++ b/docs/superpowers/specs/drafts/2026-03-24-ssc-technical-spec-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # Technical Specification: Edge Cookie (EC) **Status:** Draft diff --git a/docs/superpowers/specs/2026-03-25-streaming-response-design.md b/docs/superpowers/specs/drafts/2026-03-25-streaming-response-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-25-streaming-response-design.md rename to docs/superpowers/specs/drafts/2026-03-25-streaming-response-design.md index 414c49547..42330964f 100644 --- a/docs/superpowers/specs/2026-03-25-streaming-response-design.md +++ b/docs/superpowers/specs/drafts/2026-03-25-streaming-response-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # Streaming Response Optimization (Next.js Disabled) ## Problem diff --git a/docs/superpowers/specs/2026-03-30-pr7-geo-client-info-design.md b/docs/superpowers/specs/drafts/2026-03-30-pr7-geo-client-info-design.md similarity index 99% rename from docs/superpowers/specs/2026-03-30-pr7-geo-client-info-design.md rename to docs/superpowers/specs/drafts/2026-03-30-pr7-geo-client-info-design.md index 821ad101c..3c4c27923 100644 --- a/docs/superpowers/specs/2026-03-30-pr7-geo-client-info-design.md +++ b/docs/superpowers/specs/drafts/2026-03-30-pr7-geo-client-info-design.md @@ -1,3 +1,7 @@ +--- +status: draft +--- + # PR 7 Design — Geo Lookup + Client Info (Extract-Once) > Phase 1, PR 7 of the EdgeZero migration. diff --git a/docs/superpowers/specs/drafts/2026-04-02-ec-kv-schema-extensions-design.md b/docs/superpowers/specs/drafts/2026-04-02-ec-kv-schema-extensions-design.md new file mode 100644 index 000000000..33ffdd50e --- /dev/null +++ b/docs/superpowers/specs/drafts/2026-04-02-ec-kv-schema-extensions-design.md @@ -0,0 +1,500 @@ +--- +status: draft +--- + +# EC KV Schema Extensions + +**Status:** Draft +**Author:** Trusted Server Product +**Last updated:** 2026-04-03 +**Extends:** `docs/superpowers/specs/2026-03-24-ssc-prd-design.md` (§8 KV Store Identity Graph) +**Based on:** IABTechLab/trusted-server#582 + +--- + +## Overview + +This document specifies additive changes to the EC KV identity graph schema +introduced in PR #582. It does not replace the original PRD — it amends §8.2 +(schema) with four new namespaces and extends two existing structs. + +**Motivation:** Cross-property identity resolution for publisher consortiums +(e.g. Arena Group sharing an EC passphrase across autoblog.com, menshealth.com, +etc.) requires durable per-domain visit history. Corporate VPN disambiguation +requires a lazily-evaluated network cluster signal. Cross-browser identity +propagation (Chrome→Safari on the same device) requires durable device class +signals derived from JA4 TLS fingerprints and UA platform parsing. + +--- + +## 1. Schema version bump: `v: 1` → `v: 2` + +All new fields are `Option`-typed, so existing `v: 1` entries deserialize +without error. The version bump signals to future readers that `pub_properties`, +`network`, and `device` may be present. + +--- + +## 2. `KvGeo` — add `asn` and `dma` + +Both fields are available from Fastly's `geo_lookup()` on the client IP and +are non-PII network signals. + +```rust +pub struct KvGeo { + pub country: String, + pub region: Option, + /// Autonomous System Number (e.g. `7922` = Comcast). + /// Primary signal for distinguishing home ISP vs. corporate VPN. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub asn: Option, + /// DMA/metro code (e.g. `807` = San Francisco). + /// Market-level targeting signal; not personal data. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub dma: Option, +} +``` + +**Written:** on initial `KvEntry::new()` from `GeoInfo`. Never updated after +creation — geo is a first-seen signal, not a real-time one. + +**Source:** `GeoInfo::metro_code` → `dma`; a new `asn` field to be added to +`GeoInfo` from Fastly's `Geo::as_number()`. + +--- + +## 3. New `KvPubProperties` — publisher domain history + +Tracks which publisher properties a user has been seen on, keyed by apex domain. +Enables consortium-level identity sharing without cross-site tracking: history +only accumulates within a shared-passphrase group (same EC hash). + +```rust +pub struct KvPubProperties { + /// Apex domain where this EC entry was first created. + pub origin_domain: String, + /// Per-domain visit history, keyed by apex domain. + /// Updated on each organic request; capped at 50 entries. + #[serde(default, skip_serializing_if = "HashMap::is_empty")] + pub seen_domains: HashMap, +} + +pub struct KvDomainVisit { + /// Unix timestamp (seconds) of first visit to this domain. + pub first: u64, + /// Unix timestamp (seconds) of most recent visit to this domain. + pub last: u64, + /// Lifetime visit count for this domain. + pub visits: u32, +} +``` + +Added to `KvEntry`: + +```rust +#[serde(default, skip_serializing_if = "Option::is_none")] +pub pub_properties: Option, +``` + +**Written:** on `create_or_revive` (sets `origin_domain`, adds first +`seen_domains` entry). Updated on `update_last_seen` — the existing 300-second +debounce applies, so `visits` and `last` are incremented at most once per 5 +minutes per key. + +**Cap:** `seen_domains` is capped at 50 entries. If the cap is reached, new +domains are silently dropped (log at `debug`). This prevents unbounded growth +for shared-passphrase consortiums with many properties. + +### JSON example + +```json +"pub_properties": { + "origin_domain": "autoblog.com", + "seen_domains": { + "autoblog.com": { "first": 1774921179, "last": 1774985000, "visits": 4 }, + "menshealth.com": { "first": 1774985001, "last": 1774990000, "visits": 1 } + } +} +``` + +--- + +## 4. New `KvNetwork` — cluster disambiguation + +Tracks how many distinct EC entries share the same hash prefix. A high count +indicates a shared network (corporate VPN, campus); a low count indicates an +individual or household. + +```rust +pub struct KvNetwork { + /// Number of distinct EC suffixes matching this hash prefix. + /// `None` = not yet evaluated. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub cluster_size: Option, + /// Unix timestamp (seconds) of last cluster check. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub cluster_checked: Option, +} +``` + +Added to `KvEntry`: + +```rust +#[serde(default, skip_serializing_if = "Option::is_none")] +pub network: Option, +``` + +**Written:** only by the `/identify` endpoint, never on the organic proxy path. +The prefix-match list API call required to compute `cluster_size` is too +expensive for the hot path. + +**Re-evaluation TTL:** re-check if `cluster_checked` is older than 1 hour +(configurable via `trusted-server.toml`). + +### Threshold guidance + +| Cluster size | Likely scenario | +| ------------ | ----------------------------------------- | +| 1–3 | Individual / household | +| 4–10 | Small shared space (family, small office) | +| 11–50 | Medium office, hotel, coworking | +| 50+ | Corporate VPN, university, campus | + +**Default trust threshold:** entries with `cluster_size <= 10` are treated as +individual users for identity resolution purposes. Configurable per publisher +via `trusted-server.toml`: + +```toml +[ec] +cluster_trust_threshold = 10 # default +``` + +B2B publishers (trade media, finance) should raise this to 50+ since their +readers are frequently on office networks. + +--- + +## 5. New `KvDevice` — browser class and bot detection + +Captures coarse, non-PII device signals derived from the TLS handshake and UA +at EC creation time. Used by the `/identify` endpoint to make cross-suffix +propagation decisions and to signal buyer-facing device quality. + +### 5.1 Signal derivation + +No Client Hints are used — JA4 and UA platform parsing provide equivalent or +superior signal for every browser including Safari and Firefox, which do not +send Client Hints. + +**`is_mobile`** — derived in priority order: + +| Condition | Value | +| ---------------------------------------------- | -------------------------------------------------------------------------- | +| UA contains `iPhone`, `iPad`, or `Android` | `1` — confirmed mobile | +| UA contains `Macintosh`, `Windows`, or `Linux` | `0` — confirmed desktop | +| Neither pattern matches | `2` — genuinely unknown (rare; typically bots or heavily hardened clients) | + +Note: `is_mobile: 2` in practice signals a non-standard client rather than +Safari, since Safari always produces a recognizable UA platform string. + +**`ja4_class`** — Section 1 of the JA4 fingerprint only (e.g. `t13d1516h2`). +Available via `req.get_tls_ja4()` in the Fastly Compute Rust SDK. Section 1 +identifies browser family (cipher count, extension count, ALPN) without +uniquely fingerprinting a device. The full JA4 is never stored. + +**`platform_class`** — coarse OS family parsed from UA: + +| UA segment | `platform_class` | +| --------------------------- | ---------------- | +| `Macintosh; Intel Mac OS X` | `mac` | +| `Windows NT` | `windows` | +| `iPhone; CPU iPhone OS` | `ios` | +| `iPad; CPU OS` | `ios` | +| `Linux; Android` | `android` | +| `Linux` (non-Android) | `linux` | +| No match | `null` | + +**`h2_fp_hash`** — first 12 hex characters of SHA256 of the raw HTTP/2 +SETTINGS fingerprint string, available via `req.get_client_h2_fingerprint()`. +Used alongside `ja4_class` to confirm browser family and detect bots. + +**`known_browser`** — set `true` when `ja4_class` + `h2_fp_hash` match a +known legitimate browser pattern from the allowlist below. Set `false` when +they match a known bot/scraper pattern. `null` when unknown. + +### 5.2 Known browser fingerprint allowlist + +Empirically derived from Fastly Compute production responses (2026-04-03): + +| Browser | `ja4_class` | `h2_fp` prefix | `known_browser` | +| ------------------ | ------------ | -------------------------------- | --------------- | +| Chrome/Mac (v146) | `t13d1516h2` | `1:65536;2:0;4:6291456;6:262144` | `true` | +| Safari/Mac (v26) | `t13d2013h2` | `2:0;3:100;4:2097152` | `true` | +| Safari/iOS (v26) | `t13d2013h2` | `2:0;3:100;4:2097152` | `true` | +| Firefox/Mac (v149) | `t13d1717h2` | `1:65536;2:0;4:131072;5:16384` | `true` | + +Safari Mac and Safari iOS share identical TLS/H2 stacks — distinguished only +by `platform_class` (`mac` vs `ios`) and `is_mobile` (`0` vs `1`). + +This allowlist will expand as new browser versions are observed in production. +Entries not matching any allowlist row get `known_browser: null` (not `false`) +unless they match a confirmed bot pattern. + +### 5.3 Rust struct + +```rust +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +pub struct KvDevice { + /// Mobile signal: 0 = confirmed desktop, 1 = confirmed mobile, + /// 2 = genuinely unknown (non-standard client). + /// Derived from UA platform string — no Client Hints required. + pub is_mobile: u8, + /// JA4 Section 1 only — browser family class identifier. + /// e.g. "t13d1516h2" = Chrome, "t13d2013h2" = Safari, "t13d1717h2" = Firefox. + /// Never stores the full JA4 fingerprint. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub ja4_class: Option, + /// Coarse OS family from UA: "mac", "windows", "ios", "android", "linux". + #[serde(default, skip_serializing_if = "Option::is_none")] + pub platform_class: Option, + /// SHA256 prefix (12 hex chars) of the HTTP/2 SETTINGS fingerprint. + /// Used alongside ja4_class for browser confirmation and bot detection. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub h2_fp_hash: Option, + /// true = known legitimate browser; false = known bot/scraper; null = unknown. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub known_browser: Option, +} +``` + +Added to `KvEntry`: + +```rust +#[serde(default, skip_serializing_if = "Option::is_none")] +pub device: Option, +``` + +**Written:** on `create_or_revive`. Never updated after creation — device +signals are a first-seen record of how this EC entry was established. + +### 5.4 Bot gate + +Device signals are derived on every request as pure in-memory computation — +no KV I/O. The result gates all downstream KV and cookie operations: + +| `known_browser` | KV entry created | Cookie set | Partner IDs written | +| --------------- | ---------------- | ---------- | ------------------- | +| `true` | Yes | Yes | Yes | +| `false` | **No** | **No** | **No** | +| `null` | **No** | **No** | **No** | + +`null` (unrecognised client) is treated the same as `false`. An advertiser +cannot bid on a session we cannot verify as human — allowing `null` entries +into the identity graph would degrade buyer trust with no offsetting benefit. + +**Current bot response:** the request is served normally (proxied to origin) +without any KV operations or cookie writes. The bot receives a valid HTML +response but leaves no trace in the identity graph. + +**Future bot response (see issue #81):** this pass-through behaviour is a +placeholder. The detection point will evolve into conditional routing that +returns HTTP 402 + an RSL Open License Protocol challenge for crawlers that +can acquire a license. The detection logic (`known_browser` derivation) is +intentionally separated from the response decision to make this transition +additive rather than a rewrite. + +### 5.5 Privacy rationale + +`ja4_class` (Section 1 only) and `platform_class` are category signals, not +unique device identifiers. They are equivalent in precision to `geo.country` +— they identify a class of client, not an individual. The full JA4 fingerprint +(Sections 2 and 3) is never stored, as it approaches unique device +identification and would require explicit consent basis under GDPR Art. 4(1). + +--- + +## 6. `KvMetadata` — add `cluster_size`, `is_mobile`, and `known_browser` + +Allows batch sync and `/identify` fast paths to make propagation and quality +decisions without streaming the full body: + +```rust +pub struct KvMetadata { + pub ok: bool, + pub country: String, + pub v: u8, + /// Mirrors [`KvNetwork::cluster_size`]. `None` = not yet evaluated. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub cluster_size: Option, + /// Mirrors [`KvDevice::is_mobile`]. Enables propagation gating without body read. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub is_mobile: Option, + /// Mirrors [`KvDevice::known_browser`]. Buyer-facing quality signal. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub known_browser: Option, +} +``` + +Worst-case metadata size with all additions: ~90 bytes — well within the +2048-byte Fastly limit. + +--- + +## 7. IP address storage policy + +Raw IP addresses are personal data under GDPR (CJEU _Breyer v. Germany_, 2016) +and must not be stored in KV entries. The EC hash already derives from the IP +without persisting it. + +Permitted IP-derived signals (written at creation time): + +- `geo.country` — ISO 3166-1 alpha-2 +- `geo.region` — ISO 3166-2 subdivision +- `geo.asn` — ASN number (network identifier, not personal data) +- `geo.dma` — DMA/metro code (market identifier, not personal data) + +--- + +## 8. Updated full `KvEntry` shape (v: 2) + +Three representative entries showing the Chrome seed, Safari/Mac propagation +target, and Safari/iOS mobile entry. + +**Chrome/Mac (seed entry):** + +```json +{ + "v": 2, + "created": 1775162556, + "last_seen": 1775162556, + "consent": { + "tcf": "CP...", + "gpp": "DBA...", + "ok": true, + "updated": 1775162556 + }, + "geo": { "country": "US", "region": "TN", "asn": 7922, "dma": 659 }, + "device": { + "is_mobile": 0, + "ja4_class": "t13d1516h2", + "platform_class": "mac", + "h2_fp_hash": "a3f9d21c8b04", + "known_browser": true + }, + "pub_properties": { + "origin_domain": "autoblog.com", + "seen_domains": { + "autoblog.com": { "first": 1775162556, "last": 1775162556, "visits": 1 } + } + }, + "network": { "cluster_size": 2, "cluster_checked": 1775162556 }, + "ids": { + "id5": { "uid": "ID5*qe8VHv...", "synced": 1775162556 }, + "trade_desk": { + "uid": "226fb4b3-6032-405a-a5a5-4fe4d6303932", + "synced": 1775162556 + }, + "liveramp_ats": { "uid": "Ag2z1TDAfChu...", "synced": 1775162556 }, + "lockr": { + "uid": "b545e78c-2c4f-4fd3-8a99-32c02ada962d", + "synced": 1775162556 + }, + "prebid_sharedid": { + "uid": "16d913a7-d56c-4e0d-8036-d0dce637707e", + "synced": 1775162556 + } + } +} +``` + +**Safari/Mac (same machine — `platform_class: mac` + differing `ja4_class` → propagate):** + +```json +{ + "v": 2, + "created": 1775165000, + "last_seen": 1775165000, + "consent": { "ok": true, "updated": 1775165000 }, + "geo": { "country": "US", "region": "TN", "asn": 7922, "dma": 659 }, + "device": { + "is_mobile": 0, + "ja4_class": "t13d2013h2", + "platform_class": "mac", + "h2_fp_hash": "f7c341a92e18", + "known_browser": true + }, + "pub_properties": { + "origin_domain": "autoblog.com", + "seen_domains": { + "autoblog.com": { "first": 1775165000, "last": 1775165000, "visits": 1 } + } + }, + "network": { "cluster_size": 2, "cluster_checked": 1775165000 }, + "ids": { + "id5": { "uid": "ID5*qe8VHv...", "synced": 1775162556 }, + "trade_desk": { + "uid": "226fb4b3-6032-405a-a5a5-4fe4d6303932", + "synced": 1775162556 + }, + "liveramp_ats": { "uid": "Ag2z1TDAfChu...", "synced": 1775162556 }, + "lockr": { + "uid": "b545e78c-2c4f-4fd3-8a99-32c02ada962d", + "synced": 1775162556 + }, + "prebid_sharedid": { + "uid": "16d913a7-d56c-4e0d-8036-d0dce637707e", + "synced": 1775162556 + } + } +} +``` + +**Safari/iOS (mobile carrier ASN 21928 — individual device signal):** + +```json +{ + "v": 2, + "created": 1775168000, + "last_seen": 1775168000, + "consent": { "ok": true, "updated": 1775168000 }, + "geo": { "country": "US", "region": "TN", "asn": 21928, "dma": 659 }, + "device": { + "is_mobile": 1, + "ja4_class": "t13d2013h2", + "platform_class": "ios", + "h2_fp_hash": "f7c341a92e18", + "known_browser": true + }, + "pub_properties": { + "origin_domain": "autoblog.com", + "seen_domains": { + "autoblog.com": { "first": 1775168000, "last": 1775168000, "visits": 1 } + } + }, + "network": { "cluster_size": 1, "cluster_checked": 1775168000 }, + "ids": {} +} +``` + +**Updated `KvMetadata`:** + +```json +{ + "ok": true, + "country": "US", + "v": 2, + "cluster_size": 2, + "is_mobile": 0, + "known_browser": true +} +``` + +--- + +## 9. Open questions + +- Should `seen_domains` cap (50) be configurable, or is a hardcoded sentinel sufficient? +- Should `cluster_checked` re-evaluation TTL (1 hour) be per-publisher config or global? +- Should `pub_properties.seen_domains` be written on sync-pixel requests (non-organic) or only on organic HTML proxy requests? +- Should the known browser allowlist be hardcoded or configurable via `partner_store`? Hardcoded is simpler but requires a deploy to add new browser versions. +- Should `ja4_class` and `h2_fp_hash` be surfaced in `/identify` responses and `user.ext` for buyer-facing device quality scoring? diff --git a/docs/superpowers/specs/drafts/2026-04-02-ec-kv-seeding-design.md b/docs/superpowers/specs/drafts/2026-04-02-ec-kv-seeding-design.md new file mode 100644 index 000000000..a2408429d --- /dev/null +++ b/docs/superpowers/specs/drafts/2026-04-02-ec-kv-seeding-design.md @@ -0,0 +1,658 @@ +--- +status: draft +--- + +# EC KV Store Seeding + +**Status:** Draft +**Author:** jevans / TS Product +**Last updated:** 2026-04-03 +**Depends on:** IABTechLab/trusted-server#582 (EC identity system) +**Extends:** `docs/superpowers/specs/2026-03-24-ssc-prd-design.md` (§8 KV Store Identity Graph) +**Also see:** `docs/superpowers/specs/2026-04-02-ec-kv-schema-extensions-design.md` (§5 `KvDevice`) + +--- + +## Overview + +PR #582 establishes three KV write paths: pixel sync (`GET /sync`), S2S batch +push (`POST /_ts/api/v1/sync`), and pull sync (post-send background fetch). All +three require a prior sync event — a partner pixel, a batch job, or a registered +pull endpoint — before any UID lands in the identity graph. + +This spec adds two seeding paths that populate the identity graph **before any +sync event**, using signals already present on the organic request: + +1. **First-party signal collection** — read known partner ID cookies that the + browser already carries, extract UIDs, and write them to KV at request time. + Zero new infrastructure required on the publisher side; the partner's + existing client-side tags have already done the work. + +2. **HEM resolution** — when a publisher sends a hashed email via a request + header, call LiveRamp's API server-side to resolve a RampID and write it to + KV. Scoped to publishers with meaningful login rates; not applicable to + open-web publishers like autoblog.com where login rates are sub-1%. + +Both paths respect existing consent gating (`consent.ok = true`) and use the +existing `upsert_partner_id` write path in `kv.rs`. + +A third, pre-seeding concern sits above both: **bot detection**. All KV write +paths — EC creation, first-party signal collection, HEM resolution, and +cross-browser propagation — are gated on `known_browser`. Non-human clients +bypass the graph entirely and receive a pass-through response. + +--- + +## 0. Bot Detection Gate + +### 0.1 Rationale + +Fastly KV Store charges per operation at list price (Class A writes: $0.65 / +100 k; Class B reads: $0.55 / M; we assume other CDNs will follow similar +pricing or be cheaper). Bots, crawlers, and LLM scrapers are high-volume, +low-value clients: advertisers will not bid on them, they inflate write costs, +and their presence pollutes the identity graph. Writing a KV entry for a +non-human client produces zero CPM lift and measurable cost. + +Bot detection also provides forward-compatibility with RSL / HTTP 402 responses +(see issue [#81](https://github.com/IABTechLab/trusted-server/issues/81)). + +### 0.2 Detection signals + +Bot classification uses three signals computed at the edge before any KV I/O: + +| Signal | Source | How used | +| ------------------------ | ----------------------- | ------------------------------------------------------ | +| `known_browser` | JA4 Section 1 allowlist | Primary gate — see below | +| `ja4_class` cipher count | `req.get_tls_ja4()` | Count > 25 → confirmed bot | +| UA string | `User-Agent` header | Platform class derivation; empty/absent → treat as bot | + +JA4 is available via `req.get_tls_ja4()` in the Fastly Compute Rust SDK. +H2 fingerprint is available via `req.get_client_h2_fingerprint()`. + +**Known browser allowlist** (Section 1 of JA4 only — browser class, not +unique device): + +| Browser | `ja4_class` (JA4 §1) | +| -------------------- | -------------------- | +| Chrome / Chromium | `t13d1516h2` | +| Safari (Mac and iOS) | `t13d2013h2` | +| Firefox | `t13d1717h2` | + +Any JA4 Section 1 value not in this allowlist sets `known_browser = null`. +Confirmed bot patterns (cipher count > 25, or curl/libcurl fingerprints) set +`known_browser = false`. + +### 0.3 Gate logic + +``` +known_browser = classify_client(ja4, user_agent) + +if known_browser is false or known_browser is null: + → Skip EC creation + → Skip cookie write + → Skip first-party signal collection + → Skip HEM resolution + → Skip cross-browser propagation + → Pass request through to origin unchanged + → Return +``` + +Both `false` (confirmed bot) and `null` (unrecognised client) block all KV +operations. The distinction between the two values is preserved in `KvDevice` +for future routing logic but has no behavioural difference in this version. + +### 0.4 Updated request flow + +``` +Request received + → classify_client(ja4, ua) ← NEW — bot gate + known_browser false or null → pass through, return + → EcContext::read_from_request + → EcContext::generate_if_needed + → collect_first_party_signals(ec, jar) + → finalize + send response + → dispatch_pull_sync (post-send) + → dispatch_hem_resolution(ec, hem) (post-send, if applicable) +``` + +### 0.5 Forward-compatibility: RSL / HTTP 402 + +The current behaviour for bots is a silent pass-through — the request reaches +origin as if TS were not present. This is intentionally minimal. + +A future iteration (tracked in issue +[#81](https://github.com/IABTechLab/trusted-server/issues/81)) will evolve this +into a conditional HTTP 402 response for unlicensed crawlers, using the IAB +Real Simple Licensing (RSL) / Open License Protocol (OLP). When that work +lands, the `known_browser = false` branch becomes the insertion point for the +402 challenge. The `null` branch may continue to pass through, or may be routed +to a separate RSL handler — TBD in the issue #81 spec. + +No structural changes to the gate logic will be needed; the branching point is +already isolated. + +### 0.6 Cost impact + +At 100 M monthly uniques, bot filtering is estimated to reduce KV writes by +~15–25 % based on industry crawl-rate benchmarks. Full cost modelling at +100 M uniques: ~$6,500 month 1 (high write volume), ~$3,400/month steady +state — with bot filtering applied before any KV operation. + +--- + +## 1. First-party Signal Collection + +### 1.1 Motivation + +A Chrome user on autoblog.com already carries IDs from ~12 partners in their +first-party cookie jar. These IDs are readable by TS server-side on the +publisher's own domain. When the same user's household visits on Safari or +Firefox — where those cookies were never set — the identity graph has no +partner UIDs, bidstream is unenriched, and CPM is degraded. + +Seeding the KV entry from first-party signals at request time means: by the +time that user's Safari browser arrives, the KV entry already has IDs, and the +`/identify` response is fully populated without waiting for any sync event. + +### 1.2 Approach: PartnerRecord-driven signal table + +First-party signal collection configuration lives on `PartnerRecord`, not in +`trusted-server.toml`. `PartnerRecord` is already the single source of truth +for all per-partner behaviour; adding signal collection config there keeps it +consistent with pull sync, batch sync, and bidstream settings. + +**Alternative considered:** a separate `[[ec.fp_signals]]` TOML table. +Rejected because it splits partner config across two locations, requiring +operators to keep them in sync manually. + +New fields on `PartnerRecord`: + +```rust +/// One or more first-party cookie names that may carry this partner's UID. +/// Checked in order; first match wins. +#[serde(default, skip_serializing_if = "Vec::is_empty")] +pub fp_signal_cookie_names: Vec, + +/// Optional JSON path to extract the UID from a cookie whose value is a +/// JSON object (e.g. `"universal_uid"` for id5, `"v.userId"` for kargo). +/// When absent, the raw cookie value is used as the UID. +#[serde(default, skip_serializing_if = "Option::is_none")] +pub fp_signal_json_path: Option, + +/// Minimum seconds between re-collection writes for this partner. +/// Prevents write thrashing when the cookie changes on every request. +/// Defaults to 86400 (24 hours). Re-collection is skipped if +/// `ids[partner].synced` is within this window. +#[serde(default = "PartnerRecord::default_fp_signal_ttl_sec")] +pub fp_signal_ttl_sec: u64, +``` + +### 1.3 Known partner cookie mapping + +Derived from a real autoblog.com Chrome cookie jar (2026-04-02): + +| Partner ID | `fp_signal_cookie_names` | `fp_signal_json_path` | Notes | +| ----------------- | ---------------------------------------- | --------------------- | --------------------------------------------------- | +| `id5` | `["id5id"]` | `"universal_uid"` | Value is JSON object | +| `trade_desk` | `["pbjs-unifiedid"]` | `"TDID"` | Value is JSON object; check `TDID_LOOKUP == "TRUE"` | +| `liveramp_ats` | `["idl_env"]` | — | Raw envelope string; opaque to TS | +| `lockr` | `["lockr_tracking_id"]` | — | Raw UUID string | +| `kargo` | `["krg_uid"]` | `"v.userId"` | Doubly-nested JSON | +| `prebid_sharedid` | `["sharedId", "_sharedid", "_sharedID"]` | — | Multiple cookie names, same UID | +| `lotame` | `["panoramaId"]` | — | Raw hex string | +| `audigent` | `["_au_1d"]` | — | Raw string | +| `yahoo_connectid` | `["connectId"]` | `"connectId"` | Value is JSON object | +| `lotame_cc` | `["_cc_id"]` | — | Raw hex string | +| `uid2` | `["__uid2_advertising_token"]` | `"advertising_token"` | Short-TTL token; see §1.6 | +| `arena` | `["ArenaID", "_ig"]` | — | Arena Group first-party ID | + +This table ships as the default partner registry seed. Publishers register +partners via `POST /_ts/admin/partners/register`; the first-party signal +fields are included in the registration payload. + +### 1.4 Request flow + +First-party signal collection runs in the organic handler after +`EcContext::generate_if_needed` and before response finalization. It is +post-EC and post-consent — the EC must exist and consent must be `ok` before +any write occurs. + +``` +Request received + → EcContext::read_from_request (parse cookie jar, extract EC ID) + → EcContext::generate_if_needed (create EC + initial KV entry if new) + → collect_first_party_signals(ec, jar) ← NEW + → finalize + send response + → dispatch_pull_sync (post-send) +``` + +`collect_first_party_signals` pseudocode: + +``` +for partner in partner_store.all_with_fp_signal_config(): + uid = extract_uid_from_jar(jar, partner.fp_signal_cookie_names, partner.fp_signal_json_path) + if uid is None: continue + + existing = kv.get(ec_id).ids.get(partner.id) + if existing and now - existing.synced < partner.fp_signal_ttl_sec: continue + + kv.upsert_partner_id(ec_id, partner.id, uid, now) + log::debug!("Collected first-party {} UID for EC {}", partner.id, ec_hash) +``` + +The existing `upsert_partner_id` CAS loop handles concurrent writes safely — +no new concurrency logic needed. + +### 1.5 JSON path extraction + +`fp_signal_json_path` uses dot-notation for nested fields: + +| Path | Input | Extracted | +| ----------------- | ----------------------------------------- | --------------------- | +| `"universal_uid"` | `{"universal_uid":"ID5*...","version":1}` | `"ID5*..."` | +| `"v.userId"` | `{"v":{"userId":"d8f4..."}}` | `"d8f4..."` | +| `"connectId"` | `{"connectId":"7vsQ..."}` | `"7vsQ..."` | +| _(absent)_ | `16d913a7-d56c-...` | `"16d913a7-d56c-..."` | + +Implementation: split on `.`, walk the JSON tree, extract string leaf. If +parsing fails or path is missing, log at `debug` and skip — never error. + +### 1.6 UID2 special handling + +`__uid2_advertising_token` contains a short-TTL advertising token +(`identity_expires` is ~1 hour from issue). The cookie value is a JSON object: + +```json +{ + "advertising_token": "A4AAADA...", + "refresh_token": "AAAAMCQR...", + "identity_expires": 1775421703943, + "refresh_expires": 1777754503943, + "refresh_from": 1775166103943 +} +``` + +Harvest policy: only write the advertising token if `identity_expires > +now_ms + 300_000` (at least 5 minutes of validity remaining). Do not store +the refresh token in KV — it is a credential, not an identity signal. + +Set `fp_signal_ttl_sec = 3600` for UID2 to align with token lifetime. + +UID2 token refresh (calling the UID2 `/token/refresh` endpoint to extend +lifetime) is deferred — it requires a separate operator integration and is +not part of this spec. + +### 1.7 Consent gating + +First-party signal collection inherits the existing consent check: no write +occurs unless `ec_context.ec_allowed()` is true. This is already enforced at +`generate_if_needed`; collection runs after that gate, so no additional +consent check is needed. + +### 1.8 Error handling + +All collection errors are swallowed and logged at `warn`. A collection failure +must never affect the client response. This matches the degraded-behavior +policy from PRD §8.6. + +### 1.9 Performance + +First-party signal collection adds one KV read (existing entry for TTL check) +and up to N KV writes (one per partner with a missing or stale UID) to the +organic path. To bound latency: + +- Partners with no first-party signal config are skipped in O(1) via the + existing `pull_enabled_index` pattern — a `_fp_signal_enabled` secondary + index in `partner_store` lists only partners with `fp_signal_cookie_names` + populated. +- KV writes for already-fresh UIDs are skipped via the TTL check. +- In practice, most requests are returning users — all UIDs already collected, + zero writes, one metadata read per relevant partner. + +--- + +## 2. HEM Resolution (LiveRamp) + +### 2.1 Scope + +Scoped to publishers with logged-in user populations. Not applicable to +open-web publishers (login rates < 1%). A publisher like autoblog.com should +not enable this; a publisher with a subscription wall or registration gate +(trade media, sports, news) should. + +For autoblog.com and similar properties, `idl_env` is already set by LiveRamp +ATS.js on Chrome, and is collected by strategy 1 (`fp_signal_cookie_names: +["idl_env"]`). Strategy 2 is specifically for the cold-start case: a +logged-in user whose `idl_env` was never set because they browse primarily +on Safari or Firefox. + +### 2.2 Header convention + +The publisher signals a logged-in user by adding a request header at their +origin or CDN layer before proxying to TS: + +``` +X-ts-hem: +``` + +The publisher is responsible for normalizing (lowercase, trim) and hashing +the email before sending. TS never receives or processes plaintext email. + +If the header is absent or malformed (not a 64-character lowercase hex string), +HEM resolution is skipped silently. + +### 2.3 PartnerRecord extension for HEM resolution + +HEM resolution is modeled as a pull-sync variant on `PartnerRecord`. New fields: + +```rust +/// Whether this partner supports HEM-based identity resolution. +#[serde(default)] +pub hem_resolution_enabled: bool, + +/// HTTPS endpoint to call with the hashed email for UID resolution. +/// Required when `hem_resolution_enabled` is true. +#[serde(default, skip_serializing_if = "Option::is_none")] +pub hem_resolution_url: Option, + +/// Allowlist of domains TS may call for HEM resolution. +#[serde(default, skip_serializing_if = "Vec::is_empty")] +pub hem_resolution_allowed_domains: Vec, + +/// JSON path to extract the resolved UID from the API response. +#[serde(default, skip_serializing_if = "Option::is_none")] +pub hem_resolution_response_path: Option, + +/// Publisher ID or client ID required by the partner's API. +/// Included as a query parameter or header per partner spec. +#[serde(default, skip_serializing_if = "Option::is_none")] +pub hem_resolution_publisher_id: Option, + +/// Minimum seconds between HEM resolution calls for the same EC. +/// Defaults to 86400 (24 hours). +#[serde(default = "PartnerRecord::default_hem_ttl_sec")] +pub hem_resolution_ttl_sec: u64, +``` + +Validation mirrors `validate_pull_sync_config`: when `hem_resolution_enabled`, +both `hem_resolution_url` and `hem_resolution_allowed_domains` must be present, +URL must be HTTPS, and hostname must be in the allowed domains list. + +### 2.4 LiveRamp API integration + +LiveRamp's server-side HEM resolution uses their RampID API. The exact endpoint +and request shape must be confirmed against LiveRamp's current API documentation +and the publisher's LiveRamp account configuration (publisher ID / placement ID). + +**Expected shape (to be confirmed with LiveRamp):** + +``` +GET https://api.rlcdn.com/api/identity/v1/envelope + ?pid= + &it=4 (identifier type: SHA-256 email) + &iv= +Authorization: Bearer +``` + +Response: + +```json +{ "envelope": "" } +``` + +`hem_resolution_response_path = "envelope"` extracts the value. + +The resolved value is written to KV as `ids["liveramp_ats"].uid`, consistent +with how the `idl_env` cookie harvest writes it. Downstream auction code sees +no difference between a harvested `idl_env` and an HEM-resolved RampID. + +**Action required before implementation:** confirm endpoint, auth scheme, +rate limits, and `pid` parameter with LiveRamp account team. Record confirmed +values in this spec before engineering begins. + +### 2.5 Dispatch timing + +HEM resolution dispatches post-send, after client response is flushed — same +pattern as `dispatch_pull_sync`. It must never add latency to the organic +request path. + +``` +Request received + → EcContext::read_from_request + → EcContext::generate_if_needed + → collect_first_party_signals(ec, jar) + → finalize + send response + → dispatch_pull_sync (existing) (post-send) + → dispatch_hem_resolution(ec, hem) (post-send) ← NEW +``` + +### 2.6 TTL and staleness + +Once a RampID is written to KV for a given EC, re-resolution is skipped until +`hem_resolution_ttl_sec` has elapsed (default 24 hours). The `ids["liveramp_ats"].synced` +timestamp is used for the staleness check — same field as cookie harvest. + +### 2.7 Consent gating + +HEM resolution only dispatches when `ec_context.ec_allowed()` is true. No +additional consent check beyond the existing EC gate. + +### 2.8 Error handling + +All HEM resolution errors are swallowed post-send. Log at `warn` on API +failure, `debug` on skip (stale, no HEM header). Never propagate to client. + +--- + +## 3. Cross-browser Propagation + +### 3.1 Purpose + +First-party signal collection (§1) populates the KV entry for the browser that +carries those cookies — typically Chrome on a desktop. Cross-browser propagation +copies those IDs to sibling entries sharing the same EC hash prefix, enabling +Safari and Firefox users to benefit from IDs already resolved on Chrome without +waiting for a sync event. + +### 3.2 When to propagate + +Propagation runs inside the `/identify` endpoint, which already performs the +prefix-match list query for `cluster_size`. No new I/O is required — device +signals from `KvMetadata` are read during the same operation. + +Decision table (evaluated in order, first match wins): + +| Condition | Decision | +| ----------------------------------------------------------------------- | --------------------------------------------------------------------------- | +| `known_browser` is `false` or `null` | **Never** — non-human client | +| `cluster_size > threshold` (default 10) | **Never** — corporate/shared network | +| `geo.asn` is a known mobile carrier ASN | **Propagate** — individual device confirmed | +| Source `platform_class` == target `platform_class`, `ja4_class` differs | **Propagate** — same machine, different browser (e.g. Chrome→Safari on Mac) | +| Source `platform_class` != target `platform_class`, `cluster_size` <= 3 | **Propagate** — probable personal device | +| Anything else | **Skip** — insufficient confidence | + +### 3.3 The same-machine case + +The Chrome→Safari same-Mac case is **deterministic**, not probabilistic: + +- Chrome/Mac: `ja4_class: t13d1516h2`, `platform_class: mac`, `is_mobile: 0` +- Safari/Mac: `ja4_class: t13d2013h2`, `platform_class: mac`, `is_mobile: 0` + +`platform_class` matches, `ja4_class` differs → same OS, different browser → +propagate with full confidence. No cluster size check needed. + +Safari/Mac and Safari/iOS share identical JA4 and H2 fingerprints (Apple uses +the same TLS stack across platforms). `platform_class` (`mac` vs `ios`) is the +sole distinguishing signal, making it load-bearing for this decision. + +### 3.4 The `is_mobile: 2` signal + +`is_mobile: 2` (unknown) now in practice means a non-standard client, not +Safari — because Safari always produces a recognizable UA platform string +(`iPhone`, `iPad`, `Macintosh`). An entry arriving with `is_mobile: 2` +alongside an unrecognized `ja4_class` should be treated as a potential bot +and excluded from propagation regardless of cluster size. + +### 3.5 What is propagated + +Only `ids` entries are copied — not `device`, `geo`, or `pub_properties`. Each +suffix entry retains its own device fingerprint and geo signals. Only the +resolved partner UIDs are shared, since those represent identity assertions +the user has already authorized across the consortium. + +### 3.6 Compliance note + +Propagation is scoped to suffix entries sharing the same hash prefix. All +entries in the prefix group derived from the same IP + passphrase combination. +Within a small cluster on a home ASN, propagation is equivalent to a CRM +recognizing the same household across devices — an established and accepted +practice in both digital advertising (CTV household graphs) and privacy +regulation guidance (ICO cookie guidance, EDPB shared device guidance). + +The `cluster_size` threshold is the primary guard against household +cross-contamination. Publishers with concern about household-level matching +can set `cluster_trust_threshold = 1` in `trusted-server.toml` to disable +propagation entirely. + +--- + +## 4. Configuration example + +```toml +[ec] +passphrase = "" +ec_store = "ec_store" +partner_store = "partner_store" +``` + +Partners are registered via `POST /_ts/admin/partners/register`. Example +registration payload for id5 with first-party signal collection enabled: + +```json +{ + "id": "id5", + "name": "ID5", + "allowed_return_domains": ["id5-sync.com"], + "api_key": "", + "bidstream_enabled": true, + "source_domain": "id5-sync.com", + "openrtb_atype": 3, + "sync_rate_limit": 10, + "fp_signal_cookie_names": ["id5id"], + "fp_signal_json_path": "universal_uid", + "fp_signal_ttl_sec": 86400 +} +``` + +Example for LiveRamp with both first-party signal collection and HEM resolution: + +```json +{ + "id": "liveramp_ats", + "name": "LiveRamp ATS", + "allowed_return_domains": ["ats.rlcdn.com"], + "api_key": "", + "bidstream_enabled": true, + "source_domain": "liveramp.com", + "openrtb_atype": 3, + "sync_rate_limit": 10, + "fp_signal_cookie_names": ["idl_env"], + "fp_signal_ttl_sec": 86400, + "hem_resolution_enabled": true, + "hem_resolution_url": "https://api.rlcdn.com/api/identity/v1/envelope", + "hem_resolution_allowed_domains": ["api.rlcdn.com"], + "hem_resolution_response_path": "envelope", + "hem_resolution_publisher_id": "", + "hem_resolution_ttl_sec": 86400 +} +``` + +--- + +## 5. Legal and Compliance + +### 5.1 Why this isn't considered ID Bridging + +ID bridging — sometimes called ID laundering — is the practice of taking an +identifier established in one consent context and using it to resolve or track +a user in a different context where they have no relationship and have not +consented. It is non-compliant with GDPR Purpose Limitation (Art. 5(1)(b)), +TCF v2, and IAB Tech Lab guidance, and is under active regulatory scrutiny. + +Trusted Server's EC system is categorically different on every dimension that +makes ID bridging problematic: + +**Consent is a structural gate, not a policy.** An EC cannot be created without +a valid TCF Purpose 1 signal (EU/UK) or passing GPP opt-out check (applicable +US states). On consent withdrawal, the cookie and the KV entry are deleted in +real time. There is no path through the code that creates or maintains an EC +without consent. + +**First-party signal collection persists authorizations the user already gave.** +When TS reads `idl_env`, `id5id`, or similar cookies from the browser, those +cookies were set by LiveRamp ATS.js, ID5's SDK, or equivalent — on the +publisher's own domain, under the publisher's CMP consent dialog, with the +user's prior consent. TS is durably storing an identity link that the user +already authorized. No new linkage is created; no new consent basis is required. + +**The publisher controls the passphrase and the partner registry.** The EC hash +is `HMAC-SHA256(IP, publisher_passphrase)`. The publisher chooses who is in +their partner registry and which signals are collected. There is no third-party +infrastructure making inferences the publisher hasn't sanctioned. + +**The system is deterministic and auditable.** Given an IP address and a +passphrase, the EC hash is reproducible. A regulator, an auditor, or a +publisher's legal team can verify exactly how a hash was derived. Probabilistic +ID bridging systems cannot offer this. + +### 5.2 Consortium scope + +Publishers sharing a passphrase (e.g. a media group across multiple +properties) are recognizing their own reader across their own publications — +the same practice as a CRM matching a subscriber across a company's owned +brands. This is first-party data activation: the publisher has a direct +relationship with the user on each property, and the user's relationship is +with the media group, not a single URL. The relevant privacy policy covers all +consortium properties. + +This is distinct from cross-site tracking, where an identity vendor uses a +signal from Publisher A to buy media on Publisher B without the user's +knowledge of that link. + +### 5.3 Deletion + +TS implements the IAB Data Subject Rights DSR deletion framework. The EC hash +is the registered identifier. A deletion request triggers cookie expiry, KV +entry deletion, and propagation to all registered partners. This chain of +custody is a compliance requirement before any regulated publisher goes live +(see PRD §7.4). + +--- + +## 6. What is not stored + +Per the IP address storage policy in the schema extensions spec: + +- Raw email addresses — never received by TS; publisher hashes before sending +- Plaintext IP addresses — never stored (hash derivation only) +- UID2 refresh tokens — credentials, not identity signals +- Cookie values that fail JSON path extraction — silently skipped, not stored +- Full JA4 fingerprint (Sections 2 and 3) — approaches unique device identification; only Section 1 (`ja4_class`) is stored +- Raw H2 fingerprint string — stored only as a 12-char SHA256 prefix (`h2_fp_hash`) +- Client Hints headers — not used; JA4 and UA platform parsing provide equivalent signal + +--- + +## 7. Open questions + +- Should `_fp_signal_enabled` secondary index in `partner_store` be implemented + from day one, or is a full scan acceptable given typical partner counts (< 20)? +- What is the correct LiveRamp HEM API endpoint, auth scheme, and `pid` + parameter format? Needs confirmation with LiveRamp account team before + implementation begins. +- Should `trade_desk` harvest be conditional on `TDID_LOOKUP == "TRUE"` in the + cookie JSON, or is presence of `TDID` sufficient? (The lookup flag indicates + a confirmed server-side match, not just a local guess.) +- Should UID2 token refresh be a phase-2 follow-on to this spec, or deferred + to a separate initiative? diff --git a/docs/superpowers/specs/drafts/2026-04-18-microsoft-monetize-server-side-ad-templates-codex-reviewed-design.md b/docs/superpowers/specs/drafts/2026-04-18-microsoft-monetize-server-side-ad-templates-codex-reviewed-design.md new file mode 100644 index 000000000..73a341a2f --- /dev/null +++ b/docs/superpowers/specs/drafts/2026-04-18-microsoft-monetize-server-side-ad-templates-codex-reviewed-design.md @@ -0,0 +1,843 @@ +--- +status: draft +--- + +# Microsoft Monetize Integration with Server-Side Ad Templates - Codex Reviewed Spec + +_April 2026_ + +--- + +## Codex Review Status + +This is a Codex-reviewed rewrite of +`2026-04-16-microsoft-monetize-server-side-ad-templates-design.md`. + +The original concept is preserved: Trusted Server should move auction and ad +server decisioning off the page-rendering path, call Microsoft Monetize +server-side, and render creatives through lightweight browser-side iframe +injection. + +This reviewed version tightens the product and technical claims in four areas: + +1. `srcdoc` iframe isolation is not sufficient by itself. Creative rendering + must use an explicit sandbox policy or a separate creative origin. +2. `defer` scripts can delay `DOMContentLoaded`. The ad bundle should be + loaded with `async` and use a DOM-ready/slot-ready guard before injection. +3. Fast first-byte streaming of a JavaScript response is not the user-visible + performance win. The value is early request start plus server-side decisioning. +4. Page HTML is cache-compatible, not universally cacheable by URL path. Real + cache eligibility must account for publisher personalization, cookies, + query strings, consent, auth state, and page invalidation. + +This spec is recommended for a proof-of-concept, not yet a full production ad +stack replacement. + +--- + +## 1. Executive Summary + +Trusted Server will support a Microsoft Monetize server-side ad delivery path +for a single-publisher proof-of-concept. + +When enabled, the page response does no auction work. Trusted Server injects one +early-loading ad-bundle script tag into ``. That script request runs the +batched server-side auction, sends the auction evidence and slot context to +Microsoft Monetize, receives final creative markup, and injects sandboxed +iframes into known ad slots. + +The goal is to prove that above-the-fold display ads can become visible in under +1 second on eligible pages while removing Prebid.js, GPT, APS tags, and other +client-side ad SDKs from the rendering path. + +The POC should be measured against real publisher traffic with explicit +guardrails: fill rate, revenue impact, viewability, time to iframe insertion, +time to iframe load, creative paint/visible signal where available, +`DOMContentLoaded` impact, CLS, and error/no-fill rates. + +--- + +## 2. Product Goals + +Enable Trusted Server to: + +1. Serve eligible page HTML without waiting for ad auctions or ad-server calls. +2. Start ad decisioning early through one browser-requested ad-bundle endpoint. +3. Run all configured demand sources in one batched edge-side auction. +4. Pass full auction evidence and slot context to Microsoft Monetize for final + ad-server decisioning. +5. Render returned creatives into sandboxed iframes at pre-defined slot + positions. +6. Preserve publisher revenue safety through a kill switch, rollout controls, + and observable fallback behavior. + +Target POC outcome: + +- P75 above-the-fold iframe insertion under 1 second on cache-eligible pages. +- No material regression to `DOMContentLoaded`, CLS, or content rendering. +- No material revenue or fill regression versus the control cohort. +- Clear evidence that removing browser-side ad SDK work reduces ad render time. + +--- + +## 3. Non-Goals + +- Replacing the publisher's full ad stack in the first launch. +- Supporting mixed-mode pages where some slots are Monetize-rendered and others + are GPT-rendered. +- Dynamic DOM slot discovery. +- Building Microsoft Monetize wire-format support before Microsoft confirms the + endpoint, authentication model, request format, response format, and creative + execution requirements. +- Guaranteeing that all publisher HTML can be cached by path alone. +- Guaranteeing creative viewability measurement unless Microsoft confirms the + returned creatives are self-contained inside the iframe. + +--- + +## 4. Dependency: Server-Side Ad Templates + +This design depends on the server-side ad template work: + +- `creative-opportunities.toml` +- URL pattern matching against the incoming page path +- `CreativeOpportunitySlot` +- conversion from template slots into auction `AdSlot` values + +The Monetize path should reuse that slot definition layer, but it should not +reuse the ad-template spec's browser-facing `window.__ts_bids`, +`window.__ts_ad_slots`, or `__tsAdInit` model. + +When `ad_server.provider = "microsoft_monetize"`, the ad-template output is: + +- one ad-bundle script tag in `` +- no browser-visible bid objects +- no GPT bootstrap for the matched slots + +When the provider is absent or configured for the existing GAM path, the +ad-template behavior remains unchanged. + +--- + +## 5. Reviewed Architecture + +### 5.1 Two-Request Model + +Request 1: page HTML + +- Match request URL to configured creative opportunities. +- If Monetize is enabled and at least one slot matches, inject one ad-bundle + script tag into ``. +- Do not run an auction on the page request. +- Do not block ``, ``, or origin streaming on ad decisioning. +- Preserve compatibility with existing HTML rewriting and integration hooks. + +Request 2: ad bundle script + +- Validate the requested page and slot set. +- Build auction request context from the original browser request. +- Run all configured auction providers in one batched server-side auction. +- Send full auction evidence and slot context to Microsoft Monetize. +- Return JavaScript that injects sandboxed iframes into matching slot divs. + +No KV store is required between requests. + +However, the ad-bundle request must not trust arbitrary client-provided slot +IDs or page paths. It should either recompute slots from the page path on the +server or verify a stateless signed token emitted by the page response. + +### 5.2 Script Loading Model + +The reviewed design uses `async`, not `defer`: + +```html + +``` + +Rationale: + +- `async` starts the fetch early without blocking HTML parsing. +- `async` execution does not hold `DOMContentLoaded`. +- The ad-bundle code can wait until the target slot elements exist before + injection. + +The generated script must be safe to execute before or after the slot divs are +available. It should use a small helper: + +- try `document.getElementById(slot_id)` immediately +- if missing and document is still loading, retry on `DOMContentLoaded` +- optionally use a short bounded `MutationObserver` for late-rendered slots +- never throw if a slot is absent + +This is a material change from the original spec. The product goal is not just +fast ad rendering; it is fast ad rendering without delaying core page lifecycle +events. + +### 5.3 Request Validation + +The original design passes `slots` and `page` as query parameters. That is +convenient but insufficient as a trust boundary. + +The endpoint must implement one of these validation models: + +Preferred for POC: + +1. Page request emits `page`, `slots`, and `token`. +2. `token` is an HMAC over: + - publisher ID + - request path + - slot IDs + - creative-opportunities config version + - expiry timestamp +3. `/ts/ad-bundle` rejects requests with invalid, expired, or mismatched tokens. + +Acceptable alternative: + +1. `/ts/ad-bundle` accepts only `page`. +2. The endpoint recomputes matching slots from `creative-opportunities.toml`. +3. Any client-provided `slots` parameter is ignored or used only as a hint. + +Do not allow arbitrary browsers to request arbitrary slot IDs against arbitrary +page contexts. That weakens auction integrity, reporting quality, and abuse +resistance. + +### 5.4 Sequence + +Cached or cache-eligible page: + +```text +t=0ms Browser requests page +t=5ms Trusted Server serves eligible edge-cached or fast-streamed HTML +t=15ms Browser parses and starts /ts/ad-bundle async request +t=25ms /ts/ad-bundle validates token and builds auction context +t=25ms Batched PBS/APS auction starts +t=525ms Auction deadline reached or all providers complete +t=525ms Microsoft Monetize request starts with slot context and auction evidence +t=625ms Monetize returns final creative markup +t=635ms Ad bundle completes; browser injects sandboxed iframes +t=650ms ATF iframe inserted; creative load/paint measured separately +``` + +Uncached page: + +```text +t=0ms Browser requests page +t=150ms Origin HTML arrives or streams through Trusted Server +t=160ms Browser parses and starts /ts/ad-bundle async request +t=170ms /ts/ad-bundle validates token and starts batched auction +t=670ms Auction deadline reached or all providers complete +t=770ms Monetize returns final creative markup +t=785ms ATF iframe inserted; creative load/paint measured separately +``` + +These numbers are POC assumptions, not guarantees. The measurement plan must +separate: + +- ad-bundle request start +- auction complete +- Monetize complete +- iframe inserted +- iframe load event +- creative visible or measurable event, if available + +### 5.5 Streaming Script Response + +Streaming the first bytes of the JavaScript response may still be useful for +transport behavior and observability, but it is not the primary user-visible +performance win because the browser cannot execute an external script until the +full script is received. + +The endpoint may stream: + +Phase 1: + +```javascript +(function(){var c= +``` + +Phase 2: + +```javascript +{"slot":{"m":"...","w":300,"h":250}};/* injection code */})(); +``` + +But the spec must not claim that Phase 1 makes ads visible sooner. The real +performance value comes from: + +- moving ad work out of the page request +- starting the ad-bundle fetch early +- running auction/ad-server calls from the edge +- eliminating large browser-side SDK load and parse work + +Implementation note: the current Fastly adapter should be validated for true +client streaming before depending on chunked response behavior. If the adapter +still returns buffered `Response` values, the POC can still prove most of the +product value, but it should not claim immediate ad-bundle TTFB until +`stream_to_client()` support is implemented for this route. + +--- + +## 6. Ad Decisioning Contract + +### 6.1 Decisioning Principle + +Microsoft Monetize should be treated as the final ad server decisioner, not +only as a creative wrapper around Trusted Server's pre-selected winning bids. + +The original spec sends only `winning_bids` to Monetize. That may be too narrow. +It can prevent Monetize from correctly considering: + +- direct-sold campaigns +- guaranteed and sponsorship priorities +- deal priority +- pacing +- frequency constraints +- APS encoded-price demand +- no-bid slots that Monetize can fill directly +- future auction/provider diagnostics + +The reviewed design passes the full auction result and the slot list to the ad +server client. Monetize can then decide how much of that evidence it supports. + +### 6.2 Reviewed `AdServerClient` Trait + +```rust +/// Client for server-side ad-server decisioning. +/// +/// Implementations receive the full auction result and slot context so the ad +/// server can perform final decisioning across direct-sold and programmatic +/// demand. +pub trait AdServerClient: Send + Sync { + /// Request final ad creatives from the ad server. + /// + /// # Errors + /// + /// Returns [`AdServerError`] when the ad-server request fails, times out, + /// or returns an unsupported payload. + fn request_creatives( + &self, + auction: &OrchestrationResult, + slots: &[CreativeOpportunitySlot], + context: &AdServerRequestContext, + ) -> Result>; +} + +/// Context passed alongside auction evidence and slots. +pub struct AdServerRequestContext { + /// Full page URL that triggered the ad-bundle request. + pub page_url: String, + /// Publisher identifier in Trusted Server configuration. + pub publisher_id: String, + /// Microsoft member, seat, or placement identifier as configured. + pub monetize_member_id: String, + /// Edge Cookie ID when consent permits use. + pub ec_id: Option, + /// Request user-agent. + pub user_agent: Option, + /// Client IP or platform-provided client address when consent and policy allow. + pub ip: Option, + /// Referrer from the browser request. + pub referrer: Option, + /// Consent strings and decoded consent context. + pub consent: Option, + /// Geo information from the platform lookup. + pub geo: Option, + /// Whether this request is part of a test cohort. + pub test_mode: bool, +} +``` + +The exact wire format remains dependent on Microsoft documentation. If Microsoft +supports OpenRTB 2.6, `MicrosoftMonetizeClient` should map the context and +auction evidence into OpenRTB fields and extensions. If Microsoft expects +Prebid-style key-values, the implementation should perform that mapping behind +the trait. + +### 6.3 Creative Response Shape + +```rust +pub struct AdServerResponse { + /// Creative results keyed by slot ID. + pub creatives: HashMap, + /// Total ad-server response time in milliseconds. + pub response_time_ms: u64, + /// Provider-specific diagnostics safe for logs and metrics. + pub diagnostics: HashMap, +} + +pub struct CreativeResult { + /// HTML creative markup to render inside the sandboxed iframe. + pub markup: String, + /// Creative width. + pub width: u32, + /// Creative height. + pub height: u32, + /// Impression tracking URLs if they are not already embedded in markup. + pub impression_urls: Vec, + /// Click-through URL if it is separate from markup. + pub click_url: Option, + /// Optional creative ID for logging and diagnostics. + pub creative_id: Option, + /// Optional line item, campaign, or deal identifier. + pub decision_id: Option, +} +``` + +--- + +## 7. Creative Rendering and Isolation + +### 7.1 Required Iframe Policy + +Creatives must not be injected as raw HTML into the publisher DOM. + +Creatives also must not be placed in an unrestricted `srcdoc` iframe. The +generated iframe must use an explicit sandbox policy. + +Baseline POC policy: + +```javascript +f.setAttribute( + 'sandbox', + 'allow-scripts allow-popups allow-popups-to-escape-sandbox allow-forms' +) +f.setAttribute('referrerpolicy', 'strict-origin-when-cross-origin') +f.srcdoc = c[id].m +``` + +Do not include `allow-same-origin` unless Microsoft confirms it is required and +Trusted Server serves the creative from a separate origin. `allow-scripts` plus +`allow-same-origin` on a same-origin `srcdoc` iframe can undermine the sandbox. + +### 7.2 Sanitization and Rewriting + +Before serializing creative markup into the ad-bundle script, Trusted Server +should apply the existing creative processing path where compatible: + +- sanitize unsafe creative markup +- rewrite external resource URLs through the first-party proxy when required by + publisher policy +- preserve known-good impression and click tracking behavior + +If sanitization or URL rewriting breaks Microsoft creative behavior, the POC +must document the tradeoff explicitly and use a separate creative origin or +another containment strategy instead of unrestricted parent-origin `srcdoc`. + +### 7.3 Layout Reservation + +`creative-opportunities.toml` should include enough information to reserve slot +space before the ad arrives. At minimum: + +- expected ATF width and height +- allowed responsive sizes +- collapse behavior for no-fill +- whether the slot is above-the-fold or lazy + +The injector should set stable iframe dimensions from the ad-server response and +the publisher template should reserve the expected space to avoid layout shift. + +--- + +## 8. `/ts/ad-bundle` Endpoint + +### 8.1 Route + +```text +GET /ts/ad-bundle?page={url_path}&slots={slot_ids}&token={signed_token} +``` + +For the recompute-only validation model: + +```text +GET /ts/ad-bundle?page={url_path}&token={signed_token} +``` + +### 8.2 Behavior + +1. Validate request method and query parameters. +2. Validate the stateless token or recompute slots from page path. +3. Look up matched `CreativeOpportunitySlot` records. +4. Build an `AuctionRequest` from slot config and browser request context. +5. Extract EC ID and consent using the same server-side path as `/auction`. +6. Run `AuctionOrchestrator::run_auction()` with a POC-specific timeout. +7. Call `AdServerClient::request_creatives()` with the full orchestration + result, slots, and ad-server context. +8. Sanitize/rewrite creative markup according to publisher policy. +9. Serialize creative results into a generated JavaScript response. +10. The generated script injects sandboxed iframes when slots are available. + +### 8.3 Error Behavior + +- Invalid token: return a small no-op JavaScript response with 403 or 200, + depending on monitoring preference. For POC observability, prefer 403. +- No matching slots: return a no-op JavaScript response. +- Auction timeout: call Monetize with the bids and provider responses available + before timeout. +- Monetize timeout: return a no-op JavaScript response and log the failure. +- Creative parse/sanitization failure: omit that creative, keep other slots. +- Slot not found in DOM: no-op for that slot. +- Script fails to load: page renders without ads. + +### 8.4 Response Headers + +```text +Content-Type: application/javascript; charset=utf-8 +Cache-Control: no-store +X-Content-Type-Options: nosniff +``` + +Use `Server-Timing` diagnostics where available: + +```text +Server-Timing: auction;dur=500, monetize;dur=100, serialize;dur=2 +``` + +If true streaming is implemented: + +```text +Transfer-Encoding: chunked +``` + +Do not set `Transfer-Encoding` manually unless the platform requires it. Prefer +the platform streaming API to manage transfer details. + +### 8.5 Generated JavaScript Shape + +The generated JavaScript should be compact but explicit: + +```javascript +;(function () { + var c = CREATIVE_JSON + function render() { + Object.keys(c).forEach(function (id) { + var r = c[id] + var el = document.getElementById(id) + if (!el || el.getAttribute('data-ts-ad-rendered') === '1') return + var f = document.createElement('iframe') + f.width = String(r.w) + f.height = String(r.h) + f.style.border = '0' + f.scrolling = 'no' + f.setAttribute( + 'sandbox', + 'allow-scripts allow-popups allow-popups-to-escape-sandbox allow-forms' + ) + f.setAttribute('referrerpolicy', 'strict-origin-when-cross-origin') + f.srcdoc = r.m + el.setAttribute('data-ts-ad-rendered', '1') + el.appendChild(f) + }) + } + render() + if (document.readyState === 'loading') { + document.addEventListener('DOMContentLoaded', render, { once: true }) + } +})() +``` + +For late-rendered slots, the POC may add a bounded `MutationObserver`. Keep the +observer scoped and time-limited to avoid a permanent page-wide observer. + +--- + +## 9. HTML Injection + +The HTML processor should inject the ad-bundle tag only when all are true: + +- Monetize ad-server path is enabled. +- The page URL matches one or more creative opportunity slots. +- The request is eligible for this POC cohort. +- The existing page is not excluded by route, auth state, or cache policy. + +The injected tag should be placed early in ``: + +```html + +``` + +If other non-ad Trusted Server integrations are enabled, the normal `tsjs` bundle +can still be injected. The Monetize ad rendering path must not depend on `tsjs`. + +For implementation in the current repo, this likely fits best as a dedicated +head injector or a small extension to `html_processor.rs`, but the slot-matching +context must be available to the HTML processor. Avoid spreading ad-server +state across unrelated integration modules. + +--- + +## 10. Cache Strategy + +The reviewed position: page HTML is cache-compatible, not automatically +cacheable by path. + +Cache eligibility must account for: + +- publisher route type +- query string behavior +- auth/paywall state +- cookies that personalize page content +- consent/CMP state +- geo/device variants +- A/B testing +- preview/editor routes +- page content freshness +- `creative-opportunities.toml` version + +Recommended POC cache policy: + +1. Start with a narrow allowlist of article routes. +2. Exclude logged-in, paywalled, preview, admin, checkout, and personalized + pages. +3. Include query strings in the cache key unless the publisher confirms they do + not vary content. +4. Vary only on the smallest necessary set of headers/cookies. +5. Purge HTML when publisher content changes or when creative opportunity config + changes. +6. Keep `/ts/ad-bundle` `no-store` for every page view. + +The page response should not be the only place EC cookies are established, +because cached HTML may bypass per-user origin-like response behavior. The +ad-bundle endpoint and existing Trusted Server routes can still set EC headers +and cookies when consent permits. + +--- + +## 11. Configuration + +```toml +[ad_server] +provider = "microsoft_monetize" +enabled = true +timeout_ms = 150 +auction_timeout_ms = 500 +rollout_percent = 5 +kill_switch = false + +[ad_server.microsoft_monetize] +endpoint = "https://..." +member_id = "..." +auth_mode = "tbd" +test_mode = true + +[creative_opportunities] +path = "creative-opportunities.toml" +config_version = "2026-04-18-poc-1" +``` + +Open items for implementation: + +- whether `creative_opportunities` belongs in `[ad_server]`, `[auction]`, or a + top-level section +- whether `auction_timeout_ms` should override `settings.auction.timeout_ms` only + for ad-bundle requests +- how rollout cohorts are assigned and logged +- whether token signing uses the existing request-signing infrastructure or a + separate HMAC secret + +--- + +## 12. POC Guardrails + +The POC must have: + +- global kill switch +- rollout percentage +- publisher route allowlist +- no-fill-safe behavior +- timeout budgets for auction and Monetize separately +- error logging with sampled diagnostics +- explicit control cohort +- daily revenue/fill monitoring +- fast rollback to existing GAM/GPT path + +Recommended default budget: + +```text +auction_timeout_ms = 500 +monetize_timeout_ms = 150 +total_ad_bundle_budget_ms = 700 +``` + +If the ad bundle exceeds the total budget, return no-fill JavaScript rather than +letting connections hang. + +--- + +## 13. Measurement Plan + +Server-side metrics: + +- ad-bundle requests +- token validation failures +- matched slot count +- auction provider timings +- auction bid count +- winning bid count +- Monetize request timing +- Monetize fill count +- creative sanitization/rewrite failures +- no-fill count by reason +- generated script byte size + +Client-side marks emitted by generated script: + +- `ts-ad-bundle-exec` +- `ts-ad-render-start` +- `ts-ad-iframe-inserted:{slot_id}` +- `ts-ad-iframe-load:{slot_id}` where browser permits +- `ts-ad-slot-missing:{slot_id}` sampled or aggregated + +Browser/page metrics: + +- `DOMContentLoaded` +- LCP +- CLS +- INP where available +- ad iframe insertion time +- ad iframe load time +- viewability signal where available from Microsoft/verification vendors + +Business metrics: + +- fill rate +- CPM/RPM +- viewability +- click-through rate +- revenue per session +- ad-blocker interaction if measurable +- discrepancy versus Microsoft reporting + +Success criteria should be set before live traffic. Suggested POC success: + +- P75 ATF iframe insertion under 1 second on eligible pages +- P75 iframe load materially faster than control +- no statistically meaningful regression in revenue per eligible page view +- no material CLS regression +- no material `DOMContentLoaded` regression + +--- + +## 14. Edge Cases + +No matching slots: + +- no ad-bundle tag is injected, or the endpoint returns no-op JavaScript. + +Slot exists in config but not DOM: + +- injector no-ops for that slot and emits sampled diagnostic. + +Auction returns no bids: + +- call Monetize with empty/zero programmatic evidence for those slots so direct + sold demand can still fill, if Microsoft supports that model. + +APS returns encoded prices: + +- pass APS evidence through to Monetize if Microsoft can interpret it, or require + configured mediation before APS participates in this POC. + +Monetize returns creative requiring parent-page SDK: + +- reject for the initial POC or add the companion SDK explicitly to scope. Do + not silently claim SDK-free rendering if a parent SDK is required. + +Creative requires same-origin iframe access: + +- do not use unrestricted `srcdoc`. Use a separate creative origin or decline + that creative class for the POC. + +JavaScript disabled: + +- no ads render. This is equivalent to most JavaScript-based ad stacks. + +Ad blocker blocks `/ts/ad-bundle`: + +- page renders without ads. Consider route naming if blockers target obvious ad + paths, but do not obscure behavior in a way that violates publisher or user + expectations. + +--- + +## 15. Implementation Scope + +### New + +- `crates/trusted-server-core/src/ad_server/mod.rs` +- `crates/trusted-server-core/src/ad_server/config.rs` +- `crates/trusted-server-core/src/ad_server/endpoints.rs` +- `crates/trusted-server-core/src/ad_server/microsoft_monetize.rs` +- `crates/trusted-server-core/src/creative_opportunities.rs` +- token signing/validation helper for ad-bundle URLs +- tests for generated JavaScript escaping and sandbox attributes + +### Modified + +- `crates/trusted-server-core/src/settings.rs` +- `crates/trusted-server-core/src/html_processor.rs` +- `crates/trusted-server-adapter-fastly/src/main.rs` +- `trusted-server.toml` +- docs and POC runbook + +### Validate Before Depending On + +- true Fastly response streaming for `/ts/ad-bundle` +- interaction with current `#[fastly::main]` return-based adapter model +- whether creative sanitization/rewrite is compatible with Microsoft creatives +- whether Microsoft supports server-side OpenRTB decisioning with the required + direct-sold and programmatic semantics + +--- + +## 16. Open Questions for Microsoft + +1. What exact server-side endpoint should Trusted Server call? +2. Is the endpoint standard OpenRTB 2.6, Microsoft-specific OpenRTB extensions, + or a proprietary request format? +3. What authentication model is required? +4. Can Monetize perform final decisioning with upstream auction evidence from + PBS and APS? +5. Should Trusted Server send all bids, winning bids only, or ad-server + key-values? +6. How should APS encoded-price bids be represented? +7. Are returned banner creatives fully self-contained inside an iframe? +8. Do creatives require a parent-page SDK, SafeFrame, or Microsoft JavaScript + library? +9. Can creatives run inside a sandboxed iframe without `allow-same-origin`? +10. Are impression, click, and viewability trackers embedded in `adm`, or + returned separately? +11. What reporting identifiers should Trusted Server persist in logs? +12. Does Microsoft require win-notification, billing-notification, or render + notification calls from Trusted Server? + +--- + +## 17. Product Recommendation + +Proceed with the POC, but position it as an ATF display speed experiment rather +than a complete ad stack replacement. + +The highest-confidence parts of the concept are: + +- edge-side batched auction +- early ad-bundle request +- removing browser-side SDK load/parse +- server-side consent and identity handling +- single publisher, narrow route allowlist + +The main risks to retire before live traffic are: + +- Microsoft API and creative execution requirements +- iframe sandbox compatibility +- ad-bundle request validation +- cache eligibility +- real creative load/viewability timing +- revenue and fill impact + +The original direction is strong. The POC becomes much more defensible when the +claims shift from "full isolation and fully cacheable HTML" to "validated, +cache-compatible, SDK-free server-side ad rendering with sandboxed creative +delivery and explicit measurement." diff --git a/docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md b/docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md new file mode 100644 index 000000000..17b41684a --- /dev/null +++ b/docs/superpowers/specs/drafts/2026-04-28-generate-feature-docs-skill-design.md @@ -0,0 +1,384 @@ +--- +status: draft +--- + +# Generate Feature Docs Skill Design + +_April 2026_ + +## 1. Overview + +A Claude Code skill that converts implemented engineering design specs into publisher-facing documentation pages on the Trusted Server VitePress site. The skill runs in two interactive stages: an extraction pass that produces a structured outline for user review, and a generation pass that writes prose, applies mechanical updates to reference docs, and shows a diff before any commit. + +The skill exists to close a recurring gap. Specs are written, code ships, but the docs at iabtechlab.github.io/trusted-server do not get updated. The team is moving fast and doc-writing is the work that gets dropped. This skill compresses that work from hours of authoring to minutes of review, on demand. + +## 2. Audience + +The generated docs are aimed at publishers and integrators running Trusted Server. They want concrete answers: what a feature is, when to enable it, how to configure it, and what its API contract looks like. The output voice and structure match existing pages in `docs/guide/`. Where applicable, generated docs include concrete API specifics: endpoint paths, request and response headers, request and response shapes, and error variant names. + +## 3. Goals + +1. Produce a publishable feature page from one implemented spec, in one invocation, that the spec's author would merge with light editing rather than rewrite from scratch. +2. Apply mechanical, additive updates to `configuration.md`, `api-reference.md`, and `error-reference.md` when a spec introduces new config keys, endpoints, headers, or error variants. +3. Verify every concrete handle (config key, file path, endpoint, header, error variant) against the actual code before writing prose. Mismatches surface as outline issues, not as silent false claims in generated docs. +4. Surface drift between spec and code at the outline stage, before any prose is written. +5. Keep human authors in control. The user reviews the outline, can redirect any field, reviews the diff, and explicitly approves the commit. + +## 4. Non-goals + +The following are deliberately out of scope for this skill, to keep the implementation focused and to define a clean handoff to a future skill #2 (spec-vs-reality gap analysis). + +1. Detecting drift between spec and code _behavior_. The skill verifies that handles exist; it does not verify that the code does what the spec says it does. That is skill #2. +2. Updating non-reference narrative docs. `getting-started.md`, `gdpr-compliance.md`, `architecture.md`, and similar pages are humans' responsibility. +3. Translating, localizing, or summarizing the spec for marketing. +4. Generating diagrams. If a Sequence section is needed, the skill emits a numbered list. Mermaid diagrams are added by humans in follow-up edits. +5. Touching code under `crates/`. The skill is read-only against the codebase. +6. Pulling engineering feedback from GitHub PR review comments. Discussed in Section 13 (Out of scope, deferred to skill #2). +7. Running in CI. The skill is interactive by design. CI integration is a future option once the skill is mature. + +## 5. Skill identity and invocation + +**Skill location:** `.claude/skills/generate-feature-docs/SKILL.md` at the repo root. Project-level so the convention ships in git and the whole team gets it via `git pull`. + +**Slash command:** `.claude/commands/generate-feature-docs.md`. A thin file that takes `$ARGUMENTS` (the spec path) and delegates to the skill. Matches the existing convention for `check-ci.md`, `verify.md`, and the other project-level commands. + +**Invocation:** `/generate-feature-docs `. The argument is a path to a spec file under `docs/superpowers/specs/implemented/`. If the user invokes the command without an argument, the skill resolves to the most recent file in that directory and confirms the choice before proceeding. + +**Output contract:** the skill writes only to: + +- One file under `docs/guide/.md` (created or augmented). +- Up to three additive updates to `docs/guide/configuration.md`, `docs/guide/api-reference.md`, and `docs/guide/error-reference.md`. + +The skill never writes anything else, never opens PRs, never pushes, never deploys, never modifies code under `crates/`, and never modifies the spec it is reading. + +## 6. Spec readiness convention + +The skill operates only on specs that the team considers final. To make that signal explicit, every spec carries YAML frontmatter: + +```yaml +--- +status: implemented +implemented_in: PR#581 +last_reviewed: 2026-04-15 +--- +``` + +Three accepted values for `status`: + +- `draft`: brainstorm output. Not ready for documentation. +- `in-progress`: implementation in flight, design may still evolve. +- `implemented`: code has shipped, spec reflects what shipped, ready to document. + +Three optional fields: + +- `implemented_in`: PR number where the implementation landed. Used by future skill #2. +- `last_reviewed`: date of the most recent engineering review of the spec, in `YYYY-MM-DD` format. +- `verified_against_commit`: commit SHA the engineer asserts the spec was verified against at promotion time. Audit trail; the skill records but does not validate. Added in Section 16.3. + +**Skill behavior on `status`:** + +- `implemented`: proceed normally (subject to the verification-rate gate added in Section 16.1). +- Any other value, or missing `status` field: skill stops and prompts `Continue without status: implemented? Reply y to proceed.` Wait for the user's reply. Treat any reply other than a single `y` (case-insensitive) as abort. Explicit `y` proceeds with a one-line warning that the docs may drift from product. The override path is intentionally a little annoying so it does not become the default. + +The skill never adds frontmatter on the user's behalf. Frontmatter is added at spec-authoring time, by the brainstorming skill (for new specs) or by hand (for existing specs). Section 12 lists the one-time backfill of the 12 existing specs as a prerequisite. + +## 7. Directory layout for specs + +Specs live under `docs/superpowers/specs/`, split into two subdirectories by lifecycle stage: + +- `docs/superpowers/specs/drafts/`: brainstorm output. Every file here has `status: draft`. The brainstorming skill writes here. Engineers refine here. The doc-generation skill ignores this directory. +- `docs/superpowers/specs/implemented/`: post-implementation truth. Every file here has `status: implemented`. The doc-generation skill operates only on this directory. Engineers move specs here when implementation is complete and the spec body has been updated to match what shipped. + +Promotion is one operation: `git mv drafts/.md implemented/.md`, with body edits to reflect reality and a frontmatter update from `status: draft` to `status: implemented`. The promotion lands in a PR alongside any final spec edits. + +This directory split serves three purposes: + +1. The directory listing is the truth at a glance: drafts and implemented specs are visibly separated. +2. The promotion is a visible signal in PR review, more meaningful than a frontmatter line edit. +3. The skill's input filtering is trivial: read only from `implemented/`. + +## 8. Stage 1: extraction pass + +Read-only. Produces a structured outline shown to the user in chat. + +**Inputs:** + +- Spec file path under `docs/superpowers/specs/implemented/`. +- The codebase under `crates/`. +- The existing `docs/guide/` directory. + +**Steps:** + +1. **Validate spec readiness.** Check frontmatter `status` field. If not `implemented`, run the override prompt described in Section 6. +2. **Parse the spec.** Pull the H1 title (feature name), intro paragraph (description), section headings, and code blocks. Identify TOML config blocks, JSON examples, Rust error enums, and endpoint URLs. +3. **Detect spec kind.** Heuristic on section names: a spec with sections like "Configuration", "Public API", or "Endpoints" is a feature spec. A spec with "Migration phases" or "Rollout plan" is a migration spec. A spec with "Pre-prod checklist" is a readiness report. Only feature specs proceed automatically. Other kinds trigger a prompt: "this looks like a `` spec, not a feature spec. Continue anyway, or abort?" +4. **Resolve target page path.** Slug the feature name (e.g., "RSL AI Crawler Licensing" becomes `ai-crawler-licensing.md`) and check for `docs/guide/.md`. If a near-match exists (e.g., the spec is a v2 of an existing feature), surface the candidate so the user can confirm "augment existing" vs. "create new". +5. **Detect Sequence-section need.** Heuristic: numbered request-flow steps in the spec, or language like "first ... then ... finally". If present, mark `needs_sequence_section: yes` in the outline. +6. **Detect multi-feature specs.** If the spec has 2 or more top-level "Feature" sections, or the H1 is ambiguous, list candidate features and ask the user to choose: one page per feature, one combined page, or a subset. No default. The user must pick. +7. **Extract handles.** Walk the spec body for: config keys (TOML keys, `[section]` headers), endpoint paths (URL strings), HTTP headers (`X-...` patterns), and error variants (`SomethingError::Variant` patterns). +8. **Verify each handle against code.** Grep `crates/**/*.rs` and `trusted-server.toml` for each handle. Capture `file:line` when found. Mark `NOT FOUND` when not. +9. **Detect spec inconsistencies.** Same config key spelled two ways, two endpoints with the same path, two error variants with conflicting descriptions. Surface as an "Inconsistencies" subsection. +10. **Render the outline** as a markdown chat message and wait for the user. + +**Outline format** (rendered in chat, not written to disk): + +```markdown +## Extraction summary for `2026-04-22-rsl-ai-crawler-licensing-design.md` + +**Feature:** RSL AI Crawler Licensing +**Target page:** `docs/guide/ai-crawler-licensing.md` (NEW) +**Spec kind:** feature +**Sequence section:** yes (request, token check, log, response) + +### Config keys + +| Key | Status | Location | +| --------------- | --------- | ----------------- | +| `rsl.enabled` | verified | `settings.rs:142` | +| `rsl.allowlist` | NOT FOUND | spec only | + +### Endpoints + +| Path | Methods | Status | Location | +| ----------------------- | ------- | -------- | ------------------- | +| `/.well-known/rsl.json` | GET | verified | `rsl_handler.rs:25` | + +### Headers + +| Name | Direction | Status | Location | +| ------------- | --------- | -------- | ------------------ | +| `X-RSL-Token` | request | verified | `rsl/headers.rs:8` | + +### Error variants + +| Variant | Status | Location | +| ------------------------ | -------- | ----------------- | +| `RslError::InvalidToken` | verified | `rsl/error.rs:14` | + +### Issues + +- `rsl.allowlist` referenced in spec but not in code. Options: + (A) Mark inline as "planned, not yet shipped" + (B) Drop the row from configuration.md + (C) Pause and let me fix the spec or the code first + +Reply `proceed`, redirect specific fields, or pick A, B, or C for each issue. +``` + +The user can redirect any field. Wrong slug, wrong target page, drop a handle that is a spec typo, override the spec-kind heuristic. Substantial redirects regenerate the outline. Minor ones are noted and the skill proceeds. The skill never proceeds to stage 2 without an explicit `proceed` (or equivalent affirmative). + +The structured representation behind the rendered outline is kept internally as a JSON-shaped object, since it is the input for the future skill #2. + +## 9. Stage 2: generation pass + +Runs only after the user types `proceed`. Inputs are the spec, the approved outline, and the existing docs. Output is files written to disk; nothing is committed until the user approves the diff. + +### 9.1 Branch check before any writes + +Before stage 2 writes any files: + +1. Detect current git branch. If `main` or `master`: stop, do not write anything yet. Propose a branch name in the form `docs/` (e.g., `docs/ai-crawler-licensing`) and ask: "you are on `main`. Create branch `docs/ai-crawler-licensing` and switch to it?" The skill refuses to proceed on `main` under any circumstance. The user can specify a different branch name if they want. +2. Check working tree. If uncommitted changes exist outside the planned doc files, refuse: "uncommitted changes detected. Commit, stash, or revert them before running this skill, since the doc commit must contain only doc files." Hard stop, no override. + +### 9.2 Prose-writing rules + +The skill follows these constraints when writing prose: + +- **Voice:** second-person, direct, present tense. Match the register of `edge-cookies.md` and `integration-guide.md`. +- **No marketing language.** Forbidden words: "powerful", "seamless", "robust", "efficiently", "appropriately", "leveraging". The skill scans its own output for these and removes them. +- **No em-dashes.** Use commas, colons, or semicolons. The skill scans its own output for em-dashes and replaces them. +- **No emojis, no decorative characters, no exclamation marks.** Status indicators in tables use text (`verified`, `NOT FOUND`), not symbols. +- **VitePress-flavored markdown.** Relative links (`[link text](/guide/page)`), code blocks with language tags, callouts (`::: tip`, `::: warning`) only for genuinely non-obvious gotchas. +- **Grounding.** Every concrete reference (config key, file path, endpoint, header, error variant) must be one of the verified handles from stage 1, or an explicit `` for something the user opted into during the issues prompt. +- **Empty sections drop.** A feature with no consent implications has no Privacy section. The template is a maximum, not a minimum. +- **No filler.** If the spec does not say enough to write a section, the skill writes a single sentence, not a paragraph of speculation. + +### 9.3 Output template + +Standard sections, in order. Empty sections are omitted. + +1. **Overview.** What the feature is, who it is for. One to three short paragraphs. +2. **How it works.** Mechanism, key concepts, anything an operator needs to understand the feature's behavior at a high level. +3. **Sequence** (optional). Numbered list describing a multi-step user-visible flow. Included only when stage 1 detected this need. +4. **Configuration.** One to two paragraphs naming the config keys, with a link to `configuration.md` for the full reference table. +5. **API contract.** Endpoints, headers, request and response shapes. Code blocks for each. +6. **Error handling.** Error variants, what triggers them, what the response looks like. +7. **Privacy and consent considerations.** Included only when the feature has consent or PII implications. +8. **Related docs.** Internal links to adjacent feature pages. + +### 9.4 Augment-in-place rules for existing pages + +When the target page already exists: + +1. Walk the page's H2 and H3 structure. +2. For each template section that already exists in the page: leave existing prose alone. Only add new items, for example a new row in a config table or a new bullet in a list. Never rewrite human-authored prose for stylistic reasons. +3. For sections in the template that do not exist in the page: insert them in template order. +4. For prose that _contradicts_ the new spec or current code (e.g., a sentence mentioning a config key that no longer exists, or a behavioral claim that the spec has revised): show the existing text and the proposed replacement, and ask the user to approve, skip, or edit per item. This is the only path by which the skill rewrites existing prose. + +The default posture is conservative. Under-augmenting is recoverable; destroying a teammate's hand-edits is not. + +### 9.5 Mechanical reference-doc updates + +For each of `configuration.md`, `api-reference.md`, and `error-reference.md`: + +1. The skill reads the file first to learn its existing structural pattern (column layout in tables, section ordering, code-block formatting). +2. New entries are appended or inserted following that pattern. +3. If an entry already exists for the same key (config key, endpoint path, error variant) and the spec defines it differently: prompt the user with the existing text, the new text, and three options (overwrite, keep existing, pause). +4. All updates are otherwise additive and idempotent. Running the skill twice on the same spec produces no second diff. + +### 9.6 Diff review + +After all files are written, the skill posts a chat message: + +``` +Generated 4 files: + - [docs/guide/ai-crawler-licensing.md](docs/guide/ai-crawler-licensing.md) (NEW, 87 lines) + - [docs/guide/configuration.md](docs/guide/configuration.md) (+12 lines, 1 section updated) + - [docs/guide/api-reference.md](docs/guide/api-reference.md) (+18 lines, 1 endpoint added) + - [docs/guide/error-reference.md](docs/guide/error-reference.md) (+4 lines, 2 errors added) + +Inline TODOs: 1 (`rsl.allowlist` marked "planned, not yet shipped") + +Reply `commit`, `show diff`, or redirect a section. +``` + +File paths are clickable in the user's editor (the skill emits markdown links with relative paths). The user can: + +- `commit`: the skill creates one commit on the current branch with message `Add docs for ` (new pages) or `Update docs for ` (augmentations). The skill stages files explicitly via `git add `, never `git add -A` or `git add .`. The commit, by construction, contains only doc files. +- `show diff`: the skill prints the diff inline in chat. +- Redirect a section, e.g. "Overview is too long, cut it in half". The skill rewrites only that section and re-shows the diff. + +## 10. Edge cases and failure modes + +| Case | Behavior | +| ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| Spec lacks `status: implemented` | Prompt with `(y/N)` default N, abort unless explicit `y`. | +| Spec covers multiple features | List candidates, ask user to pick: one page per feature, combined page, or subset. No default. | +| Non-feature spec (migration, readiness, tech-spec) | Prompt: "this looks like a `` spec, continue anyway?". No automatic fallback. | +| No shipped code (zero handles verify) | Prompt: "no shipped code found. Generate stub page with sections marked 'planned, not yet shipped', or abort?". The deeper "is the behavior correct" question is for skill #2. | +| Spec is internally contradictory | Surface in stage 1 under "Inconsistencies", ask user to resolve before proceeding. | +| Target page name cannot be determined | Ask the user for the target path explicitly. | +| Spec file not found | Hard error, abort. | +| Spec file outside `docs/superpowers/specs/implemented/` | Warn once, ask "is this really an implemented spec?", then proceed if confirmed. | +| Current branch is `main` or `master` | Hard stop, no override. Propose `docs/` branch name. | +| Working tree has unrelated uncommitted changes | Hard stop, no override. User must clean up first. | +| Re-run on a spec that has already produced docs | Supported. Stage 1 finds the existing page. Stage 2 augments per Section 9.4. A clean re-run with no spec or code changes produces zero diff (idempotency). | + +## 11. Verification and validation + +The skill is considered shippable when the following validation cases pass. + +1. **Greenfield case.** Run on the RSL AI crawler licensing spec (after backfilling its frontmatter and moving it to `implemented/`). No `docs/guide/.md` exists. Skill produces a new page that an integrator could read and act on. Manual review verifies: Configuration section is complete, all referenced handles are verified against `crates/`, and the page passes `cd docs && npm run build` without broken links. +2. **Augmentation case.** Run on the EC KV schema extensions spec (after promotion) against the existing `edge-cookies.md`. Skill detects the existing page, extends it per the new spec, leaves existing prose intact except where it contradicts current code. Manual review verifies: no human-authored content was destroyed, stale-prose detection fired correctly. +3. **Non-feature case.** Run on the EdgeZero migration spec. Skill detects this is not a feature spec and prompts before proceeding. +4. **Drift case.** Run on a spec with one or more deliberately-broken handles (e.g., a config key that does not exist in code). Skill surfaces the broken handle in stage 1, the user picks an option, the generated docs reflect that choice rather than silently writing the false claim. +5. **Idempotency case.** Re-run the skill from case 1 with no intervening spec or code changes. Output is zero diff. +6. **Style case.** Search the generated docs for em-dashes, emojis, exclamation marks, and the words "powerful", "seamless", "robust", "efficiently", "leveraging". Any hit is a bug. + +**Success bar:** cases 1 and 2 produce drafts that the spec's author would merge with light editing, not start-from-scratch rewrites. Cases 3 and 4 prompt correctly without silently proceeding. Case 5 produces no diff. Case 6 finds zero violations. Generated docs build cleanly with `cd docs && npm run build`. + +This bar is partially subjective. Reasonable people disagree on what "merge with light editing" means. The skill is iterated based on real usage. When output is wrong in some pattern, the SKILL.md is edited and committed. The skill is versioned in git and improvements ship with normal PRs. + +## 12. Prerequisites + +The following one-time tasks must complete before the skill can be used in normal operation. They are listed here so the implementation plan can sequence them correctly. + +1. **Exclude internal specs from the public docs site.** Add `srcExclude` (or equivalent) to `docs/.vitepress/config.mts` so that everything under `docs/superpowers/` is omitted from the VitePress build. Verify by running `cd docs && npm run build` and confirming the spec pages are not in the output. +2. **Create directory layout.** Add `docs/superpowers/specs/drafts/` and `docs/superpowers/specs/implemented/` to the repo. Both are committed (with `.gitkeep` if empty) to establish the convention. +3. **Bulk-move existing specs to `drafts/`.** Move all 12 existing specs into `drafts/`. Add `status: draft` frontmatter to each. This is a single PR. +4. **Update CLAUDE.md.** Add a short section describing the spec-readiness convention (Sections 6 and 7) and instructing the brainstorming skill to write to `drafts/` when invoked from this project. The CLAUDE.md instruction takes precedence over the skill default per the harness rules. +5. **(For testing only) Promote one spec to `implemented/`.** To validate the skill end-to-end, one spec needs to live in `implemented/` with `status: implemented` frontmatter. The RSL AI crawler licensing spec is a good candidate since it is recent and has no corresponding guide page. Promotion is a one-step `git mv` plus frontmatter update. + +The skill itself depends on tasks 1, 2, and 4 being complete. Task 3 is necessary for clean repository state but does not block the skill mechanically. Task 5 is required only for validation, not for normal operation. + +## 13. Out of scope, deferred to skill #2 + +The companion skill, planned next, is responsible for: + +- Comparing actual code behavior against the spec (not just handle existence). +- Identifying gaps where the implementation falls short of the spec's promises. +- Producing a roadmap-style "planned for later" list with rough estimates. +- Optionally consuming GitHub PR review comments via `gh pr view --comments` to incorporate post-spec engineering decisions. + +This skill (#1) is intentionally limited to handle existence and prose generation so that skill #2 has a clear boundary. The structured outline produced by stage 1 is the contract between the two skills: skill #2 consumes the same outline shape and adds behavior verification on top. + +## 14. Related work + +**Upstream contribution to the Superpowers brainstorming skill.** The directory split (`drafts/` and `implemented/`) and the `status:` frontmatter convention are useful beyond this project. A PR to the Superpowers plugin should: + +1. Update the brainstorming skill to write new specs to `/drafts/` instead of `/`. +2. Add `status: draft` frontmatter to newly-written specs by default. +3. Document the lifecycle in the brainstorming skill's README. + +This is a separate change from skill #1, with its own PR upstream. Listed here as related work so it does not get forgotten. + +## 15. Implementation summary + +The implementation plan, written next, will sequence the prerequisites in Section 12, the slash-command and skill files described in Section 5, and the validation cases in Section 11. The skill itself is one markdown file (`.claude/skills/generate-feature-docs/SKILL.md`) plus one slash-command file (`.claude/commands/generate-feature-docs.md`). The skill content encodes the rules in Sections 6 through 10. + +## 16. Post-validation amendment: stricter readiness verification + +The first validation run (greenfield case against the JS Asset Auditor spec on 2026-04-28) exposed a workflow gap: a spec can be promoted to `implemented/` with `status: implemented` even when most of the described feature is not yet in `crates/`. The skill correctly produced a stub feature page with `` annotations, which is graceful, but the deeper signal is that promotion happened prematurely. This section documents three amendments to close that gap. They are not a re-design; they are tightening of existing rules. + +### 16.1 Verification-rate threshold + +Stage 1 already verifies each handle (config keys, endpoints, headers, error variants) against `crates/` and `trusted-server.toml`. The skill currently refuses only when zero handles verify (the "no shipped code" edge case in Section 10). Tighten this: + +- Compute a verification rate during stage 1: `verified_count / total_extracted_count`. +- If the rate is **below 50%**, surface this in the outline's "Issues" subsection as a hard prompt: + > "Stage 1 verified `` of `` handles in this codebase (`%`). Below 50% suggests this spec may not be fully implemented in this branch. Options: (A) generate stubs for unverified handles, (B) abort and check status, (C) override and proceed normally." +- The threshold of 50% is an initial value; tune based on real usage. Tracked as a known knob. +- The skill still emits the outline normally so the user can see exactly which handles failed verification before choosing. + +This catches the JS Asset Auditor case directly: 1 of ~5 handles verified, roughly 20%, well below 50%. + +### 16.2 Branch-state heuristic + +In addition to handle verification, stage 1 inspects whether the current branch has actually touched code relevant to the feature: + +```bash +git log --name-only ..HEAD -- crates/ trusted-server.toml +``` + +Where `` is the merge base with `main` (or the equivalent default branch). If the result is empty (the current branch has not touched any code), surface this as an additional signal in the outline: + +> "Note: this branch has no commits touching `crates/` or `trusted-server.toml`. If you expect the implementation to be on this branch, you may be on the wrong branch." + +This is informational, not a hard stop. It pairs with 16.1 to give a more complete picture: low verification rate combined with no code on the branch is a strong indicator the engineer is on `main` rather than on the implementation branch. + +### 16.3 `verified_against_commit` frontmatter field + +Add an optional field to the spec readiness convention (Section 6): + +```yaml +--- +status: implemented +implemented_in: PR#581 +last_reviewed: 2026-04-15 +verified_against_commit: a1b2c3d4 +--- +``` + +`verified_against_commit` records the commit SHA the engineer asserts the spec was verified against when promoted. The skill records but does not validate this field; it is an audit trail field for future review and for use by skill #2. + +Use case: if the skill produces a stub page (low verification rate), the user can later compare the recorded `verified_against_commit` against the current branch state to understand whether the promotion was premature or whether the implementation has been reverted since. + +### 16.4 Out of scope, deferred to a future enhancement + +Validation against the GitHub PR (calling `gh pr view `) was considered and explicitly deferred. Reasons: external network calls in a hot path complicate testing, require `gh` auth, and tie the skill to GitHub. If we find we still need this signal after 16.1 and 16.2, it can be added as a `--strict` flag or a CI-only mode in a follow-up. + +### 16.5 Implementation impact + +These amendments require: + +1. `SKILL.md` update: add verification-rate computation and the threshold prompt to Stage 1 (Step 1.7 or a new Step 1.7a). +2. `SKILL.md` update: add the branch-state check to Stage 1, surfaced in the outline. +3. Section 6 update: extend the frontmatter convention with the optional `verified_against_commit` field. +4. `CLAUDE.md` update: keep the documented frontmatter schema in sync with the extended Section 6. + +These amendments do NOT require: + +- Changes to the prerequisites in Section 12 (those have already landed). +- Changes to the directory layout in Section 7. +- Changes to the prose-writing rules or template in Section 9. +- Re-running the validation cases that already passed in Section 11. The amendments are additive checks and do not affect happy-path behavior. diff --git a/docs/superpowers/specs/implemented/.gitkeep b/docs/superpowers/specs/implemented/.gitkeep new file mode 100644 index 000000000..e69de29bb