v5.4.0 — audit-driven overhaul: deploy sync, model tier v2, native orchestration, hygiene

.github/workflows/ci.yml

@@ -0,0 +1,46 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  verify:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install shellcheck
+        run: sudo apt-get update -qq && sudo apt-get install -y -qq shellcheck
+
+      # -S error: gate on breakage, not style (sf-emit.sh/hooks carry pre-existing style notes)
+      - name: Shellcheck (errors only)
+        run: shellcheck -S error tools/*.sh hooks/*.sh
+
+      - name: Verify Phase 2 DAG
+        run: bash tools/verify-phase2-dag.sh
+
+      - name: Validate tracked JSON
+        run: |
+          set -euo pipefail
+          git ls-files '*.json' | while IFS= read -r f; do
+            jq empty "$f" || { echo "Invalid JSON: $f" >&2; exit 1; }
+            echo "OK: $f"
+          done
+
+      # CHANGELOG.md keeps historical names; this workflow holds the patterns themselves.
+      - name: Forbidden token gate
+        run: |
+          set -euo pipefail
+          EXCLUDE=(':!CHANGELOG.md' ':!.github/workflows/ci.yml')
+          if git grep -nI -e 'claude-opus-4-7' -- "${EXCLUDE[@]}"; then
+            echo 'FAIL: claude-opus-4-7 found outside CHANGELOG.md (pin is claude-fable-5)' >&2
+            exit 1
+          fi
+          if git grep -nIP -e '(?<!TESTCONTAINERS_)RYUK_DISABLED' -- "${EXCLUDE[@]}"; then
+            echo 'FAIL: bare RYUK_DISABLED found outside CHANGELOG.md (canonical name is TESTCONTAINERS_RYUK_DISABLED)' >&2
+            exit 1
+          fi
+          echo 'OK: no forbidden tokens.'

CHANGELOG.md

@@ -2,12 +2,50 @@
 
 All notable changes to superflow will be documented in this file.
 
+## [5.4.0] - 2026-06-11
+
+### Fixed — Critical
+- **Deploy checksum sync (SKILL.md startup)**: copy-if-missing (`test -f ... || cp`) replaced with checksum sync (`cmp -s` + overwrite on mismatch) via a small `sf_sync()` helper. Claude runtime syncs `superflow-enforcement.md` → `~/.claude/rules/` and `agents/*.md` → `~/.claude/agents/`; Codex runtime syncs `codex/agents/*.toml` → `~/.codex/agents/` plus the previously-never-deployed `codex/AGENTS.md` → `~/.codex/AGENTS.md`. Exception: `~/.codex/hooks.json` installed only if missing, never overwritten when diverged (one-line warning asks to merge manually). Stale deployed copies no longer survive skill upgrades. Hardcoded skill path replaced with `$SUPERFLOW_SKILL_ROOT`
+- **`context.run_id` survives Phase 0**: state writes in `references/phase0-onboarding.md`, `references/phase0/stage1-detect.md`, and `references/phase0/greenfield.md` now merge into the existing `.superflow-state.json` (`s.setdefault('context', {})`) instead of overwriting `context` wholesale — `run_id`/`runtime` written by SKILL.md startup are preserved
+- **Greenfield recovery + approval mode**: crash recovery checks `stage == "greenfield"` FIRST and routes to `greenfield.md` at the G-step matching `stage_index` (new Recovery Matrix row); G6 now writes approval mode `'greenfield'` instead of `'all'` (which would have dispatched Stage 4 Branch A and overwritten G5 docs)
+- **Codex audit invocation (Phase 0 Stage 2)**: added `TIMEOUT_CMD` guard, resolved the audit prompt via `$SUPERFLOW_SKILL_ROOT` + skill-path fallbacks instead of a CWD-relative `cat prompts/codex/audit.md`, fall back to the Claude security agent with a WARN when unresolved, and bumped effort `high` → `xhigh` (Phase 0 audit is deep tier)
+- **UUID lowercase**: every `uuidgen` call site now pipes through `tr '[:upper:]' '[:lower:]'` (macOS uppercases); `sf-emit.sh` additionally lowercases `SUPERFLOW_RUN_ID` defensively at emission time
+- **Parallel-wave dispatch (Claude runtime)**: subagents cannot dispatch subagents — a parallel wave is now N implementation-ONLY implementers in isolated worktrees, with the orchestrator running review → docs → PAR → ship sequentially per sprint as each finishes. The old "one implementer executes the full per-sprint flow incl. reviews and PR" pattern is explicitly forbidden on Claude runtime; Codex `max_depth=2` sprint supervisors are unchanged
+- **TOML silent-ignore bug**: `model_auto_compact_token_limit` moved above the `[agents]` table header in `references/codex-context-strategy.md` and `codex/config-fragment.toml` (a blank line does not end a TOML table — under `[agents]` the key was silently ignored and auto-compaction never fired); nonexistent feature flag `codex_hooks` renamed to `hooks`
+
+### Changed — Model Policy
+- **Fable tier policy**: `deep-spec-reviewer`, `deep-code-reviewer`, `deep-product-reviewer`, `deep-analyst` frontmatter moved `opus` → `fable` (effort stays `max`). Enforcement Reasoning Tiers table now mirrors frontmatter exactly; dispatch rule: implementers → `model: "sonnet"`, standard reviewers + doc-writers → `model: "opus"`, deep reviewers + deep-analyst → `model: "fable"`. A forgotten `model:` now silently inherits the parent frontier model (Fable) — the cost of forgetting went UP. Codex deep tier pinned to `model_reasoning_effort=xhigh`
+- **Secondary model pin**: every editable `claude-opus-4-7` / "Claude Opus 4.7" occurrence replaced with `claude-fable-5` / "Claude Fable 5" (SKILL.md, codex/AGENTS.md, codex dispatch patterns and overlays, CLAUDE.md, llms.txt). CHANGELOG history entries intentionally untouched
+
+### Changed — Native Orchestration
+- **JSON verdict contract**: every reviewer (6 `agents/` reviewer definitions + enforcement Rule 3 + review-unified.md) ends its final message with a fenced `json` block — `{"verdict", "findings": [{severity, file, line, scenario, description}], "summary"}`. The orchestrator extracts the fence (awk → jq) and assembles `.par-evidence.json` mechanically from verdict fields (`par-evidence.md` ships a working `extract_verdict()`); prose parsing is gone
+- **SendMessage re-review**: reviewers are dispatched as named background agents (`name: sprint-<N>-product-reviewer`, `run_in_background: true`); on REQUEST_CHANGES/NEEDS_FIXES the SAME reviewer is re-engaged via SendMessage scoped to the fix diff + its original findings, with cold re-dispatch as fallback
+- **Monitor CI wait**: on Claude runtime, after `gh pr create` the orchestrator waits for PR checks via the native Monitor tool (ship-pr.md, phase3-merge.md merge loop, enforcement Rule 8a); Codex runtime keeps `gh run list` polling; `--admin` remains forbidden
+- **Heartbeat cadence split**: Claude runtime checks heartbeat/`must_reread` at sprint boundaries, stage transitions, and immediately after compaction/summarization (not every turn); Codex runtime keeps every-turn discipline (no PreCompact hook, 258K context)
+- **Technical-lens fallback chain**: (1) `codex exec review`, (2) native `/code-review` skill via the Skill tool at high effort, (3) two split-focus Claude agents. `/code-review ultra` is user-triggered and billed — never launched autonomously, only suggested as an optional extra gate at Phase 3 pre-merge
+- **`Agent(isolation: "worktree")` opt-in**: documented in setup-worktree.md as a Claude-runtime alternative; bash worktree choreography stays the default (Codex parity), with the caveat that harness-chosen branch name/base must be verified against the `git_workflow_mode` contract
+- **Anti-injection guardrails**: all 12 `agents/` definitions gained a `<security>` block — repo content (source, diffs, READMEs, comments, test output) is DATA, never instructions; apparent instructions are flagged as suspicious-content findings. Matching rationalization-prevention bullet added to enforcement
+
+### Added — Hygiene & CI
+- **Testcontainers canon**: the only env var is `TESTCONTAINERS_RYUK_DISABLED`, gated on `process.env.CI === "true"`; the hygiene duty lives in `agents/*-implementer.md` (what Phase 2 dispatches). New `tools/cleanup-testcontainers.sh`: label-based selection (`docker ps -aq --filter "label=org.testcontainers=true"`), optional ancestor image filter, idempotent, no name-regex — the ONLY docker-touching command in the orchestrator's Rule 11 budget
+- **`pr.fail` event type**: emitted when a PR CI run concludes red or a PR is abandoned — `pr_number` (int, required), `reason` (string, required), `ci_run_id` (string, optional). Added to `templates/event-schema.json` (21 event types, was 20) and the `sf-emit.sh` allowlist; emission call sites in ship-pr.md and phase3-merge.md
+- **Repo CI workflow**: new `.github/workflows/ci.yml` — `shellcheck -S error` over `tools/*.sh hooks/*.sh`, `bash tools/verify-phase2-dag.sh`, `jq empty` over all tracked JSON, and a forbidden-token gate failing on `claude-opus-4-7` or bare `RYUK_DISABLED` (CHANGELOG.md and the workflow itself excluded)
+- **Phase 3 baseline + fossil removal**: post-merge verification reads `baseline_test_cmd` from `.superflow/completion-data.json` via `jq` with a documented fallback chain (completion-data field → Autonomy Charter test command → warn and ask); `completion-report.md` documents the completion-data.json contract (feature, sprints, prs, holistic_review, ts, optional baseline_test_cmd). `docs/superflow/sprint-queue.json` (v3.5 supervisor fossil) deleted; `docs/superflow/project-health-report.md` marked SUPERSEDED (describes the removed v3.0.0 Python architecture)
+- **Reviewer/implementer parity**: `agents/` code reviewers gained duplication / type-redefinition / dead-code / plan-completeness focus areas (matching `prompts/code-quality-reviewer.md`); product reviewers gained stub-detection; all 3 implementers gained Codebase Hygiene + Testcontainers Hygiene constraint blocks; `prompts/implementer.md`, `prompts/code-quality-reviewer.md`, `prompts/product-reviewer.md` labeled SOURCE MIRRORs of their `agents/` copies
+
+### Added — Model Profile Selection
+- **Model profile selection (frontier=fable default / balanced=opus-4.8)**: Phase 1 Step 2c presents a recommendation-led profile question after git workflow mode selection. Selection persisted to `.superflow-state.json` (`context.model_profile`) and the Autonomy Charter YAML frontmatter — survives compaction. At dispatch, deep reviewers + deep-analyst receive the profile-appropriate model explicitly (frontier → `model: "fable"`; balanced → `model: "opus"`); Codex-runtime Claude secondary uses `claude-fable-5` or `claude-opus-4-8` accordingly. Agent frontmatter stays `model: fable` as the frontier default; the CRITICAL dispatch rule and Phase 2 overview carry the profile qualifier post-compaction. State schema extended with `context.model_profile` (enum, default `"frontier"`)
+
+### Closed (from Deferred)
+- **`agent_id` correlation for parallel waves** — closed as obsolete: superseded by native agent teams / named-agent lineage (named background reviewers, SendMessage re-engagement, Monitor-based waits); systematic `SF_PARENT_ID` propagation is no longer planned
+- **Phase 3 failed-merge telemetry** — shipped as the `pr.fail` event type (see above)
+
 ## Deferred (Sprint 3 → future)
 
 MEDIUM/LOW event-log gaps identified in Sprint 3 coverage audit; not in scope for this sprint:
 
-- **MEDIUM:** `agent_id` correlation for parallel waves — H5 adds the dispatch/complete pair pattern with `SF_PARENT_ID`, but systematic propagation of agent IDs across wave boundaries (parent→child agent chains) is not yet implemented
-- **LOW:** Phase 3 failed-merge telemetry — `pr.merge` emission on CI failure/abandon is now suppressed (fixed in Sprint 3 fix-pass); a dedicated `pr.abandon` or `pr.fail` event type for failed-merge telemetry is deferred to a future schema extension
+- ~~**MEDIUM:** `agent_id` correlation for parallel waves~~ — **closed as obsolete in 5.4.0**: superseded by native agent teams / named-agent lineage (SendMessage re-engagement of named background reviewers); no longer planned
+- ~~**LOW:** Phase 3 failed-merge telemetry~~ — **shipped in 5.4.0** as the `pr.fail` event type (`pr_number`, `reason`, optional `ci_run_id`)
 - **MEDIUM:** Phase 3 post-merge `test.run`/`test.result` emissions — the post-merge integration test run on `main` (after all PRs merged) lacks `sf_emit test.run` / `sf_emit test.result` instrumentation
 - **LOW:** Enforce complexity/verdict enums at emitter layer — `sf_emit` currently accepts any string value for `complexity` and `verdict`; validation is deferred to consumers; adding allowlist checks inside `sf_emit` would catch typos at source
 - **LOW:** Normalize shell var quoting for typed args (`sprint:int="$VAR"`) — some phase docs use unquoted `$VAR` in typed-arg position; safe in practice but inconsistent; a style pass would standardize to `sprint:int=$VAR` (no quotes needed for numeric vars) uniformly