diff --git a/.agents/skills/project-management/SKILL.md b/.agents/skills/project-management/SKILL.md index 8feb522bd0c..9690b7d695a 100644 --- a/.agents/skills/project-management/SKILL.md +++ b/.agents/skills/project-management/SKILL.md @@ -42,12 +42,18 @@ Choose that posture when adding or creating the project: - `direct-PR` pushes and opens a PR without the no-mistakes pipeline. - `local-only` has no required remote or PR and lands only through the approved local fast-forward path. - `no-mistakes-prod-only` is a conditional policy rather than one flat mode: genuinely internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`. +- `+hardened` is the highest-rigor choice on this list, adding the quality gate that runs before validation; it rides alongside one of the flat modes above rather than replacing it, so a hardened project is registered as `[no-mistakes +hardened]`, `[direct-PR +hardened]`, or `[local-only +hardened]`. `no-mistakes-prod-only` is the default for a newly added or created remote-backed project when the captain specifies nothing, and a project with no remote defaults to `local-only`. State that resolved default while confirming the source, local name, and posture instead of asking the captain to choose from scratch, and record a flat mode instead whenever they ask for one. Existing registry entries keep the meaning they already have and are never migrated or reinterpreted, so a legacy entry with no bracket stays `no-mistakes`. Registering a conditional policy is a one-time choice and never requires classifying any change; the per-task surface classification happens at each task's intake, and internal-only is never inferred from file location or project name. +`+hardened` is off for every project unless the captain asks for it, so a project registered without it is `standard`. +Refuse `+hardened` together with `no-mistakes-prod-only` and tell the captain to pick a flat delivery mode instead. +A conditional policy decides per task, so a quality standard that covers only part of a project is a posture nobody can state in one sentence. +`AGENTS.md` section 7 owns how each task's quality resolves at intake, and `bin/fm-project-mode.sh --quality` owns how the registered token is read. + The optional `+yolo` posture changes routine approval authority but does not change the delivery mode. Default it off for every project and every posture, and enable it only on the captain's explicit instruction. `AGENTS.md` section 7 owns the complete authority boundary and exceptions when it is on. diff --git a/AGENTS.md b/AGENTS.md index 81dc761f8c2..17166ec38ce 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -286,6 +286,7 @@ Resolve every ship task's concrete delivery mode and yolo posture at intake, and A current explicit captain instruction wins; otherwise the project's registry entry is the captain's standing posture, and dropping below its rigor needs a reason you can state. On a `no-mistakes-prod-only` project, classify the task's surface: internal-only tooling, automation, contributor or operator process, and release or submission work ships `direct-PR`, while product-facing, mixed, and uncertain work ships `no-mistakes`; never infer internal-only from file location or project name. An unregistered project or absent registry resolves to `no-mistakes` with yolo off, and the registration gap goes to the captain. +A task's quality posture resolves at intake with the same precedence, a current explicit captain instruction first, then the project's registered posture, then `standard`, with the one-line reason for any deviation recorded in the same backlog note. Record the resulting mode, yolo, and the one-line reason for any deviation in the backlog item note. Treat file or subsystem overlap as a risk signal rather than an automatic reason to wait, and dispatch isolated work immediately with no concurrency cap when each change can be independently implemented and validated and the selected delivery path can reconcile ordinary rebases or conflicts. diff --git a/README.md b/README.md index 8ed5226b171..4446ed5a39f 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ Launching a supported harness inside it instantiates your first mate - and makes - **A visible crew** - every crewmate works in its own tmux window, experimental herdr/zellij tab, cmux workspace, or Orca terminal you can watch or type into; the first mate reconciles. - **Disposable worktrees** - each task runs in a clean [treehouse](https://github.com/kunchenguid/treehouse) git worktree, or an Orca-managed worktree when `backend=orca`, so parallel work on one repo never collides. - **Two task shapes** - ship tasks deliver authorized changes; scout tasks leave standalone investigation reports when the intake contract warrants separate research. -- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, or `local-only`, with an optional `+yolo` autonomy flag. +- **Explicit project modes** - each project ships via `no-mistakes`, `direct-PR`, `local-only`, or one of those plus `+hardened` for the highest-rigor quality gate, with an optional `+yolo` autonomy flag. - **Optional secondmates** - opt in to persistent second mates that run from isolated firstmate homes with their own `FM_HOME`, state, projects, and session lock, either locally or as a whole home on an SSH-reachable host, with guarded updates and recovery that never turns an unavailable remote route into a local replacement. - **Event-driven, zero-token supervision** - a bash watcher sleeps on the fleet and wakes the first mate only when something needs you; verified primary harnesses also get a turn-end backstop that blocks or follows up on a blind stop when work is under way and supervision is not live. - **Optional Relay** - opt in with one local `.env` pairing token so firstmate can answer your public mentions on X and Discord alike, act on normal reversible mention requests through the same lifecycle as chat requests, acknowledge spawned work, and post up to three public-safe completion follow-ups within seven days for genuine milestones and the final outcome without changing non-Relay behavior; a final reply promised in a thread becomes durable state that is reconciled from disk, so a restart or a compacted conversation cannot lose it; dry-run preview records would-be replies and dismissals locally before go-live. diff --git a/bin/fm-brief.sh b/bin/fm-brief.sh index 595557523e7..3941dd259a8 100755 --- a/bin/fm-brief.sh +++ b/bin/fm-brief.sh @@ -6,7 +6,7 @@ # description, acceptance criteria, and context, and may adjust other sections # when the task genuinely deviates (e.g. working an existing external PR instead # of shipping a new one). -# Usage: fm-brief.sh --mode [--herdr-lab] +# Usage: fm-brief.sh --mode [--quality ] [--herdr-lab] # fm-brief.sh --scout [--herdr-lab] # fm-brief.sh --dreamer [--herdr-lab] # fm-brief.sh --secondmate {...|--no-projects} @@ -48,9 +48,22 @@ # "Delivery contract: mode=" line. bin/fm-spawn.sh reads that line and refuses # to launch a ship task whose explicit --mode disagrees, so an adjusted brief and the # recorded task metadata cannot drift apart. +# --quality is the task's quality posture, resolved at intake the same way (AGENTS.md +# section 7) from the project's registered "+hardened" annotation, and it defaults to +# standard so every existing call site scaffolds exactly as before: +# standard the ordinary path: implement, then the mode's definition of done +# hardened a clean loop then a harden loop, both against the base commit fixed at +# spawn, both before validation, driven by bin/fm-quality.sh +# A hardened brief carries the sibling machine-readable line +# "Quality contract: quality=hardened" plus one short quality-gate section; a standard +# brief carries neither, so an absent line means standard and a standard brief stays +# byte-identical to what this scaffold produced before --quality existed. bin/fm-spawn.sh +# checks that line against its own --quality exactly as it checks the mode line. +# --quality is refused on scout, dreamer and secondmate scaffolds, for the same reason +# --mode is. # Ship briefs begin with a worktree-isolation assertion before the branch step. -# --mode is refused on scout and secondmate scaffolds: a scout's deliverable is a -# report rather than a merge, and a charter is not a delivery contract. +# --mode is refused on scout, dreamer and secondmate scaffolds: a scout or dreamer +# delivers a report rather than a merge, and a charter is not a delivery contract. # There is no --yolo flag here. The worker never owns approval decisions, so yolo is # a spawn-time and firstmate-side input only (AGENTS.md section 7). # Every scaffold's status protocol distinguishes the configured @@ -115,6 +128,8 @@ HERDR_LAB=0 NO_PROJECTS=0 MODE= MODE_SET=0 +QUALITY=standard +QUALITY_SET=0 POS=() want_value= for a in "$@"; do @@ -124,6 +139,7 @@ for a in "$@"; do esac case "$want_value" in mode) MODE=$a; MODE_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac want_value= @@ -137,6 +153,8 @@ for a in "$@"; do --no-projects) NO_PROJECTS=1 ;; --mode) want_value=mode ;; --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; # yolo never reaches the worker: it is firstmate's approval authority, not a # brief input. Refuse it loudly so it is never silently dropped here and then # believed to have been recorded. @@ -164,6 +182,19 @@ elif [ "$MODE_SET" -eq 1 ]; then echo "error: --mode applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 exit 1 fi + +# Quality posture. Unlike --mode it has a safe default, so it is optional and only +# its VALUE is closed-set validated; a typo must never quietly scaffold a standard +# brief for a task firstmate resolved as hardened. +if [ "$KIND" = ship ]; then + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac +elif [ "$QUALITY_SET" -eq 1 ]; then + echo "error: --quality applies only to ship briefs; a scout or dreamer delivers a report and a secondmate charter is not a delivery contract" >&2 + exit 1 +fi [ "${#POS[@]}" -ge 1 ] || { echo "error: task id is required" >&2; exit 1; } ID=${POS[0]} @@ -479,17 +510,40 @@ echo "scaffolded: $BRIEF (dreamer; replace {TASK})" exit 0 fi +# The DOD's machine-readable contract header, owned in one place so the three +# mode bodies below cannot drift apart. A standard task emits the delivery line +# alone, exactly as this scaffold did before --quality existed; a hardened task +# adds the sibling quality line that bin/fm-spawn.sh checks against its own +# explicit --quality before launching, the same way it checks the delivery line. +CONTRACT_LINES="Delivery contract: mode=$MODE" +if [ "$QUALITY" = hardened ]; then + CONTRACT_LINES="$CONTRACT_LINES +Quality contract: quality=hardened" +fi + +# The hardened task's extra instructions. Deliberately short: bin/fm-quality.sh +# and its --help own the loop's mechanics, and a second copy here would drift. +IFS= read -r -d '' QUALITY_SECTION <" line that bin/fm-spawn.sh checks against its own -# explicit --mode before launching. +# delivery mode, validated above. Each body opens with $CONTRACT_LINES, built once +# just above. case "$MODE" in direct-PR) SETUP2="" RULE1='1. Never push to the default branch (push only your `fm/'"$ID"'` branch). Never merge a PR.' IFS= read -r -d '' DOD <"\` to start, and \`no-mistakes axi respond\` for each gate. @@ -544,6 +598,14 @@ esac # briefs stay byte-identical to the historical Bash 5 output. DOD=${DOD%$'\n'} +# A standard task's brief body is unchanged by --quality existing: nothing is +# prepended, so it stays byte-identical to the pre-quality scaffold. +if [ "$QUALITY" = hardened ]; then + DOD="$QUALITY_SECTION + +$DOD" +fi + cat > "$BRIEF" < - (added ) -> no-mistakes off (legacy default) # - [] - (added ) -> off # - [ +yolo] - (added ) -> on +# - [ +yolo +hardened] - ... -> on, quality hardened +# +# Bracket grammar: the first token that does not begin with "+" is the mode, and +# every "+" token is position-independent. A "+" this version does not +# recognize is ignored rather than refused, so an older firstmate reading a newer +# registry keeps resolving the posture it does understand. # # Registered modes: # no-mistakes full pipeline -> PR -> configured merge authority (default) @@ -30,12 +36,34 @@ # AGENTS.md section 7 is the single owner of authority exceptions, including # ask-user contract expansion and stronger captain boundaries. # +# +hardened = the registered quality posture. From the captain's side this is the +# fourth option on the same list he picks from when he registers a project, after +# no-mistakes, direct-PR and local-only; mechanically it is a separate token, so a +# hardened project still carries one of those modes too. It is read with --quality +# rather than through the two-word line, which is unchanged. +# Absent means "standard": the ordinary path, with no extra quality loop. +# # --raw prints the registered annotation unmapped, so a caller that must tell a # conditional policy apart from a flat mode sees "no-mistakes-prod-only" itself. # +# --quality prints ONE word instead, "standard" or "hardened". It is a separate +# output path precisely so the two-word stdout contract above stays untouched. +# # An unknown/missing project or unknown mode falls back to "no-mistakes off" and warns # to stderr, so a typo never silently drops the gate. -# Usage: fm-project-mode.sh [--raw] +# The quality posture resolves independently of that fallback. +# A missing registry file, or a project absent from the registry, does yield "standard". +# An unrecognised mode token resets only the mode and the yolo flag and keeps a +# "+hardened" parsed beside it, because a typo in the mode must not silently drop the +# quality gate too; the unknown-mode warning still goes to stderr. +# "+hardened" beside "no-mistakes-prod-only" is the opposite case and drops to +# "standard" with its own stderr warning: a hardened project must pick a flat delivery +# mode, because a conditional policy decides per task and a quality standard covering +# only part of a project is not a statable posture +# (.agents/skills/project-management/SKILL.md "Delivery posture"). Unlike the typo, +# that combination parses cleanly and was ruled out on purpose. The mode still resolves +# to no-mistakes-prod-only, the two-word stdout is unchanged, and the exit stays 0. +# Usage: fm-project-mode.sh [--raw] [--quality] set -eu SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -44,50 +72,75 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" REG="$DATA/projects.md" RAW=0 -if [ "${1:-}" = "--raw" ]; then - RAW=1 - shift -fi -NAME=${1:?usage: fm-project-mode.sh [--raw] } +QUALITY_ONLY=0 +while [ "$#" -gt 0 ]; do + case "$1" in + --raw) RAW=1; shift ;; + --quality) QUALITY_ONLY=1; shift ;; + *) break ;; + esac +done +NAME=${1:?usage: fm-project-mode.sh [--raw] [--quality] } + +# One owner of the output shape, so the two-word default and the one-word +# --quality answer cannot drift apart across the fallback paths below. +emit() { # + if [ "$QUALITY_ONLY" -eq 1 ]; then + echo "$3" + else + echo "$1 $2" + fi +} if [ ! -f "$REG" ]; then echo "warn: no registry at $REG; defaulting $NAME to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -# awk emits " " (one line) or nothing if the project is absent. +# awk emits " " (one line) or nothing if the project is +# absent. A "+" token is never a mode, in any position, so the mode is the +# first bracket token that does not begin with "+". parsed=$(awk -v n="$NAME" ' $1=="-" && $2==n { - mode="no-mistakes"; yolo="off"; + mode="no-mistakes"; yolo="off"; quality="standard"; have_mode=0; if ($3 ~ /^\[/) { s=""; for (i=3; i<=NF; i++) { s = s (s==""?"":" ") $i; if ($i ~ /\]$/) break } gsub(/^\[|\]$/, "", s); # strip the surrounding brackets k = split(s, a, " "); - if (a[1] != "" && a[1] != "+yolo") mode = a[1]; - for (j=1; j<=k; j++) if (a[j]=="+yolo") yolo="on"; + for (j=1; j<=k; j++) { + if (a[j]=="+yolo") yolo="on"; + else if (a[j]=="+hardened") quality="hardened"; + else if (a[j] != "" && substr(a[j], 1, 1) != "+" && !have_mode) { mode=a[j]; have_mode=1 } + } } - print mode, yolo; exit + print mode, yolo, quality; exit } ' "$REG") if [ -z "$parsed" ]; then echo "warn: project \"$NAME\" not in registry; defaulting to no-mistakes off" >&2 - echo "no-mistakes off" + emit no-mistakes off standard exit 0 fi -mode=${parsed%% *} -yolo=${parsed##* } +read -r mode yolo quality <&2; mode=no-mistakes; yolo=off ;; esac case "$yolo" in on|off) ;; *) yolo=off ;; esac +case "$quality" in standard|hardened) ;; *) quality=standard ;; esac +if [ "$mode" = no-mistakes-prod-only ] && [ "$quality" = hardened ]; then + echo "warn: +hardened is refused alongside the conditional policy no-mistakes-prod-only for $NAME; a hardened project must pick a flat delivery mode (no-mistakes, direct-PR or local-only), so defaulting quality to standard" >&2 + quality=standard +fi # A conditional policy is not a task mode. Mechanical callers get its most # rigorous leg; --raw callers get the annotation itself (see the header). if [ "$RAW" -eq 0 ] && [ "$mode" = no-mistakes-prod-only ]; then mode=no-mistakes fi -echo "$mode $yolo" +emit "$mode" "$yolo" "$quality" diff --git a/bin/fm-promote.sh b/bin/fm-promote.sh index 0ed1fd06161..868feb6f543 100755 --- a/bin/fm-promote.sh +++ b/bin/fm-promote.sh @@ -9,8 +9,15 @@ # A scout records no delivery posture, so promotion is where this task's delivery # contract is decided: --mode and --yolo are REQUIRED and written into the meta # alongside the kind= flip. Firstmate resolves both at promotion time, having just -# read the scout's report (AGENTS.md section 7); data/projects.md holds the -# captain's standing posture as context, and this script never looks it up. +# read the scout's report (AGENTS.md section 7). data/projects.md holds the captain's +# standing posture, and this script reads only its quality half, only to print one +# advisory notice on stderr after a successful promotion; the delivery mode stays the +# caller's explicit decision. +# A promoted task deliberately records no quality= and no base_sha=. The base commit +# cannot be captured here, because the promoted worker resets to a clean +# default-branch base only afterwards, and a hardened record with no anchor would look +# complete to the quality loop while being unanchored. Both keys belong to the task +# that owns the base-capture question (bin/fm-quality.sh). # no-mistakes-prod-only is a registry policy rather than a task mode and is refused. # Usage: fm-promote.sh --mode --yolo set -eu @@ -120,6 +127,19 @@ TMP= fm_lock_release "$META_LOCK" META_LOCK_HELD=0 +# The quality sibling of the standing-posture notice in bin/fm-spawn.sh: advisory +# only, printed after the record is already rewritten so it can never affect the +# promotion. A record with no project=, a missing registry, or a failed lookup simply +# skips it, exactly as the spawn notice tolerates an empty standing mode. +PROMOTED_PROJECT=$(sed -n 's/^project=//p' "$META" | tail -n 1) +PROMOTED_PROJECT=${PROMOTED_PROJECT##*/} +if [ -n "$PROMOTED_PROJECT" ]; then + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROMOTED_PROJECT" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ]; then + echo "notice: $ID promotes carrying no quality posture while the standing posture for $PROMOTED_PROJECT is hardened - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi +fi + HOME_Q=$(printf '%q' "$FM_HOME") echo "promoted $ID to ship mode=$MODE yolo=$YOLO (teardown protection restored)" echo "next: FM_HOME=$HOME_Q bin/fm-send.sh fm-$ID ''" diff --git a/bin/fm-quality-receipt.sh b/bin/fm-quality-receipt.sh new file mode 100755 index 00000000000..9d7dce9be3c --- /dev/null +++ b/bin/fm-quality-receipt.sh @@ -0,0 +1,511 @@ +#!/usr/bin/env bash +# fm-quality-receipt.sh - validate a quality-gate receipt against the D2 schema. +# +# The committed schema at docs/quality-receipt.schema.json is the owner of the +# JSON shape. This script is the check. docs/quality-gate.md owns the rationale, +# the D1 bounds including budget_minutes, and the verify-envelope decision. +# +# Usage: +# fm-quality-receipt.sh validate [--check-head ] [|-] +# fm-quality-receipt.sh schema +# fm-quality-receipt.sh -h | --help +# +# validate reads one JSON document from or stdin. +# Exit 0 on a valid receipt, 1 on an invalid one, 2 on usage or tool errors. +# +# Post-schema rules, because JSON Schema cannot state them: +# - finding ids are unique inside each findings array +# - each verify child's base_sha and head_sha equal the envelope's +# --check-head then requires head_sha to resolve to that tree's HEAD. +# Tool errors (exit 2) are faults in the wiring rather than verdicts on the +# change: a receipt path that cannot be read, git failing to run under +# --check-head, and a schema using a keyword this checker does not implement. +# An unenforced schema keyword is refused, never ignored. That includes a +# keyword written beside a $ref, which this checker does not apply, and a $ref +# that does not resolve: both are refused when the schema is loaded. +# A receipt that is read but is not JSON is the phase command's fault, exit 1. +# The schema subset understood here is draft 2020-12 minus, among others, an +# array-valued "type" such as ["string", "integer"], which is unsupported. +# FM_QUALITY_RECEIPT_SCHEMA overrides the schema path (test seam). +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +SCHEMA="${FM_QUALITY_RECEIPT_SCHEMA:-$FM_ROOT/docs/quality-receipt.schema.json}" +SELF="$SCRIPT_DIR/fm-quality-receipt.sh" + +fm_quality_receipt_usage() { + sed -n '2,/^[^#]/{/^[^#]/d;s/^# \{0,1\}//;p;}' "$SELF" +} + +if ! command -v python3 >/dev/null 2>&1; then + printf 'fm-quality-receipt: python3 is required\n' >&2 + exit 2 +fi + +CMD="" +CHECK_HEAD="" +FILE="" +FILE_SET="" + +fm_quality_receipt_set_file() { + [ -z "$FILE_SET" ] || { + printf 'fm-quality-receipt: unexpected extra argument %s\n' "$1" >&2 + exit 2 + } + [ -n "$1" ] || { + printf 'fm-quality-receipt: the file operand is empty\n' >&2 + exit 2 + } + FILE=$1 + FILE_SET=1 +} + +while [ "$#" -gt 0 ]; do + case "$1" in + -h|--help) + fm_quality_receipt_usage + exit 0 + ;; + schema) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=schema + shift + ;; + validate) + [ -z "$CMD" ] || { + printf 'fm-quality-receipt: unexpected extra command %s\n' "$1" >&2 + exit 2 + } + CMD=validate + shift + ;; + --check-head) + [ "$#" -ge 2 ] && [ -n "$2" ] || { + printf 'fm-quality-receipt: --check-head requires a git dir\n' >&2 + exit 2 + } + CHECK_HEAD=$2 + shift 2 + ;; + --) + shift + break + ;; + -) + fm_quality_receipt_set_file - + shift + ;; + -*) + printf 'fm-quality-receipt: unknown option %s\n' "$1" >&2 + exit 2 + ;; + *) + fm_quality_receipt_set_file "$1" + shift + ;; + esac +done + +while [ "$#" -gt 0 ]; do + fm_quality_receipt_set_file "$1" + shift +done + +[ -z "$CMD" ] && { + fm_quality_receipt_usage >&2 + exit 2 +} + +if [ "$CMD" = schema ]; then + [ -z "$CHECK_HEAD" ] && [ -z "$FILE_SET" ] || { + printf 'fm-quality-receipt: schema takes no extra arguments\n' >&2 + exit 2 + } + [ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 + } + cat "$SCHEMA" + exit 0 +fi + +[ "$CMD" = validate ] || { + printf 'fm-quality-receipt: unknown command %s\n' "$CMD" >&2 + exit 2 +} + +[ -f "$SCHEMA" ] || { + printf 'fm-quality-receipt: schema file missing: %s\n' "$SCHEMA" >&2 + exit 2 +} + +[ -n "$FILE_SET" ] || FILE=- + +FM_QUALITY_RECEIPT_PROGRAM=$(cat <<'PY' +from __future__ import annotations + +import json +import math +import re +import subprocess +import sys + + +class SchemaError(Exception): + def __init__(self, path: str, message: str) -> None: + super().__init__(f"{path}: {message}") + self.path = path + self.message = message + + +class ToolError(Exception): + pass + + +ANNOTATION_KEYWORDS = frozenset( + { + "$schema", + "$id", + "$comment", + "$defs", + "title", + "description", + "default", + "examples", + "deprecated", + } +) +APPLIED_KEYWORDS = frozenset( + { + "$ref", + "allOf", + "if", + "then", + "else", + "not", + "type", + "const", + "enum", + "pattern", + "minLength", + "minimum", + "minItems", + "minProperties", + "items", + "properties", + "required", + "additionalProperties", + } +) +KNOWN_KEYWORDS = ANNOTATION_KEYWORDS | APPLIED_KEYWORDS +KNOWN_TYPES = frozenset({"object", "array", "string", "integer", "number"}) + + +def check_schema(schema: object, path: str, root: dict, seen: set[str]) -> None: + if isinstance(schema, bool): + return + if not isinstance(schema, dict): + raise SchemaError(path, "invalid schema") + for key in schema: + if key not in KNOWN_KEYWORDS: + raise SchemaError(path, f"unsupported schema keyword {key}") + defs = schema.get("$defs") + if "$defs" in schema and not isinstance(defs, dict): + raise SchemaError(f"{path}/$defs", "expected object") + if isinstance(defs, dict): + for name, sub in defs.items(): + check_schema(sub, f"{path}/$defs/{name}", root, seen) + if "$ref" in schema: + for key in schema: + if key != "$ref" and key not in ANNOTATION_KEYWORDS: + raise SchemaError(path, f"{key} beside $ref is not applied") + ref = schema["$ref"] + target = resolve(ref, root) + if ref not in seen: + seen.add(ref) + check_schema(target, ref, root, seen) + return + expected_type = schema.get("type") + if expected_type is not None and expected_type not in KNOWN_TYPES: + raise SchemaError(path, f"unsupported type {expected_type!r}") + for key in ("if", "then", "else", "not", "items", "additionalProperties"): + if key in schema: + check_schema(schema[key], f"{path}/{key}", root, seen) + if "allOf" in schema: + if not isinstance(schema["allOf"], list): + raise SchemaError(f"{path}/allOf", "expected array") + for i, sub in enumerate(schema["allOf"]): + check_schema(sub, f"{path}/allOf/{i}", root, seen) + props = schema.get("properties") + if "properties" in schema and not isinstance(props, dict): + raise SchemaError(f"{path}/properties", "expected object") + if isinstance(props, dict): + for name, sub in props.items(): + check_schema(sub, f"{path}/properties/{name}", root, seen) + + +def is_int(value: object) -> bool: + return isinstance(value, int) and not isinstance(value, bool) + + +def is_number(value: object) -> bool: + if is_int(value): + return True + return isinstance(value, float) and math.isfinite(value) + + +def reject_constant(token: str) -> float: + raise ValueError(f"{token} is not a JSON number") + + +def json_equal(left: object, right: object) -> bool: + if isinstance(left, bool) != isinstance(right, bool): + return False + if isinstance(left, dict) and isinstance(right, dict): + return len(left) == len(right) and all( + key in right and json_equal(value, right[key]) for key, value in left.items() + ) + if isinstance(left, list) and isinstance(right, list): + return len(left) == len(right) and all( + json_equal(a, b) for a, b in zip(left, right) + ) + if isinstance(left, (dict, list)) or isinstance(right, (dict, list)): + return False + return left == right + + +def unescape(part: str) -> str: + return part.replace("~1", "/").replace("~0", "~") + + +def resolve(ref: str, root: dict) -> dict: + if not ref.startswith("#/"): + raise SchemaError("$", f"unsupported $ref {ref}") + cur: object = root + for part in ref[2:].split("/"): + if not isinstance(cur, dict): + raise SchemaError("$", f"broken $ref {ref}") + key = unescape(part) + if key not in cur: + raise SchemaError("$", f"broken $ref {ref}") + cur = cur[key] + if not isinstance(cur, dict) and not isinstance(cur, bool): + raise SchemaError("$", f"broken $ref {ref}") + return cur # type: ignore[return-value] + + +def matches(instance: object, schema: object, root: dict) -> bool: + try: + validate(instance, schema, root, "$") + except SchemaError: + return False + return True + + +def validate(instance: object, schema: object, root: dict, path: str) -> None: + if schema is True: + return + if schema is False: + raise SchemaError(path, "not allowed") + if not isinstance(schema, dict): + raise SchemaError(path, "invalid schema") + if "$ref" in schema: + validate(instance, resolve(schema["$ref"], root), root, path) + return + if "allOf" in schema: + for sub in schema["allOf"]: + validate(instance, sub, root, path) + if "if" in schema: + if matches(instance, schema["if"], root): + if "then" in schema: + validate(instance, schema["then"], root, path) + elif "else" in schema: + validate(instance, schema["else"], root, path) + if "not" in schema: + if matches(instance, schema["not"], root): + raise SchemaError(path, "matched a forbidden schema") + expected_type = schema.get("type") + if expected_type == "object": + if not isinstance(instance, dict): + raise SchemaError(path, "expected object") + elif expected_type == "array": + if not isinstance(instance, list): + raise SchemaError(path, "expected array") + elif expected_type == "string": + if not isinstance(instance, str): + raise SchemaError(path, "expected string") + elif expected_type == "integer": + if not is_int(instance): + raise SchemaError(path, "expected integer") + elif expected_type == "number": + if not is_number(instance): + raise SchemaError(path, "expected number") + elif expected_type is not None: + raise SchemaError(path, f"unsupported type {expected_type}") + if "const" in schema and not json_equal(instance, schema["const"]): + raise SchemaError(path, f"expected {schema['const']!r}") + if "enum" in schema and not any(json_equal(instance, item) for item in schema["enum"]): + raise SchemaError(path, f"expected one of {schema['enum']!r}") + if "pattern" in schema: + if not isinstance(instance, str) or re.fullmatch(schema["pattern"], instance) is None: + raise SchemaError(path, f"expected to match {schema['pattern']}") + if "minLength" in schema: + if not isinstance(instance, str) or len(instance) < schema["minLength"]: + raise SchemaError(path, f"shorter than {schema['minLength']}") + if "minimum" in schema and is_number(instance) and instance < schema["minimum"]: + raise SchemaError(path, f"below minimum {schema['minimum']}") + if isinstance(instance, list): + if "minItems" in schema and len(instance) < schema["minItems"]: + raise SchemaError(path, f"fewer than {schema['minItems']} items") + item_schema = schema.get("items") + if item_schema is not None: + for i, item in enumerate(instance): + validate(item, item_schema, root, f"{path}/{i}") + if isinstance(instance, dict): + if "minProperties" in schema and len(instance) < schema["minProperties"]: + raise SchemaError(path, f"fewer than {schema['minProperties']} properties") + props = schema.get("properties", {}) + required = schema.get("required", []) + for key in required: + if key not in instance: + raise SchemaError(f"{path}/{key}", "required") + additional = schema.get("additionalProperties", True) + for key, value in instance.items(): + child = f"{path}/{key}" + if key in props: + validate(value, props[key], root, child) + elif additional is False: + raise SchemaError(child, "additional property") + elif additional is not True: + validate(value, additional, root, child) + + +def unique_finding_ids(node: object, path: str) -> None: + if isinstance(node, list): + for i, item in enumerate(node): + unique_finding_ids(item, f"{path}/{i}") + return + if not isinstance(node, dict): + return + findings = node.get("findings") + if isinstance(findings, list): + seen: dict[str, int] = {} + for i, item in enumerate(findings): + if not isinstance(item, dict): + continue + finding_id = item.get("id") + if not isinstance(finding_id, str): + continue + if finding_id in seen: + raise SchemaError( + f"{path}/findings/{i}/id", + f"duplicate id {finding_id!r} (also {path}/findings/{seen[finding_id]}/id)", + ) + seen[finding_id] = i + for key, value in node.items(): + unique_finding_ids(value, f"{path}/{key}") + + +def verify_child_shas(receipt: object) -> None: + if not isinstance(receipt, dict): + return + if receipt.get("phase") != "verify": + return + phases = receipt.get("phases") + if not isinstance(phases, list): + return + for i, child in enumerate(phases): + if not isinstance(child, dict): + continue + for field in ("base_sha", "head_sha"): + if child.get(field) != receipt.get(field): + raise SchemaError( + f"$/phases/{i}/{field}", + f"must equal envelope {field}", + ) + + +def git_rev_parse(git_dir: str, *args: str) -> str: + proc = subprocess.run( + ["git", "-C", git_dir, "rev-parse", *args], + check=False, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + if proc.returncode != 0: + detail = proc.stderr.strip() or "git rev-parse failed" + raise ToolError(detail) + return proc.stdout.strip() + + +def check_head(receipt: object, git_dir: str) -> None: + if not isinstance(receipt, dict): + raise SchemaError("$", "expected object") + head_sha = receipt.get("head_sha") + if not isinstance(head_sha, str): + raise SchemaError("$/head_sha", "required") + actual = git_rev_parse(git_dir, "HEAD") + try: + resolved = git_rev_parse(git_dir, "--verify", f"{head_sha}^{{commit}}") + except ToolError as exc: + raise SchemaError("$/head_sha", f"does not resolve in {git_dir}: {exc}") from exc + if resolved != actual: + raise SchemaError( + "$/head_sha", + f"{resolved} is not HEAD {actual}", + ) + + +def main() -> int: + schema_path, check_head_dir, source = sys.argv[1], sys.argv[2], sys.argv[3] + try: + with open(schema_path, encoding="utf-8") as handle: + schema = json.load(handle, parse_constant=reject_constant) + except (OSError, ValueError) as exc: + print(f"fm-quality-receipt: cannot read schema: {exc}", file=sys.stderr) + return 2 + try: + check_schema(schema, "$", schema, set()) + except SchemaError as exc: + print(f"fm-quality-receipt: schema is not enforceable: {exc}", file=sys.stderr) + return 2 + try: + if source == "-": + raw = sys.stdin.read() + else: + with open(source, encoding="utf-8") as handle: + raw = handle.read() + except OSError as exc: + print(f"fm-quality-receipt: cannot read receipt: {exc}", file=sys.stderr) + return 2 + try: + receipt = json.loads(raw, parse_constant=reject_constant) + except ValueError as exc: + print(f"fm-quality-receipt: receipt is not JSON: {exc}", file=sys.stderr) + return 1 + try: + validate(receipt, schema, schema, "$") + unique_finding_ids(receipt, "$") + verify_child_shas(receipt) + if check_head_dir: + check_head(receipt, check_head_dir) + except SchemaError as exc: + print(f"fm-quality-receipt: {exc}", file=sys.stderr) + return 1 + except ToolError as exc: + print(f"fm-quality-receipt: git failed: {exc}", file=sys.stderr) + return 2 + return 0 + + +if __name__ == "__main__": + sys.exit(main()) +PY +) + +exec python3 -c "$FM_QUALITY_RECEIPT_PROGRAM" "$SCHEMA" "$CHECK_HEAD" "$FILE" diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index 685c133c22f..f07250ad430 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -1,7 +1,7 @@ #!/usr/bin/env bash # Spawn a direct report: a crewmate in a treehouse or Orca worktree, or a # secondmate in its isolated firstmate home. -# Usage: fm-spawn.sh --mode --yolo [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] +# Usage: fm-spawn.sh --mode --yolo [--quality ] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh --scout [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] # fm-spawn.sh [] [--harness |harness|launch-command] [--model ] [--effort ] [--backend ] --secondmate # --mode and --yolo are this task's delivery contract, REQUIRED for every ship @@ -16,6 +16,15 @@ # loud one-line deviation notice is printed and the spawn continues. # no-mistakes-prod-only is a registry policy rather than a task mode and is # refused as a flag value. +# --quality is this task's quality posture, resolved at intake the same way from +# the project's registered "+hardened" annotation (bin/fm-project-mode.sh +# --quality). Unlike --mode it has a safe default, so it is optional on a ship +# spawn and defaults to standard, and it is refused on --scout and --secondmate +# spawns. A ship spawn reads the brief's "Quality contract: quality=" +# line and REFUSES a mismatch exactly as it does for the delivery line. An absent +# line reads as standard rather than as a legacy gap, so it agrees silently with +# --quality standard, while --quality hardened against a brief that never told +# the worker to run the loop is a refusal. # fm-spawn.sh --relaunch [--harness ] [--model ] [--effort ] # --relaunch launches a replacement agent for an EXISTING task into that # task's own recorded endpoint and worktree instead of creating either. It is @@ -191,6 +200,11 @@ # A ship task records the explicit mode/yolo it was passed; a secondmate spawn records # mode=secondmate, yolo=off, home=, and projects=; a scout records neither, and both the # success line and state/.meta omit them. +# A ship task additionally records quality= (the resolved posture) and base_sha= (the +# commit its worktree starts from). base_sha is captured ONCE, at the fresh spawn, and +# a relaunch reads it back rather than recapturing it: the hardened quality loop commits +# each round, so a base recaptured later - or a gate reading HEAD~1 - would narrow its +# view to the newest commits while still reporting success. # Every fresh spawn or relaunch records a new spawn_gen= incarnation token so durable # consumers can distinguish a replacement worker that reuses the same task id. # When the home session's frozen trace-context decision is enabled (see @@ -287,6 +301,8 @@ EFFORT= BACKEND_ARG= MODE= YOLO= +QUALITY= +BASE_SHA= TRACEPARENT_ARG= HARNESS_SET=0 MODEL_SET=0 @@ -294,6 +310,7 @@ EFFORT_SET=0 BACKEND_SET=0 MODE_SET=0 YOLO_SET=0 +QUALITY_SET=0 TRACEPARENT_SET=0 RELAUNCH=0 POS=() @@ -310,6 +327,7 @@ for a in "$@"; do backend) BACKEND_ARG=$a; BACKEND_SET=1 ;; mode) MODE=$a; MODE_SET=1 ;; yolo) YOLO=$a; YOLO_SET=1 ;; + quality) QUALITY=$a; QUALITY_SET=1 ;; traceparent) TRACEPARENT_ARG=$a; TRACEPARENT_SET=1 ;; *) echo "error: internal parser state for --$want_value" >&2; exit 1 ;; esac @@ -332,6 +350,8 @@ for a in "$@"; do --mode=*) MODE=${a#--mode=}; MODE_SET=1 ;; --yolo) want_value=yolo ;; --yolo=*) YOLO=${a#--yolo=}; YOLO_SET=1 ;; + --quality) want_value=quality ;; + --quality=*) QUALITY=${a#--quality=}; QUALITY_SET=1 ;; --traceparent) want_value=traceparent ;; --traceparent=*) TRACEPARENT_ARG=${a#--traceparent=}; TRACEPARENT_SET=1 ;; *) POS+=("$a") ;; @@ -344,6 +364,7 @@ done [ "$BACKEND_SET" -eq 0 ] || [ -n "$BACKEND_ARG" ] || { echo "error: --backend requires a non-empty value" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || [ -n "$MODE" ] || { echo "error: --mode requires a non-empty value" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || [ -n "$YOLO" ] || { echo "error: --yolo requires a non-empty value" >&2; exit 1; } +[ "$QUALITY_SET" -eq 0 ] || [ -n "$QUALITY" ] || { echo "error: --quality requires a non-empty value" >&2; exit 1; } [ "$TRACEPARENT_SET" -eq 0 ] || [ -n "$TRACEPARENT_ARG" ] || { echo "error: --traceparent requires a non-empty value" >&2; exit 1; } # A parent-delivered carrier replaces this home's own resolution, so it is # refused unless it is a secondmate spawn carrying a strictly valid W3C value. @@ -372,6 +393,7 @@ if [ "$RELAUNCH" -eq 1 ]; then [ "$KIND_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded kind; --scout/--secondmate cannot override it" >&2; exit 1; } [ "$MODE_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded delivery mode; --mode cannot override it" >&2; exit 1; } [ "$YOLO_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded yolo posture; --yolo cannot override it" >&2; exit 1; } + [ "$QUALITY_SET" -eq 0 ] || { echo "error: --relaunch reuses the task's recorded quality posture; --quality cannot override it" >&2; exit 1; } else # Delivery contract (AGENTS.md section 7). A ship task's mode and yolo are # firstmate's per-task decision, so they are required and closed-set validated @@ -397,6 +419,14 @@ else on|off) ;; *) echo "error: --yolo must be on or off (got '$YOLO')" >&2; exit 1 ;; esac + # Quality has a safe default, so it is optional; only its value is closed-set + # validated, because a typo must never quietly ship a task firstmate resolved + # as hardened down the standard path. + [ "$QUALITY_SET" -eq 1 ] || QUALITY=standard + case "$QUALITY" in + standard|hardened) ;; + *) echo "error: --quality must be one of standard, hardened (got '$QUALITY')" >&2; exit 1 ;; + esac else [ "$MODE_SET" -eq 0 ] || { echo "error: --mode applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 @@ -406,6 +436,10 @@ else echo "error: --yolo applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 exit 1 } + [ "$QUALITY_SET" -eq 0 ] || { + echo "error: --quality applies only to ship spawns; a scout delivers a report and a secondmate records its own fixed posture" >&2 + exit 1 + } fi fi @@ -770,6 +804,8 @@ spawn_abort_cleanup() { echo "kind=$KIND" [ -z "${MODE:-}" ] || echo "mode=$MODE" [ -z "${YOLO:-}" ] || echo "yolo=$YOLO" + [ -z "${QUALITY:-}" ] || echo "quality=$QUALITY" + [ -z "${BASE_SHA:-}" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=${TASK_TMP:-}" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" @@ -887,6 +923,7 @@ if [ "${#POS[@]}" -gt 0 ] && [ "${POS[0]}" != "$idpart" ] && case "$idpart" in * # spanning several modes is two invocations rather than a silent mixed dispatch. [ "$MODE_SET" -eq 0 ] || shared_args+=(--mode "$MODE") [ "$YOLO_SET" -eq 0 ] || shared_args+=(--yolo "$YOLO") + [ "$QUALITY_SET" -eq 0 ] || shared_args+=(--quality "$QUALITY") for pair in "${POS[@]}"; do case "$pair" in *=*) : ;; @@ -1031,6 +1068,15 @@ if [ "$RELAUNCH" -eq 1 ]; then [ -n "$KIND" ] || KIND=ship MODE=$(fm_meta_get "$RELAUNCH_META" mode) YOLO=$(fm_meta_get "$RELAUNCH_META" yolo) + # Read back, never recaptured: the loop's whole measurement is anchored on the + # base this task actually started from (see the header). A ship task recorded + # before quality existed carries no quality= line, and absent means standard - + # the same reading the brief check applies - so it is normalized here rather + # than left empty and refused against its own brief. Its base_sha stays absent + # rather than being invented from a HEAD the worker has already moved. + QUALITY=$(fm_meta_get "$RELAUNCH_META" quality) + [ "$KIND" != ship ] || [ -n "$QUALITY" ] || QUALITY=standard + BASE_SHA=$(fm_meta_get "$RELAUNCH_META" base_sha) RELAUNCH_WT=$(fm_meta_get "$RELAUNCH_META" worktree) [ -n "$RELAUNCH_WT" ] && [ -d "$RELAUNCH_WT" ] || { echo "error: task $ID's recorded worktree '${RELAUNCH_WT:-none}' is missing; refusing to relaunch without the local copy its work lives in" >&2 @@ -1726,6 +1772,17 @@ if [ "$KIND" = ship ]; then echo "error: delivery mismatch for $ID: the brief says mode=$BRIEF_MODE but this spawn passed --mode $MODE; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 exit 1 fi + # The same agreement check for the quality posture. A standard brief carries no + # quality line at all, so an absent line IS the standard posture rather than a + # legacy gap: --quality standard agrees with it silently, and --quality hardened + # against a brief that never gave the worker the quality-gate section is the + # drift this refuses. + BRIEF_QUALITY=$(sed -n 's/^Quality contract: quality=\([^ ]*\).*$/\1/p' "$BRIEF" | head -n 1) + [ -n "$BRIEF_QUALITY" ] || BRIEF_QUALITY=standard + if [ "$BRIEF_QUALITY" != "$QUALITY" ]; then + echo "error: quality mismatch for $ID: the brief says quality=$BRIEF_QUALITY but this spawn passed --quality $QUALITY; correct the flag or re-scaffold the brief so the worker's instructions and the task record agree" >&2 + exit 1 + fi # The registry holds the captain's standing posture, so dropping below it is # allowed (a current explicit captain instruction wins) but never silent. An # unregistered project resolves to the same no-mistakes standing default, which @@ -1736,6 +1793,13 @@ if [ "$KIND" = ship ]; then && [ "$(delivery_rigor_rank "$MODE")" -lt "$(delivery_rigor_rank "$STANDING_MODE")" ]; then echo "notice: $ID ships mode=$MODE while the standing posture for $PROJ_NAME is $STANDING_MODE - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 fi + # The same notice for the quality posture, which is a plain two-value token: there + # is no conditional policy to exclude, so a hardened standing posture shipped as a + # standard task is the only downgrade there is. Advisory only, like the mode notice. + STANDING_QUALITY=$("$FM_ROOT/bin/fm-project-mode.sh" --quality "$PROJ_NAME" 2>/dev/null) || STANDING_QUALITY= + if [ "$STANDING_QUALITY" = hardened ] && [ "$QUALITY" = standard ]; then + echo "notice: $ID ships quality=$QUALITY while the standing posture for $PROJ_NAME is $STANDING_QUALITY - less rigor than the captain's standing posture; proceed only on a current explicit captain instruction or an intake judgment you can state" >&2 + fi fi BRIEF_DIR_REAL=$(cd "$(dirname "$BRIEF")" && pwd -P) @@ -2812,6 +2876,16 @@ else fi fi +# The immutable anchor for a hardened task's quality loop, captured once, here, +# while the worktree still sits on the base it was reset to. Every later phase +# measures a diff against THIS commit; a relaunch reads it back from the record +# above rather than recapturing it, because by then the loop has committed rounds +# of its own and a fresh capture would narrow the gate while still reporting +# success. A worktree git cannot read leaves it absent rather than wrong. +if [ "$RELAUNCH" -eq 0 ] && [ "$KIND" = ship ]; then + BASE_SHA=$(git -C "$WT" rev-parse HEAD 2>/dev/null || true) +fi + META_WINDOW=$T [ "$BACKEND" = orca ] && META_WINDOW=$W SPAWN_GEN="s$(date +%s).${BASHPID:-$$}.$RANDOM" @@ -2826,7 +2900,7 @@ fi preserve_relaunch_meta() { awk -F= ' BEGIN { - split("window endpoint_task_id worktree project harness kind mode yolo tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") + split("window endpoint_task_id worktree project harness kind mode yolo quality base_sha tasktmp model effort busy_gen spawn_gen traceparent backend herdr_session herdr_workspace_id herdr_tab_id herdr_pane_id zellij_session zellij_tab_id zellij_pane_id orca_worktree_id terminal cmux_workspace_id cmux_surface_id home projects control_relaunch_tx", keys, " ") for (i in keys) owned[keys[i]] = 1 } !($1 in owned) @@ -2841,6 +2915,8 @@ preserve_relaunch_meta() { echo "kind=$KIND" [ -z "$MODE" ] || echo "mode=$MODE" [ -z "$YOLO" ] || echo "yolo=$YOLO" + [ -z "$QUALITY" ] || echo "quality=$QUALITY" + [ -z "$BASE_SHA" ] || echo "base_sha=$BASE_SHA" echo "tasktmp=$TASK_TMP" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" diff --git a/bin/fm-test-run.sh b/bin/fm-test-run.sh index d96f31a4e3b..288b61430ad 100755 --- a/bin/fm-test-run.sh +++ b/bin/fm-test-run.sh @@ -374,6 +374,7 @@ family_for_basename() { fm-kimi-harness.test.sh|fm-muse-harness.test.sh|fm-herdr-lab.test.sh|fm-lint.test.sh|\ fm-lint-workflows.test.sh|\ fm-operational-input.test.sh|fm-pi-primary-types.test.sh|\ + fm-quality-receipt.test.sh|\ fm-send-popup-settle.test.sh|fm-send-settle.test.sh|\ fm-subagent-pretool-check.test.sh|\ fm-supervision-instructions.test.sh|fm-task-delivery.test.sh|\ @@ -1203,6 +1204,7 @@ families_for_changed_path() { bin/fm-tmux-lib.sh|bin/fm-marker-lib.sh|bin/fm-operational-input.sh|bin/fm-tasks-axi-lib.sh|\ bin/fm-vendor-auth-probe.sh|\ bin/fm-primary-scope-lib.sh|bin/fm-project-mode.sh|bin/fm-promote.sh|\ + bin/fm-quality-receipt.sh|\ bin/fm-ff-lib.sh|bin/fm-gotmp*|bin/*pretool*) printf '%s\n' pure-contract-unit ;; @@ -1222,7 +1224,8 @@ families_for_changed_path() { printf '%s\n' pure-contract-unit ;; .github/*|.tasks.toml|AGENTS.md|CLAUDE.md|CONTRIBUTING.md|\ - docs/configuration.md|docs/supervision-protocols/*) + docs/configuration.md|docs/supervision-protocols/*|\ + docs/quality-gate.md|docs/quality-receipt.schema.json) printf '%s\n' pure-contract-unit ;; tests/lib.sh|tests/*-helpers.sh) diff --git a/docs/architecture.md b/docs/architecture.md index 520c881893f..445402e6047 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -249,9 +249,10 @@ The `data/secondmates.md` line contract is owned by the [`secondmate-provisionin `no-mistakes` tasks run the full validation pipeline, `direct-PR` tasks open PRs without that pipeline, and `local-only` tasks stay local until firstmate performs an approved fast-forward merge. Each task's mode and `yolo` posture are firstmate's decision at intake and are passed explicitly to `bin/fm-brief.sh`, `bin/fm-spawn.sh`, and `bin/fm-promote.sh`, which refuse a ship task that does not carry them. -A ship brief records its mode as a fixed machine-readable line and the spawn refuses to launch on a different one, so the worker's instructions and the recorded task delivery cannot diverge. -`data/projects.md` records each project's standing posture and optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy; a ship spawn that drops below the registered rigor prints a deviation notice and continues. -`bin/fm-project-mode.sh` remains the one registry parser for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. +A ship brief records its mode as a fixed machine-readable line, and a hardened ship brief records its quality posture as a sibling line; the spawn refuses to launch on a value that disagrees with either, so the worker's instructions and the recorded task contract cannot diverge. +`data/projects.md` records each project's standing posture, its optional `+hardened` quality posture, and its optional `+yolo` flag as the captain's default and as context for that decision, including the conditional `no-mistakes-prod-only` policy, which `+hardened` may not ride; a ship spawn or a promotion that drops below either registered posture prints a deviation notice and continues. +The project-owned quality-gate contract and the receipt a hardened run must emit are owned by [`quality-gate.md`](quality-gate.md). +`bin/fm-project-mode.sh` remains the one registry parser, both for the standing-posture notices above and for the mechanical consumers that have no task in hand: fleet sync's `local-only` skip and home seeding's refusal and no-mistakes initialization. When a selected delivery path calls for a diff, `bin/fm-review-diff.sh` refreshes the authoritative base and, when task meta records `pr=`, always fetches and compares against `refs/pull//head` by default (recorded `pr_head=` is only an offline fallback) before falling back to the local branch with a warning. Where a no-mistakes pipeline stores evidence in the repo, it publishes that PR-viewable validation evidence to an orphan evidence branch that shares no history with code branches, so it never enters the crew branch or the default branch. This repo uses that setting, and its own `.no-mistakes/` directory remains local state that stays gitignored and is rejected by CI if tracked; [`configuration.md`](configuration.md) owns the setting. diff --git a/docs/documentation-audiences.json b/docs/documentation-audiences.json index 9597279923e..66cd9b89817 100644 --- a/docs/documentation-audiences.json +++ b/docs/documentation-audiences.json @@ -288,6 +288,10 @@ "path": "docs/orca-backend.md", "audience": "operator-current" }, + { + "path": "docs/quality-gate.md", + "audience": "maintainer-architecture" + }, { "path": "docs/remote-secondmates.md", "audience": "operator-current" diff --git a/docs/quality-gate.md b/docs/quality-gate.md new file mode 100644 index 00000000000..8443e515e93 --- /dev/null +++ b/docs/quality-gate.md @@ -0,0 +1,159 @@ +# Quality gate contract and receipt + +The project-owned quality-gate file and the receipt its commands print. +This is the moved owner for those two contracts from the parked Stage 0 spec, revised after the Stage 0a pilot on quota-axi. + +This page describes a capability. +Firstmate itself is not a project the bar is applied to, and this repository does not ship a `.quality-gate.yaml`. + +`bin/fm-quality.sh`, the loop controller, is not in this revision. +It consumes the receipt schema defined here, so the schema had to land first. + +## D1. `.quality-gate.yaml` + +Lives at the root of a hardened project, committed. +The name is vendor-neutral on purpose. +Nothing in the file mentions firstmate, so the file and its CI job survive if firstmate is never used on that repo again. + +```yaml +# .quality-gate.yaml +version: 1 + +# The one command CI runs. Exit 0 or 1. Prints one D2 verify receipt on stdout. +verify: "make quality" + +# The ordinary test suite. The loop runs this after every round's edits +# and reverts the round if it goes red. +test: "pnpm test" + +# The two phases, for the pre-flight loop only. CI does not read these. +clean: + command: "pnpm run quality:complexity" + threshold: + crap_max: 15 + +harden: + command: "pnpm run quality:mutation" + threshold: + kill_rate_min: 0.80 + exclude: + - "src/generated/**" + +bounds: + max_iterations: 4 + no_progress_limit: 2 + budget_usd: 8 + budget_minutes: 20 +``` + +The two threshold numbers in that example are the original design values. +They are captain decisions, not part of this revision, and this page does not change them. + +Field rules: + +- `version` is required and is refused if unknown, so a future schema change fails loudly rather than being half-read. +- `verify` is required. Everything else is optional, and a missing phase means that phase reports `not-applicable`. +- Thresholds are per project. There is no universal default, and `fm-quality.sh` must not invent one. +- `bounds` has the defaults listed above, applied when the key is absent. +- Commands run through the platform shell from the repo root, the same convention no-mistakes uses for `commands.*`. + +### `bounds.budget_minutes` + +The parked spec gave `max_iterations`, `no_progress_limit`, and `budget_usd` only. +Stage 0a showed the cost that actually decides affordability is measurement wall clock, and it is spent before any harness call exists to enforce spend against. +A two-hour measurement costs nothing in tokens. + +`budget_minutes` is the missing bound. +The default of 20 is the Stage 0a-affordable window: four vitest-runner measurements on the diffs that were timed. +A project whose engine is slower must set a higher number rather than overrunning a bound that was never written down. +`budget_usd` stays, because the agent-turn spend is a different resource and is still unmeasured. + +## D2. The receipt + +Every phase command prints one JSON object matching [`quality-receipt.schema.json`](quality-receipt.schema.json). +`bin/fm-quality-receipt.sh` is the check, and `bin/fm-quality-receipt.sh schema` reprints that file. + +`schema_version` stays `1`. +This revises unpublished v1 in place. +No production receipts exist yet. +The Stage 0a pilot objects that passed the old checker are not valid against this revision, and that is intentional. + +What breaks, and why: + +- `head_sha` is now required. + Without it a drifted `base_sha` reports `not-applicable` and exits 0, which is the silent miss the design called easiest to get wrong and hardest to notice. + A docs-only change still has a distinct head and base. + A receipt that cannot show both cannot tell those apart from an anchor that drifted onto `HEAD`. +- `survivors[]` is now `findings[]`. + A complexity offender is not a survivor, has no mutant, and is not `killable`. + The required classification enum was forcing a lie. +- Each finding has a stable `id`. + `file` plus `line` collide. + The pilot saw two distinct surviving mutants on one line, and three on another. + "No progress" has to mean the same findings, not the same count. +- `detail` replaces `mutant`, because the field is not mutation-specific. +- `engine` (name and version) and `threshold` (the numbers this outcome was judged against) are required on `clean` and `harden`. + The same code scored 2 to 37 points apart on two runners. + A receipt with no engine identity is not comparable to any other receipt. +- `duration_ms` is a required top-level integer. + Wall clock is the resource the bound is for, so it is not an optional key inside `metrics`. +- `phase: "verify"` is no longer a flat object with mixed findings. + +`outcome` is still the one field a caller reads to decide anything. +`metrics` is still an open map of numbers, because a Python project and a TypeScript project will not report the same keys. + +### Classification + +`clean` findings use `over-threshold`. +`harden` findings use `killable`, `equivalent`, `unreachable`, `unsupported`, or `defect`. +A clean finding classified `killable` is invalid, which is the point of splitting the vocabulary. + +Harden `id` values come from the engine's own stable mutant id when it has one. +Clean `id` values are a per-function identity the phase command controls, typically `file:line:name`. +Ids are unique inside one `findings` array. +They are not unique across the two phases of a verify receipt, because the two engines do not share an id space. + +### Verify emits one object, with `phases[]` + +`verify` prints one envelope object, not two objects, and not a flattened mix. + +One command is still the CI contract. +It exits 0 or 1, prints one JSON document on stdout, and exposes one `outcome` to branch on. +Two raw objects would preserve per-phase mapping and lose that single outcome. +One flat object with mixed `findings[]` preserves the single outcome and loses the mapping. +That is what the pilot had to do, prefixing metric keys by hand and filing complexity rows next to mutants under the same `killable` label. + +The envelope therefore carries `phase: "verify"`, the folded `outcome`, the same `base_sha` and `head_sha`, the wall-clock `duration_ms` of the wrapper, and a `phases[]` array of complete `clean` and `harden` receipts. +It does not carry `findings`, `engine`, or `threshold` of its own: those belong on the child that produced them. + +Phase commands still emit one `clean` or `harden` object each. +Only the verify wrapper builds the envelope. +Each child's `base_sha` and `head_sha` must equal the envelope's, so a wrapper cannot glue receipts from two trees. + +The folded `outcome` is a D4 rule, not a schema constraint. +`blocked` outranks `defect-found`, which outranks `exhausted` and `stuck`, which outrank `pass`. +`not-applicable` is the envelope outcome only when every child is `not-applicable`. + +## Checking a receipt + +```sh +bin/fm-quality-receipt.sh validate +bin/fm-quality-receipt.sh validate --check-head +bin/fm-quality-receipt.sh schema +``` + +Exit 0 is a valid receipt, 1 an invalid one, 2 a usage or tool error. +The split matters to the loop controller: a receipt that is read but is not JSON, or does not fit the schema, is the phase command's fault. +A receipt path that cannot be read at all, git failing to run, or a schema keyword the checker cannot enforce, is a wiring fault and must not be read as a verdict on the change. + +The checker implements a subset of JSON Schema draft 2020-12, and `bin/fm-quality-receipt.sh` is the owner of which keywords that subset covers. +An array-valued `type`, such as `"type": ["string", "integer"]`, is not supported: the committed schema never uses that form, and adding one crashes the checker with exit 1 instead of refusing it cleanly at exit 2. + +The receipt may arrive as a file operand, as `-`, or on stdin with no operand, so a phase command that prints its receipt on stdout can be piped straight in. + +`--check-head` resolves `head_sha` in that tree and requires it to be that tree's `HEAD`. +A receipt that stuffed a constant in `head_sha` fails as soon as `HEAD` moves. +The Stage 0a fail-open, a `not-applicable` object with no `head_sha` at all, fails even without that flag. + +`FM_QUALITY_RECEIPT_SCHEMA` may point the validator at a different schema file. +That seam exists so tests can prove the committed schema is the owner rather than a list of constants inside the script. diff --git a/docs/quality-receipt.schema.json b/docs/quality-receipt.schema.json new file mode 100644 index 00000000000..70c0129bd13 --- /dev/null +++ b/docs/quality-receipt.schema.json @@ -0,0 +1,210 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "title": "firstmate quality receipt", + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden", "verify"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "verify" } }, + "required": ["phase"] + }, + "then": { + "required": ["phases"], + "properties": { + "phases": { + "type": "array", + "minItems": 1, + "items": { "$ref": "#/$defs/phase_result" } + }, + "findings": false, + "engine": false, + "threshold": false + } + }, + "else": { + "$ref": "#/$defs/phase_result" + } + } + ], + "$defs": { + "sha": { + "type": "string", + "pattern": "^[0-9a-f]{7,40}$" + }, + "outcome": { + "enum": [ + "pass", + "blocked", + "not-applicable", + "exhausted", + "stuck", + "defect-found" + ] + }, + "engine": { + "type": "object", + "required": ["name", "version"], + "additionalProperties": false, + "properties": { + "name": { "type": "string", "minLength": 1 }, + "version": { "type": "string", "minLength": 1 } + } + }, + "threshold": { + "type": "object", + "minProperties": 1, + "additionalProperties": { "type": "number" } + }, + "metrics": { + "type": "object", + "additionalProperties": { "type": "number" } + }, + "finding": { + "type": "object", + "required": ["id", "file", "classification"], + "additionalProperties": false, + "properties": { + "id": { "type": "string", "minLength": 1 }, + "file": { "type": "string", "minLength": 1 }, + "line": { "type": "integer", "minimum": 1 }, + "detail": { "type": "string" }, + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect", + "over-threshold" + ] + }, + "note": { "type": "string" } + } + }, + "phase_result": { + "type": "object", + "required": [ + "schema_version", + "phase", + "outcome", + "base_sha", + "head_sha", + "duration_ms", + "engine", + "threshold", + "findings" + ], + "additionalProperties": false, + "properties": { + "schema_version": { "const": 1 }, + "phase": { "enum": ["clean", "harden"] }, + "outcome": { "$ref": "#/$defs/outcome" }, + "base_sha": { "$ref": "#/$defs/sha" }, + "head_sha": { "$ref": "#/$defs/sha" }, + "duration_ms": { "type": "integer", "minimum": 0 }, + "engine": { "$ref": "#/$defs/engine" }, + "threshold": { "$ref": "#/$defs/threshold" }, + "metrics": { "$ref": "#/$defs/metrics" }, + "findings": { + "type": "array", + "items": { "$ref": "#/$defs/finding" } + }, + "exclusions": { + "type": "array", + "items": { "type": "string" } + }, + "notes": { "type": "string" } + }, + "allOf": [ + { + "if": { + "properties": { "phase": { "const": "clean" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { "const": "over-threshold" } + } + } + ] + } + } + } + } + }, + { + "if": { + "properties": { "phase": { "const": "harden" } }, + "required": ["phase"] + }, + "then": { + "properties": { + "findings": { + "type": "array", + "items": { + "allOf": [ + { "$ref": "#/$defs/finding" }, + { + "properties": { + "classification": { + "enum": [ + "killable", + "equivalent", + "unreachable", + "unsupported", + "defect" + ] + } + } + } + ] + } + } + } + } + } + ] + } + } +} diff --git a/docs/scripts.md b/docs/scripts.md index 11e06955f26..8cf4fde5e63 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -62,7 +62,8 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize | `backends/orca.sh` | Experimental Orca backend adapter owning both worktree and terminal | | `backends/cmux.sh` | Experimental cmux session-provider adapter | | `fm-config-push.sh` | Push declared inherited local material to live local or remote secondmates and send the placement-specific config reread when changed | -| `fm-project-mode.sh` | Resolve a project's registered delivery posture from `data/projects.md` for fleet sync and home seeding | +| `fm-project-mode.sh` | Resolve a project's registered delivery and quality postures from `data/projects.md` | +| `fm-quality-receipt.sh` | Validate a quality-gate receipt against the D2 schema, or print that schema | | `fm-merge-local.sh` | Fast-forward a `local-only` project or Firstmate's own repository local default branch after approval | | `fm-review-diff.sh` | Review a crewmate branch or resolved PR head against the authoritative base | | `fm-marker-lib.sh` | Compatibility entry point for the from-firstmate carrier owned by `fm-operational-input.sh` | diff --git a/tests/fm-brief.test.sh b/tests/fm-brief.test.sh index 97368085ce0..9f69d5c65b6 100755 --- a/tests/fm-brief.test.sh +++ b/tests/fm-brief.test.sh @@ -961,6 +961,97 @@ test_task_id_reuse_refused_and_preserves_retained_report() { pass "fm-brief: task id reuse is refused, reports directory contents, gives a way forward, and preserves retained artifacts" } +# --- quality posture -------------------------------------------------------- + +# The load-bearing case for the quality wiring: a ship brief scaffolded WITHOUT +# --quality must be the brief this scaffold produced before --quality existed. +# Proven by executing the real scaffold twice - once with no flag, once with the +# explicit default - and comparing the generated files byte for byte, plus the +# two negative assertions that say what "unchanged" means here: no contract line +# and no quality-gate section reach a standard worker. +test_standard_quality_leaves_the_ship_brief_untouched() { + local home brief_default brief_explicit mode n=0 + home="$TMP_ROOT/quality-standard-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qstd-d$n" some-proj --mode "$mode" >/dev/null 2>&1 \ + || fail "$mode: a ship brief with no --quality should scaffold" + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qexp-d$n" some-proj --mode "$mode" --quality standard >/dev/null 2>&1 \ + || fail "$mode: an explicit --quality standard brief should scaffold" + brief_default="$home/data/brief-qstd-d$n/brief.md" + brief_explicit="$home/data/brief-qexp-d$n/brief.md" + # The task id is the only text that legitimately differs between the two. + sed "s/brief-qexp-d$n/brief-qstd-d$n/g" "$brief_explicit" > "$home/normalized-d$n" + cmp -s "$brief_default" "$home/normalized-d$n" \ + || fail "$mode: --quality standard changed the generated brief (diff: $(diff "$brief_default" "$home/normalized-d$n" | head -5))" + assert_no_grep "Quality contract:" "$brief_default" \ + "$mode: a standard brief recorded a quality contract line" + assert_no_grep "# Quality gate" "$brief_default" \ + "$mode: a standard brief carried the hardened quality-gate section" + grep -qx "Delivery contract: mode=$mode" "$brief_default" \ + || fail "$mode: the delivery contract line did not survive the quality wiring" + done + pass "fm-brief.sh: a standard ship brief is byte-identical with and without --quality, and carries no quality text" +} + +# A hardened brief must tell the worker the four things the loop depends on, and +# must record the machine-readable sibling line bin/fm-spawn.sh checks. Each fact +# is asserted on the generated file, not on the scaffold's source. +test_hardened_brief_records_the_contract_and_the_gate() { + local home brief mode n=0 + home="$TMP_ROOT/quality-hardened-home" + mkdir -p "$home/data" + for mode in no-mistakes direct-PR local-only; do + n=$((n + 1)) + FM_HOME="$home" "$ROOT/bin/fm-brief.sh" "brief-qhard-e$n" some-proj --mode "$mode" --quality hardened >/dev/null 2>&1 \ + || fail "$mode: a hardened ship brief should scaffold" + brief="$home/data/brief-qhard-e$n/brief.md" + grep -qx "Delivery contract: mode=$mode" "$brief" \ + || fail "$mode: the hardened brief lost its delivery contract line" + grep -qx "Quality contract: quality=hardened" "$brief" \ + || fail "$mode: the hardened brief did not record its machine-readable quality contract line" + assert_grep "# Quality gate" "$brief" "$mode: the hardened brief carried no quality-gate section" + assert_grep 'base_sha=' "$brief" "$mode: the hardened brief did not name the fixed base commit" + # shellcheck disable=SC2016 # A literal backticked phrase from the brief, matched fixed-string. + assert_grep 'never against `HEAD~1`' "$brief" "$mode: the hardened brief did not warn off HEAD~1" + assert_grep 'clean loop first, then the harden loop' "$brief" "$mode: the hardened brief did not order the two loops" + assert_grep 'before you start on that definition of done' "$brief" "$mode: the hardened brief did not put the loops before the definition of done" + assert_grep 'fm-quality.sh' "$brief" "$mode: the hardened brief did not name the script that drives the loop" + assert_grep 'Do not hand-roll either loop' "$brief" "$mode: the hardened brief did not forbid hand-rolling the loop" + assert_grep 'real product defect' "$brief" "$mode: the hardened brief did not say to report a defect rather than test around it" + assert_no_grep "EOF" "$brief" "$mode: the hardened brief leaked a heredoc EOF marker" + assert_grep "{TASK}" "$brief" "$mode: the hardened brief lost the {TASK} placeholder" + done + pass "fm-brief.sh: a hardened ship brief records the quality contract line and the short quality-gate section" +} + +# --quality has a safe default, so it is optional - but a typo must stop the +# scaffold rather than quietly producing a standard brief for a task firstmate +# resolved as hardened, and a scout, dreamer, or charter must refuse it outright +# rather than accepting and discarding it. +test_quality_is_closed_set_and_refused_where_it_does_not_apply() { + local home out status label args expect + home="$TMP_ROOT/quality-refused-home" + mkdir -p "$home/data" + while IFS='|' read -r label args expect; do + [ -n "$label" ] || continue + # shellcheck disable=SC2086 # args is an intentional word-split arg list + out=$(FM_HOME="$home" "$ROOT/bin/fm-brief.sh" $args 2>&1) + status=$? + [ "$status" -ne 0 ] || fail "$label: expected a non-zero exit" + assert_contains "$out" "$expect" "$label: refusal did not explain why" + done <<'ROWS' +unknown quality value|brief-qref-f1 some-proj --mode no-mistakes --quality nope|--quality must be one of standard, hardened +empty quality value|brief-qref-f2 some-proj --mode no-mistakes --quality|requires a value +quality on a scout brief|brief-qref-f3 some-proj --scout --quality hardened|--quality applies only to ship briefs +quality on a dreamer brief|brief-qref-f4 some-proj --dreamer --quality hardened|--quality applies only to ship briefs +quality on a secondmate charter|brief-qref-f5 --secondmate --no-projects --quality hardened|--quality applies only to ship briefs +ROWS + assert_absent "$home/data/brief-qref-f1/brief.md" "a refused quality value still wrote a brief" + pass "fm-brief.sh: --quality is closed-set validated and refused on scout, dreamer, and charter scaffolds" +} + test_script_parses test_no_heredoc_in_command_substitution test_help_includes_entire_header @@ -985,3 +1076,6 @@ test_status_protocol_shows_documented_decision_key_placement test_scout_and_secondmate_load_decision_hold_policy test_scout_and_secondmate_scaffold test_task_id_reuse_refused_and_preserves_retained_report +test_standard_quality_leaves_the_ship_brief_untouched +test_hardened_brief_records_the_contract_and_the_gate +test_quality_is_closed_set_and_refused_where_it_does_not_apply diff --git a/tests/fm-control-relaunch.test.sh b/tests/fm-control-relaunch.test.sh index 9a7b4285bab..6d2f0737377 100755 --- a/tests/fm-control-relaunch.test.sh +++ b/tests/fm-control-relaunch.test.sh @@ -297,6 +297,54 @@ test_relaunch_preserves_durable_task_metadata() { pass "fm-control relaunch: durable task metadata survives replacement launch publication" } +# The quality posture and its base commit are the two records a hardened task's +# quality loop reads back, so a replacement agent must inherit both unchanged. +# base_sha is the one that is easy to get wrong and hard to notice: each loop +# round commits, so a base recaptured at relaunch would quietly move forward and +# narrow every later measurement to the newest work while still reporting success. +# This drives the worktree's HEAD past the recorded base before relaunching, so a +# recapture would be visible rather than coincidentally equal. +test_relaunch_reuses_the_quality_posture_and_base_commit() { + local dir out rc base moved + dir=$(new_case quality-anchor rl40) + add_ship_task "$dir" rl40 claude + base=$(git -C "$dir/wt" rev-parse HEAD) + { + printf 'quality=hardened\n' + printf 'base_sha=%s\n' "$base" + } >> "$dir/home/state/rl40.meta" + # A hardened task's instructions carry the contract lines the spawn re-checks + # on every launch, so the replacement worker cannot be handed instructions that + # disagree with the task's own record. + { + printf '\n# Definition of done\n' + printf 'Delivery contract: mode=no-mistakes\n' + printf 'Quality contract: quality=hardened\n' + } >> "$dir/home/data/rl40/brief.md" + + # A round of the loop lands on the branch, exactly as it would in real work. + printf 'a killed mutant\n' > "$dir/wt/round-1.txt" + git -C "$dir/wt" add round-1.txt + git -C "$dir/wt" -c user.email=t@example.com -c user.name=t commit --quiet -m "quality round 1" + moved=$(git -C "$dir/wt" rev-parse HEAD) + [ "$moved" != "$base" ] || fail "the fixture failed to move HEAD past the recorded base" + + out=$(run_control "$dir" rl40 relaunch --note "continuing the quality loop"); rc=$? + expect_code 0 "$rc" "a hardened task should relaunch"$'\n'"$out" + [ "$(meta_field "$dir" rl40 quality)" = hardened ] \ + || fail "the quality posture must survive relaunch, got '$(meta_field "$dir" rl40 quality)'" + [ "$(meta_field "$dir" rl40 base_sha)" = "$base" ] \ + || fail "base_sha must be read back, not recaptured: got '$(meta_field "$dir" rl40 base_sha)', expected $base" + [ "$(meta_field "$dir" rl40 base_sha)" != "$moved" ] \ + || fail "base_sha was recaptured at relaunch and now points at the loop's own newest commit" + [ "$(grep -c '^quality=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one quality= line in the task record" + [ "$(grep -c '^base_sha=' "$dir/home/state/rl40.meta")" = 1 ] \ + || fail "relaunch left more than one base_sha= line in the task record" + [ "$(meta_field "$dir" rl40 mode)" = no-mistakes ] || fail "the delivery mode must survive alongside it" + pass "fm-control relaunch: the quality posture survives and the base commit is read back, never recaptured" +} + test_relaunch_serializes_concurrent_durable_metadata_publication() { local dir control_pid link_pid rc i=0 traceparent prepare ready exported release dir=$(new_case metadata-race rl28) @@ -1284,6 +1332,9 @@ test_spawn_relaunch_refuses_contradicting_flags() { out=$(run_spawn "$dir" rl16 --relaunch --scout); rc=$? expect_code 1 "$rc" "--scout should be refused alongside --relaunch" assert_contains "$out" "recorded kind" "the refusal should name the recorded kind rule" + out=$(run_spawn "$dir" rl16 --relaunch --quality hardened); rc=$? + expect_code 1 "$rc" "--quality should be refused alongside --relaunch" + assert_contains "$out" "recorded quality posture" "the refusal should name the recorded quality rule" out=$(run_spawn "$dir" rl16 "$dir/proj" --relaunch); rc=$? expect_code 1 "$rc" "a project positional should be refused alongside --relaunch" assert_contains "$out" "takes the task id only" "the refusal should name the positional rule" @@ -1314,6 +1365,7 @@ test_spawn_relaunch_refuses_a_pane_outside_the_worktree() { test_same_harness_relaunch_keeps_identity_and_reuses_the_endpoint test_relaunch_preserves_durable_task_metadata +test_relaunch_reuses_the_quality_posture_and_base_commit test_relaunch_serializes_concurrent_durable_metadata_publication test_disabled_relaunch_clears_prior_trace_context test_relaunch_appends_the_progress_note_to_the_instructions diff --git a/tests/fm-quality-receipt.test.sh b/tests/fm-quality-receipt.test.sh new file mode 100755 index 00000000000..4c40746e841 --- /dev/null +++ b/tests/fm-quality-receipt.test.sh @@ -0,0 +1,969 @@ +#!/usr/bin/env bash +# Behavior tests for bin/fm-quality-receipt.sh and the D2 receipt schema. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +TMP_ROOT=$(fm_test_tmproot fm-quality-receipt) +RECEIPT="$ROOT/bin/fm-quality-receipt.sh" + +validate() { + "$RECEIPT" validate "$@" +} + +# Build a JSON receipt from a python literal on stdin. +write_receipt() { + python3 -c 'import json,sys; sys.stdout.write(json.dumps(eval(sys.stdin.read())))' >"$1" \ + || fail "could not write receipt $1" +} + +BASE=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +HEAD=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + +# A valid clean pass. Callers override fields by name. +clean_pass_py() { + cat <