From 3983c3ec22fb14b0eb375fd3601d61cbe4d805ab Mon Sep 17 00:00:00 2001 From: firstmate-crewmate Date: Wed, 16 Sep 2026 02:09:05 +0000 Subject: [PATCH 01/23] feat(helm): route sync by project board --- .agents/skills/project-management/SKILL.md | 8 + README.md | 2 +- bin/fm-bootstrap.sh | 18 +- bin/fm-helm-lib.sh | 109 +++- bin/fm-helm-poll.sh | 237 +++++--- bin/fm-helm-project-map.sh | 610 ++++++++++++++++++++ bin/fm-helm-reconcile.sh | 187 ++++++ bin/fm-helm-sync.sh | 446 ++++++++++---- docs/1.architecture/_dir.yml | 2 + docs/1.architecture/helm-project-routing.md | 49 ++ docs/2.api/_dir.yml | 2 + docs/2.api/guides/_dir.yml | 2 + docs/2.api/guides/helm-project-routing.md | 41 ++ docs/2.api/helm-project-map.md | 53 ++ docs/4.configuration.md | 34 ++ docs/architecture.md | 12 +- docs/configuration.md | 28 +- docs/examples/helm.json.example | 10 +- docs/scripts.md | 8 +- tests/fm-helm-poll.test.sh | 45 +- tests/fm-helm-project-map.test.sh | 340 +++++++++++ tests/fm-helm-sync.test.sh | 222 +++---- 22 files changed, 2135 insertions(+), 330 deletions(-) create mode 100755 bin/fm-helm-project-map.sh create mode 100755 bin/fm-helm-reconcile.sh create mode 100644 docs/1.architecture/_dir.yml create mode 100644 docs/1.architecture/helm-project-routing.md create mode 100644 docs/2.api/_dir.yml create mode 100644 docs/2.api/guides/_dir.yml create mode 100644 docs/2.api/guides/helm-project-routing.md create mode 100644 docs/2.api/helm-project-map.md create mode 100644 docs/4.configuration.md create mode 100755 tests/fm-helm-project-map.test.sh diff --git a/.agents/skills/project-management/SKILL.md b/.agents/skills/project-management/SKILL.md index 86e37422d17..4fd4153e544 100644 --- a/.agents/skills/project-management/SKILL.md +++ b/.agents/skills/project-management/SKILL.md @@ -59,6 +59,11 @@ Clone into `projects/` and add the registry entry only after the destinati A `no-mistakes` or `no-mistakes-prod-only` project must have an `origin` remote and must complete the initialization procedure below, because a conditional policy's product-facing work runs the pipeline while its internal-only work still takes the direct PR. A `direct-PR` project needs an `origin` remote but skips no-mistakes initialization. A `local-only` project may have no remote and skips no-mistakes initialization. +When Helm is enabled (`config/helm.json` exists), also offer the new project a +dedicated GitHub Project board or the shared Helm default; skipping that choice +is fine and keeps future cards on the default. If the captain chooses a +dedicated board, run `bin/fm-helm-project-map.sh link ` after the project +is registered. ## Create a project @@ -69,6 +74,9 @@ After remote creation succeeds, clone it locally, add the registry entry, and in For a purely `local-only` project, create a local Git repository under its unused `projects/` path, add the registry entry, and make no GitHub call. The captain's request to create that local project authorizes this local initialization, but it does not authorize an unmentioned remote repository. +When Helm is enabled (`config/helm.json` exists), offer the same dedicated-board +or shared-default choice for the new project. If a dedicated board is accepted, +run `bin/fm-helm-project-map.sh link ` after registration. ## Initialize diff --git a/README.md b/README.md index 96c090ac64c..787cb9711cb 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,7 @@ Launching a supported harness inside it instantiates your first mate - and makes - **Optional secondmates** - opt in to persistent second mates that run from isolated firstmate homes with their own `FM_HOME`, state, projects, and session lock, either locally or as a whole home on an SSH-reachable host, with guarded updates and recovery that never turns an unavailable remote route into a local replacement. - **Event-driven, zero-token supervision** - a bash watcher sleeps on the fleet and wakes the first mate only when something needs you; verified primary harnesses also get a turn-end backstop that blocks or follows up on a blind stop when work is under way and supervision is not live. - **Optional Relay** - opt in with one local `.env` pairing token so firstmate can answer your public mentions on X and Discord alike, act on normal reversible mention requests through the same lifecycle as chat requests, acknowledge spawned work, and post up to three public-safe completion follow-ups within seven days for genuine milestones and the final outcome without changing non-Relay behavior; a final reply promised in a thread becomes durable state that is reconciled from disk, so a restart or a compacted conversation cannot lose it; dry-run preview records would-be replies and dismissals locally before go-live. -- **Optional Helm board** - opt in with a home-local `config/helm.json` so a GitHub Project v2 board follows the backlog and can submit durable dispatch requests back to firstmate. +- **Optional Helm boards** - opt in with a home-local `config/helm.json` so a default GitHub Project v2 board follows the backlog; `bin/fm-helm-project-map.sh` can route registered projects to dedicated or intentionally shared boards and submit durable dispatch requests back to firstmate. - **Strict project boundary** - the first mate is read-only over your projects except for the narrow guarded and captain-approved operations authorized by [hard rule 1](AGENTS.md#1-identity-and-prime-directives), including fleet sync's guarded safe branch pruning; crewmates make every other project change behind the configured merge authority. - **Restart-proof** - all state lives on disk and in the active session backend (tmux by hard default, herdr or cmux when selected or auto-detected, zellij/orca when explicitly selected); kill the session anytime and the next one reconciles, including confirmed-dead secondmate agents, and carries on. diff --git a/bin/fm-bootstrap.sh b/bin/fm-bootstrap.sh index 0cfca47834c..6ae06312f97 100755 --- a/bin/fm-bootstrap.sh +++ b/bin/fm-bootstrap.sh @@ -1124,9 +1124,10 @@ EOF # surface as durable check wakes. The sync itself aggregates local secondmate # backlogs, so a secondmate needs no competing board writer or copied config. helm_watch_setup() { - local sync_shim board_shim sync_body board_body + local sync_shim board_shim reconcile_shim sync_body board_body reconcile_body sync_shim="$STATE/helm-sync.check.sh" board_shim="$STATE/helm-board.check.sh" + reconcile_shim="$STATE/helm-reconcile.check.sh" if [ ! -f "$CONFIG/helm.json" ]; then if [ -e "$sync_shim" ] || [ -L "$sync_shim" ] || [ -e "$STATE/helm-sync.check-trust" ] || [ -L "$STATE/helm-sync.check-trust" ]; then @@ -1139,6 +1140,11 @@ helm_watch_setup() { "$SCRIPT_DIR/fm-check-unregister.sh" helm-board >/dev/null 2>&1 \ || echo "HELM: could not retire Helm watcher board poll" fi + if [ -e "$reconcile_shim" ] || [ -L "$reconcile_shim" ] || [ -e "$STATE/helm-reconcile.check-trust" ] || [ -L "$STATE/helm-reconcile.check-trust" ]; then + FM_HOME="$FM_HOME" FM_ROOT_OVERRIDE="$FM_ROOT" FM_STATE_OVERRIDE="$STATE" \ + "$SCRIPT_DIR/fm-check-unregister.sh" helm-reconcile >/dev/null 2>&1 \ + || echo "HELM: could not retire Helm watcher reconciliation" + fi return 0 fi @@ -1152,17 +1158,27 @@ EOF #!/usr/bin/env bash exec "$FM_ROOT/bin/fm-helm-poll.sh" EOF +) + reconcile_body=$(cat </dev/null 2>&1 \ || echo "HELM: could not arm Helm watcher sync" FM_HOME="$FM_HOME" FM_ROOT_OVERRIDE="$FM_ROOT" FM_STATE_OVERRIDE="$STATE" \ "$SCRIPT_DIR/fm-check-register.sh" helm-board >/dev/null 2>&1 \ || echo "HELM: could not arm Helm watcher board poll" + FM_HOME="$FM_HOME" FM_ROOT_OVERRIDE="$FM_ROOT" FM_STATE_OVERRIDE="$STATE" \ + "$SCRIPT_DIR/fm-check-register.sh" helm-reconcile >/dev/null 2>&1 \ + || echo "HELM: could not arm Helm watcher reconciliation" } crew_dispatch_validate() { diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index df1e14eb20a..29f32cf7985 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -5,8 +5,9 @@ # bin/fm-helm-poll.sh (the cheap read-only board-change poll). It owns: # - local fleet home discovery from data/secondmates.md, # - the one data/backlog.md parser, run once per discovered home, -# - the combined debounce hash over every discovered home's backlog, -# - the one board signature the poll and the sync both fold, +# - the combined debounce hash over every discovered home's backlog and the +# project map, +# - the per-board signatures the poll and the sync both fold, # - the card renderer and the one-pass reconciliation planner the sync # executes (see "Plan format" below), # - small sha256 helpers. @@ -107,10 +108,10 @@ fm_helm_discover_homes() { done <"$reg" } -# fm_helm_combined_hash ... -# One digest over every backlog file, in the given order, with a boundary token -# between homes and an explicit marker for an absent file. Any change to any -# home's backlog changes this digest. +# fm_helm_combined_hash ... +# One digest over every supplied file, in the given order, with a boundary token +# between files and an explicit marker for an absent file. Any change to a +# home's backlog or the project map changes this digest. fm_helm_combined_hash() { local f { @@ -125,6 +126,44 @@ fm_helm_combined_hash() { } | fm_helm_sha256_stdin } +# fm_helm_project_names ... +# Print each registered project name once, in first-seen home order. The +# project registry remains the authority for Helm's Project field and board +# routing; an absent registry contributes no names. +fm_helm_project_names() { + local home + for home in "$@"; do + [ -f "$home/data/projects.md" ] && [ ! -L "$home/data/projects.md" ] || continue + awk '$1 == "-" && $2 != "" { print $2 }' "$home/data/projects.md" + done | awk '!seen[$0]++' +} + +# fm_helm_mapping_orphan_id - derive the stable captain-hold id +# used for a broken local-project or GitHub-board mapping. +fm_helm_mapping_orphan_id() { + printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9._-]/-/g; s/-\{2,\}/-/g; s/^-*//; s/-*$//' +} + +# fm_helm_raise_mapping_orphan +# Create or reuse the existing captain-hold primitive for one broken mapping. +# The caller records the returned id in the mapping document; this helper owns +# only the shared hold operation and never invents a second decision channel. +fm_helm_raise_mapping_orphan() { + local home=$1 project=$2 reason=$3 id hold_reason + id="helm-map-$(fm_helm_mapping_orphan_id "$project")" + [ "$id" != helm-map- ] || return 1 + # tasks-axi's captain-hold reason grammar rejects parentheses. Keep the + # report's meaning while making the shared hold call valid for both orphan + # shapes. + hold_reason=${reason//\(/} + hold_reason=${hold_reason//\)/} + FM_HOME="$home" FM_ROOT_OVERRIDE="${FM_ROOT_OVERRIDE:-$FM_HELM_LIB_DIR/..}" \ + "$FM_HELM_LIB_DIR/fm-captain-hold.sh" hold "$id" \ + --title "Helm routing for $project needs a decision" --reason "$hold_reason" >/dev/null \ + || return 1 + printf '%s\n' "$id" +} + # fm_helm_backlog_parse_program - print the jq -Rn program that turns a # data/backlog.md stream into a records array. One owner; both the aggregator's # per-home parse and any future caller use this exact program. @@ -280,8 +319,8 @@ JQ # fm_helm_desired_program - print the jq program that renders every record of # the fleet backlog union into its desired card: title, body, Status, Kind, -# Project, and Priority option names, plus the owning home path and any -# stderr note. Input: the union array. $report_ids: the task ids that have a +# Project, and Priority option names, plus the owning home path, routed board, +# and any stderr note. Input: the union array. $report_ids: the task ids that have a # data//report.md in the main home. Output: the same array with .desired # and .home_path added to each record. fm_helm_desired_program() { @@ -298,15 +337,27 @@ fm_helm_desired_program() { elif $kind == "decision" then "Waiting on you" elif .state == "in_flight" then "In flight" else "Queued" end; - def project_of: + def project_of($registered): (.repo // "") as $r - | if $r == "firetabs" or $r == "geojitsu/firetabs" then "firetabs" - elif $r == "BetterBlueToo" or $r == "geojitsu/BetterBlueToo" then "BetterBlueToo" - elif $r == "firstmate" or $r == "geojitsu/firstmate" then "firstmate" - elif $r == "nocout" or $r == "dc-noc/nocout" then "nocout" - elif $r == "cryptoseacurrents" or $r == "copium/cryptoseacurrents" then "cryptoseacurrents" + | ($r | if contains("/") then split("/") | .[-1] else . end) as $base + | if ($registered | index($r)) != null then $r + elif ($registered | index($base)) != null then $base elif $r == "other" then "other" else null end; + def board_of($routing; $default_owner; $default_number): + (.repo // "") as $r + | ($r | if contains("/") then split("/") | .[-1] else . end) as $base + | ($routing[0].projects // {}) as $projects + | ([$projects | to_entries[] | select(.key == $r or .key == $base)][0].value // null) as $entry + | if $entry != null + and (($entry.state // "active") == "active" or ($entry.state // "") == "migrating") + and (($entry.owner // "") | type) == "string" + and (($entry.owner // "") | length) > 0 + and (($entry.number // 0) | type) == "number" + and (($entry.number // 0) > 0) + then {owner: $entry.owner, number: $entry.number} + else {owner: $default_owner, number: $default_number} + end; def type_line($kind): if $kind == "ship" then "ship - produces a change and a PR" elif $kind == "investigation" then "investigation - produces knowledge, not code" @@ -329,11 +380,12 @@ fm_helm_desired_program() { + (if $pr == "" then "" else "- **PR:** " + $pr + "\n" end) + "\n## Notes\n\n" + ((.body_lines // []) | map(. + "\n") | join("")) - + "\n---\n_Source of truth: `data/backlog.md` in the owning firstmate home._"; + + "\n---\n_Source of truth: `data/backlog.md` in the owning local home._"; map( kind_of as $kind | ((.priority // "3") | priority_name) as $priority - | project_of as $project + | project_of($registered) as $project + | board_of($routing; $default_owner; $default_number) as $board | .id as $id | (if (.report_path // "") != "" then .report_path elif any($report_ids[]; . == $id) then "data/" + $id + "/report.md" @@ -348,6 +400,7 @@ fm_helm_desired_program() { project: ($project // "other"), priority_n: (.priority // "3"), priority: $priority, + board: $board, note: (if $project == null then "fm-helm-sync: unsupported repository " + (.repo // "") + " for " + $id + "; using other" else "" end) @@ -369,6 +422,10 @@ JQ # $dispatch_status the configured dispatch Status option name # $now the epoch second recorded in every cache row # $tsv_existed "true" when the identity cache existed before this run +# $board_owner owner of the board group being planned +# $board_number number of the board group being planned +# $default_owner owner used to normalize legacy cache rows +# $default_number number used to normalize legacy cache rows fm_helm_plan_program() { cat <<'JQ' ([0] | implode) as $nul @@ -393,6 +450,9 @@ fm_helm_plan_program() { def priority_digit: if . == "P0" then "0" elif . == "P1" then "1" elif . == "P2" then "2" elif . == "P3" then "3" elif . == "P4" then "4" else "" end; + def old_cache($o): + [$o.task, $o.item, $o.node, $o.type, $o.status, $o.priority, $o.title, $o.body, + $o.epoch, $o.owner, ($o.number | tostring)] | join("\t"); def entry($o): [ $o.phase, $o.action, ($o.task // ""), ($o.item // ""), ($o.cache // ""), ($o.draft // ""), ($o.title // ""), ($o.body // ""), @@ -409,7 +469,7 @@ fm_helm_plan_program() { | (reduce $items[] as $i ({}; .[($i | line1)] = true)) as $line1_set | ($items | map(select(.content.__typename == "DraftIssue" or .content.__typename == "Issue"))) as $cards_all | (reduce $cards_all[] as $c ({}; .[($c | line1)] += [$c])) as $by_line1 - | (rows($cards) | map({task: (.[0] // ""), item: (.[1] // ""), node: (.[2] // ""), type: (.[3] // ""), status: (.[4] // ""), priority: (.[5] // ""), title: (.[6] // ""), body: (.[7] // ""), fp: (.[4] // ""), v2: (length >= 9)})) as $old_rows + | (rows($cards) | map({task: (.[0] // ""), item: (.[1] // ""), node: (.[2] // ""), type: (.[3] // ""), status: (.[4] // ""), priority: (.[5] // ""), title: (.[6] // ""), body: (.[7] // ""), epoch: (.[8] // ""), owner: (.[9] // $default_owner), number: (.[10] // $default_number), fp: (.[4] // ""), v2: (length >= 9)})) as $old_rows | ($old_rows | first_by("task")) as $old_by_task | (rows($deleted) | map({task: (.[0] // ""), line: join("\t")}) | first_by("task")) as $deleted_by_task | (rows($markers) | map({task: (.[0] // ""), item: (.[1] // ""), option: (.[2] // ""), fp: (.[3] // "")})) as $marker_rows @@ -444,6 +504,8 @@ fm_helm_plan_program() { elif ($matches | length) == 0 then if $deleted_by_task[$r.id] != null then {phase: "record", action: "skip", task: $r.id, tombstone: $deleted_by_task[$r.id].line, note: $d.note} + elif $old != null and $old.item != "" and (($old.owner != $board_owner) or ($old.number != $board_number)) then + {phase: "record", action: "none", task: $r.id, cache: old_cache($old), note: $d.note} elif $old != null and $r.state != "done" and $old.item != "" and ($item_set[$old.item] | not) then {phase: "record", action: "skip", task: $r.id, note: $d.note} else @@ -452,7 +514,7 @@ fm_helm_plan_program() { {id: $kind_field, name: "Kind", value: $d.kind, option: $ko}, {id: $priority_field, name: "Priority", value: $d.priority, option: $pro} ] as $writes | {phase: "record", action: "create", task: $r.id, title: $d.title, body: $d.body, - cache: ($r.id + "\t\t\tdraft\t" + $so + "\t" + $pro + "\t" + ($d.title | @base64) + "\t" + ($d.body | @base64) + "\t" + $now), + cache: ($r.id + "\t\t\tdraft\t" + $so + "\t" + $pro + "\t" + ($d.title | @base64) + "\t" + ($d.body | @base64) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)), fields: $writes, marker: (if has_marker($r.id) then "remove" else "" end), home: $r.home_path, note: $d.note, fp: $fp, expected: ({title: $d.title, body: $d.body, fields: []} | tojson), @@ -461,7 +523,11 @@ fm_helm_plan_program() { end else $matches[0] as $card - | ($card.content.__typename == "Issue") as $is_issue + | if $old != null and $old.item == $card.id + and (($d.board.owner != $board_owner) or ($d.board.number != $board_number)) then + {phase: "record", action: "none", task: $r.id, item: $card.id, cache: old_cache($old), note: $d.note} + else + ($card.content.__typename == "Issue") as $is_issue | ($card.content.id // "") as $node | ($d.title | @base64) as $dt | ($d.body | @base64) as $db @@ -508,7 +574,7 @@ fm_helm_plan_program() { (if $conflict then $bs elif $rebuilt or $status_normal or $cs == $so then $so elif $waiting_status_changed then $cs else $bs end) + "\t" + (if $conflict then $bp elif $rebuilt then $pro else (if $priority_board_changed then $cp else $pro end) end) + "\t" + (if $conflict then $bt elif $rebuilt or ($ct == $dt and $cb == $db) or ($text_board_changed | not) then $dt else $bt end) + "\t" + - (if $conflict then $bb elif $rebuilt or ($ct == $dt and $cb == $db) or ($text_board_changed | not) then $db else $bb end) + "\t" + $now) as $cache + (if $conflict then $bb elif $rebuilt or ($ct == $dt and $cb == $db) or ($text_board_changed | not) then $db else $bb end) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)) as $cache | if $force == "0" and $old != null and $old.v2 and $old.status == $so and $old.priority == $pro and $old.title == $dt and $old.body == $db then {phase: "record", action: "none", task: $r.id, item: $card.id, cache: $cache, note: $d.note} else @@ -563,7 +629,7 @@ fm_helm_plan_program() { (if any($writes[]; .name == "Status") then $so elif $status_conflict then $bs elif $waiting_status_changed then $cs elif $rebuilt or $status_normal or $cs == $so then $so else $bs end) + "\t" + (if any($writes[]; .name == "Priority") then $pro elif $priority_conflict then $bp elif $rebuilt then $pro else (if $priority_board_changed then $cp else $pro end) end) + "\t" + (if $title_write then $dt elif $title_conflict then $bt elif $rebuilt or $ct == $dt or ($title_board_changed | not) then $dt else $bt end) + "\t" + - (if $body_write then $db elif $body_conflict then $bb elif $rebuilt or $cb == $db or ($body_board_changed | not) then $db else $bb end) + "\t" + $now) as $cache + (if $body_write then $db elif $body_conflict then $bb elif $rebuilt or $cb == $db or ($body_board_changed | not) then $db else $bb end) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)) as $cache | {phase: "record", action: (if $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), task: $r.id, item: $card.id, cache: $cache, draft: $text.draft, @@ -590,6 +656,7 @@ fm_helm_plan_program() { ack_write: ({new: false, text: ($text.draft != ""), title: (if $title_write then $d.title else $card_title end), body: (if $body_write then $d.body else $card_body end), fields: ($writes | map({name, value, option}))} | tojson)} end + end end end ]; def missing_entries: diff --git a/bin/fm-helm-poll.sh b/bin/fm-helm-poll.sh index d4bb4c8c86d..252afbc1ddd 100755 --- a/bin/fm-helm-poll.sh +++ b/bin/fm-helm-poll.sh @@ -1,32 +1,12 @@ #!/usr/bin/env bash -# fm-helm-poll.sh - one cheap read-only poll for a captain edit to the Helm -# board. +# fm-helm-poll.sh - one cheap read-only poll for captain edits on Helm boards. # -# Inert by default: a HARD no-op (exit 0, no output) unless config/helm.json -# exists. The watcher invokes this trusted repository script only after -# state/helm-board.check.sh matches its byte-static identity shim. +# Helm is inert until the main home's config/helm.json exists. The watcher runs +# this trusted repository script through state/helm-board.check.sh. Output wakes +# firstmate; silence keeps the watcher asleep. The poll never mutates a board or +# backlog. It stores one signature per owner/number in state/.helm-board-poll. # -# Contract: "output => wake firstmate, silence => keep sleeping". It prints ONE -# line when the board changed; when a backlog change is pending, the line asks -# firstmate to force a reconciliation rather than silently losing the board -# edit. It never mutates the board or the backlog. It finishes well inside -# FM_CHECK_TIMEOUT. One 20-second deadline covers the whole paginated board -# read. When `timeout` is unavailable, a watchdog stops the `gh` request at -# that same deadline. -# -# Enable (firstmate, main home, once, alongside creating config/helm.json): -# printf 'exec "%s/bin/fm-helm-poll.sh" "$@"\n' "$FM_ROOT" > state/helm-board.check.sh -# chmod 0700 state/helm-board.check.sh -# bin/fm-check-register.sh helm-board -# Retire with bin/fm-check-unregister.sh helm-board. -# -# Snapshot: state/.helm-board-poll (mode 0600) holds the last board signature. -# The signature (fm_helm_board_signature_program in bin/fm-helm-lib.sh, shared -# with the sync) folds, per card, the (Status, Priority, title, body) it shows, -# plus the card count. Any captain edit to those changes it. -# -# When a board and backlog change coincide, the poll emits a reconciliation wake. -# The forced sync leaves any field-level conflict for firstmate to resolve. +# Usage: bin/fm-helm-poll.sh set -u SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -36,9 +16,11 @@ CONFIG_PATH="${FM_CONFIG_OVERRIDE:-$FM_HOME_PATH/config}" DATA_PATH="${FM_DATA_OVERRIDE:-$FM_HOME_PATH/data}" STATE_PATH="${FM_STATE_OVERRIDE:-$FM_HOME_PATH/state}" CONFIG_FILE="$CONFIG_PATH/helm.json" +ROUTING_FILE="$DATA_PATH/helm-project-map.json" SECONDMATES_PATH="$DATA_PATH/secondmates.md" SYNC_HASH_FILE="$STATE_PATH/.helm-sync-backlog.sha256" POLL_FILE="$STATE_PATH/.helm-board-poll" +CARDS_FILE="$STATE_PATH/helm-cards.tsv" TMP_DIR= poll_cleanup() { @@ -50,11 +32,11 @@ trap poll_cleanup EXIT trap 'exit 130' INT trap 'exit 143' TERM -# Hard no-op when Helm is off. [ -f "$CONFIG_FILE" ] && [ -r "$CONFIG_FILE" ] || exit 0 command -v jq >/dev/null 2>&1 || exit 0 command -v gh >/dev/null 2>&1 || exit 0 [ -L "$POLL_FILE" ] && exit 0 +[ -L "$ROUTING_FILE" ] && exit 0 # shellcheck source=bin/fm-helm-lib.sh . "$SCRIPT_DIR/fm-helm-lib.sh" @@ -63,7 +45,6 @@ CONFIG_JSON=$(sed -E '/^[[:space:]]*(\/\/|#)/d' "$CONFIG_FILE") || exit 0 OWNER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.owner | strings | select(length > 0)' 2>/dev/null) || exit 0 PROJECT_NUMBER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.number | numbers | select(. > 0)' 2>/dev/null) || exit 0 -# A pending backlog change means the ordinary sync is about to run: stay quiet. HOMES_TSV="$(fm_helm_discover_homes "$FM_HOME_PATH" "$SECONDMATES_PATH" 2>/dev/null)" || exit 0 BACKLOG_PATHS=() while IFS=$'\t' read -r _hid _hpath; do @@ -73,7 +54,16 @@ done </dev/null) || exit 0 + +TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-poll.XXXXXX") || exit 0 +ROUTING_JSON="$TMP_DIR/routing.json" +if [ -f "$ROUTING_FILE" ]; then + jq -e '.version == 1 and ((.projects // {}) | type == "object")' "$ROUTING_FILE" >/dev/null 2>&1 || exit 0 + cp -- "$ROUTING_FILE" "$ROUTING_JSON" || exit 0 +else + printf '%s\n' '{"version":1,"projects":{},"nudges":{}}' >"$ROUTING_JSON" || exit 0 +fi +BACKLOG_HASH=$(fm_helm_combined_hash "${BACKLOG_PATHS[@]}" "$ROUTING_FILE" 2>/dev/null) || exit 0 BACKLOG_PENDING=1 if [ -f "$SYNC_HASH_FILE" ] && [ "$(sed -n '1p' "$SYNC_HASH_FILE" 2>/dev/null)" = "$BACKLOG_HASH" ]; then BACKLOG_PENDING=0 @@ -82,13 +72,57 @@ fi AUTH_OUTPUT=$(gh auth status 2>&1) || exit 0 printf '%s\n' "$AUTH_OUTPUT" | grep -Eiq "(['\"]project['\"]|(^|[[:space:],])project([[:space:],]|$))" || exit 0 -TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-poll.XXXXXX") || exit 0 -BOARD_JSON="$TMP_DIR/board.json" +BOARD_KEYS_RAW="$TMP_DIR/board-keys.raw" +BOARD_KEYS_FILE="$TMP_DIR/board-keys.tsv" +{ + printf '%s\t%s\n' "$OWNER" "$PROJECT_NUMBER" + jq -r '.projects // {} | to_entries[] | select((.value.state // "active") == "active" or (.value.state // "") == "migrating") | [.value.owner, (.value.number | tostring)] | @tsv' "$ROUTING_JSON" + awk -F '\t' 'NF >= 11 && $10 != "" && $11 != "" { print $10 "\t" $11 }' "$CARDS_FILE" 2>/dev/null || true +} | awk -F '\t' '!seen[$1 SUBSEP $2]++' >"$BOARD_KEYS_RAW" || exit 0 +{ + awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 == owner && $2 == number' "$BOARD_KEYS_RAW" + awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 != owner || $2 != number' "$BOARD_KEYS_RAW" \ + | sort -t $'\t' -k1,1 -k2,2n +} >"$BOARD_KEYS_FILE" || exit 0 +BOARD_COUNT=$(awk 'END { print NR + 0 }' "$BOARD_KEYS_FILE") +POLL_BUDGET=$((20 * BOARD_COUNT)) +[ "$POLL_BUDGET" -le 120 ] || POLL_BUDGET=120 +POLL_DEADLINE=$(( $(date +%s) + POLL_BUDGET )) # shellcheck disable=SC2016 # GraphQL variables must remain literal for gh api. QUERY='query($owner:String!, $number:Int!, $cursor:String) { user(login:$owner) { projectV2(number:$number) { + id + items(first:100, after:$cursor) { + pageInfo { hasNextPage endCursor } + nodes { + id + content { + __typename + ... on DraftIssue { title body } + ... on Issue { title body } + } + fieldValues(first:30) { + nodes { + __typename + ... on ProjectV2ItemFieldSingleSelectValue { + name + field { ... on ProjectV2SingleSelectField { name } } + } + } + } + } + } + } + } +}' + +# shellcheck disable=SC2016 # GraphQL variables must remain literal for gh api. +ORG_QUERY='query($owner:String!, $number:Int!, $cursor:String) { + organization(login:$owner) { + projectV2(number:$number) { + id items(first:100, after:$cursor) { pageInfo { hasNextPage endCursor } nodes { @@ -131,66 +165,105 @@ run_gh_bounded() { return "$status" } -PAGE_COUNT=0 -CURSOR= -PAGINATION_DEADLINE=$(( $(date +%s) + 20 )) -while :; do - PAGE_COUNT=$((PAGE_COUNT + 1)) - [ "$PAGE_COUNT" -le 50 ] || exit 0 - PAGE_JSON="$TMP_DIR/board-page-$PAGE_COUNT.json" - GH_ARGS=( - --field "query=$QUERY" - --field "owner=$OWNER" - --field "number=$PROJECT_NUMBER" - ) - [ -z "$CURSOR" ] || GH_ARGS+=(--field "cursor=$CURSOR") - REMAINING=$(( PAGINATION_DEADLINE - $(date +%s) )) - [ "$REMAINING" -gt 0 ] || exit 0 - run_gh_bounded "$REMAINING" gh api graphql "${GH_ARGS[@]}" >"$PAGE_JSON" 2>/dev/null || exit 0 - jq -e '(.errors // []) | length == 0' "$PAGE_JSON" >/dev/null 2>&1 || exit 0 - if [ "$PAGE_COUNT" -eq 1 ]; then - mv -f -- "$PAGE_JSON" "$BOARD_JSON" - else - jq -s '.[0] as $all | .[1] as $page - | $all - | .data.user.projectV2.items.nodes += $page.data.user.projectV2.items.nodes - | .data.user.projectV2.items.pageInfo = $page.data.user.projectV2.items.pageInfo' \ - "$BOARD_JSON" "$PAGE_JSON" >"$BOARD_JSON.next" 2>/dev/null || exit 0 - mv -f -- "$BOARD_JSON.next" "$BOARD_JSON" || exit 0 - fi - if [ "$(jq -r '.data.user.projectV2.items.pageInfo.hasNextPage' "$BOARD_JSON")" = false ]; then - break - fi - CURSOR=$(jq -r '.data.user.projectV2.items.pageInfo.endCursor // empty' "$BOARD_JSON") - [ -n "$CURSOR" ] || exit 0 -done +BOARD_READ_SEQUENCE=0 +read_board() { + local owner=$1 number=$2 output=$3 query=$QUERY page_count=0 cursor='' page_json remaining + local -a gh_args + while :; do + page_count=$((page_count + 1)) + [ "$page_count" -le 50 ] || return 1 + page_json="$TMP_DIR/board-page-$BOARD_READ_SEQUENCE-$page_count.json" + gh_args=(--field "query=$query" --field "owner=$owner" --field "number=$number") + [ -z "$cursor" ] || gh_args+=(--field "cursor=$cursor") + remaining=$((POLL_DEADLINE - $(date +%s))) + [ "$remaining" -gt 0 ] || return 1 + run_gh_bounded "$remaining" gh api graphql "${gh_args[@]}" >"$page_json" 2>/dev/null || return 1 + if jq -e '(.errors // []) | length > 0' "$page_json" >/dev/null 2>&1; then + return 1 + fi + if [ "$query" = "$ORG_QUERY" ]; then + jq '.data.user.projectV2 = (.data.organization.projectV2 // null)' "$page_json" >"$page_json.normalized" || return 1 + mv -f -- "$page_json.normalized" "$page_json" || return 1 + fi + if ! jq -e '.data.user.projectV2.id' "$page_json" >/dev/null 2>&1; then + if [ "$query" = "$QUERY" ]; then + query=$ORG_QUERY + page_count=0 + cursor= + continue + fi + return 1 + fi + if [ "$page_count" -eq 1 ]; then + cat -- "$page_json" >"$output" || return 1 + else + jq -s '.[0] as $all | .[1] as $page + | $all + | .data.user.projectV2.items.nodes += $page.data.user.projectV2.items.nodes + | .data.user.projectV2.items.pageInfo = $page.data.user.projectV2.items.pageInfo' \ + "$output" "$page_json" >"$output.next" || return 1 + mv -f -- "$output.next" "$output" || return 1 + fi + if [ "$(jq -r '.data.user.projectV2.items.pageInfo.hasNextPage' "$output")" = false ]; then + return 0 + fi + cursor=$(jq -r '.data.user.projectV2.items.pageInfo.endCursor // empty' "$output") + [ -n "$cursor" ] || return 1 + done +} -SIGNATURE=$(jq -r "$(fm_helm_board_signature_program)" "$BOARD_JSON" | fm_helm_sha256_stdin) || exit 0 -[ -n "$SIGNATURE" ] || exit 0 +POLL_WORK="$TMP_DIR/poll-work.tsv" +: >"$POLL_WORK" +if [ -f "$POLL_FILE" ]; then + while IFS= read -r poll_line || [ -n "$poll_line" ]; do + case "$poll_line" in + *$'\t'*) printf '%s\n' "$poll_line" >>"$POLL_WORK" ;; + '') ;; + *) printf '%s/%s\t%s\n' "$OWNER" "$PROJECT_NUMBER" "$poll_line" >>"$POLL_WORK" ;; + esac + done <"$POLL_FILE" +fi -store_signature() { +store_poll() { local tmp tmp=$(mktemp "$TMP_DIR/poll.XXXXXX") || return 1 - printf '%s\n' "$SIGNATURE" >"$tmp" || return 1 + cat -- "$POLL_WORK" >"$tmp" || return 1 chmod 0600 "$tmp" || return 1 mv -f -- "$tmp" "$POLL_FILE" } -# First run (or a cleared snapshot): baseline silently, never wake. -if [ ! -f "$POLL_FILE" ]; then - store_signature - exit 0 -fi -PREVIOUS=$(cat "$POLL_FILE" 2>/dev/null || true) -if [ "$SIGNATURE" = "$PREVIOUS" ]; then - exit 0 -fi - -store_signature || exit 0 +CHANGED=0 +SUCCEEDED_BOARDS=0 +FAILED_BOARDS=() +while IFS=$'\t' read -r board_owner board_number; do + [ -n "$board_owner" ] && [ -n "$board_number" ] || continue + BOARD_READ_SEQUENCE=$((BOARD_READ_SEQUENCE + 1)) + board_key="$board_owner/$board_number" + board_json="$TMP_DIR/board-$BOARD_READ_SEQUENCE.json" + if ! read_board "$board_owner" "$board_number" "$board_json"; then + FAILED_BOARDS+=("$board_key") + continue + fi + SUCCEEDED_BOARDS=$((SUCCEEDED_BOARDS + 1)) + signature=$(jq -r "$(fm_helm_board_signature_program)" "$board_json" | fm_helm_sha256_stdin) || continue + [ -n "$signature" ] || continue + previous=$(awk -F '\t' -v k="$board_key" '$1 == k { print $2; exit }' "$POLL_WORK") + [ -z "$previous" ] || [ "$signature" = "$previous" ] || CHANGED=1 + poll_tmp=$(mktemp "$TMP_DIR/poll.XXXXXX") || exit 0 + awk -F '\t' -v k="$board_key" '$1 != k' "$POLL_WORK" >"$poll_tmp" || exit 0 + printf '%s\t%s\n' "$board_key" "$signature" >>"$poll_tmp" || exit 0 + mv -f -- "$poll_tmp" "$POLL_WORK" || exit 0 +done <"$BOARD_KEYS_FILE" +if [ "${#FAILED_BOARDS[@]}" -gt 0 ]; then + # Preserve successful board baselines and any edit they exposed, while a + # failed board remains retryable on the next bounded poll. + [ "$SUCCEEDED_BOARDS" -gt 0 ] || exit 0 +fi +store_poll || exit 0 +[ "$CHANGED" -eq 1 ] || exit 0 if [ "$BACKLOG_PENDING" -eq 1 ]; then printf 'check: Helm board and backlog both changed; run bin/fm-helm-sync.sh --force to reconcile\n' - exit 0 +else + printf 'check: Helm board edited by the captain; run bin/fm-helm-sync.sh --force to reconcile\n' fi - -printf 'check: Helm board edited by the captain; run bin/fm-helm-sync.sh --force to reconcile\n' diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh new file mode 100755 index 00000000000..a619a6ba88b --- /dev/null +++ b/bin/fm-helm-project-map.sh @@ -0,0 +1,610 @@ +#!/usr/bin/env bash +# fm-helm-project-map.sh - manage per-project Helm board routing. +# +# Usage: +# bin/fm-helm-project-map.sh list [--counts] +# bin/fm-helm-project-map.sh link [] [--owner <login>] [--existing <owner>/<number>] +# bin/fm-helm-project-map.sh move <local-project> [<title>] [--owner <login>] [--existing <owner>/<number>] [--default] [--yes] +# bin/fm-helm-project-map.sh unlink <local-project> +# bin/fm-helm-project-map.sh sync +# +# The main home's data/helm-project-map.json is the durable routing document. +# Board creation, lookup, and card relocation use gh-axi; ordinary card sync +# remains owned by fm-helm-sync.sh. A move records each add/create and delete +# boundary in state/helm-moves.tsv so an interrupted move resumes safely. +set -u + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT_PATH="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME_PATH="${FM_HOME:-$FM_ROOT_PATH}" +CONFIG_PATH="${FM_CONFIG_OVERRIDE:-$FM_HOME_PATH/config}" +DATA_PATH="${FM_DATA_OVERRIDE:-$FM_HOME_PATH/data}" +STATE_PATH="${FM_STATE_OVERRIDE:-$FM_HOME_PATH/state}" +CONFIG_FILE="$CONFIG_PATH/helm.json" +MAP_FILE="$DATA_PATH/helm-project-map.json" +CARDS_FILE="$STATE_PATH/helm-cards.tsv" +MOVES_FILE="$STATE_PATH/helm-moves.tsv" +LOCK_FILE="$STATE_PATH/.helm-sync.lock" +TMP_DIR= +LOCK_HELD=0 +MOVE_TASKS= + +map_cleanup() { + local status=$? + if [ "$LOCK_HELD" -eq 1 ]; then + fm_lock_release "$LOCK_FILE" 2>/dev/null || true + fi + [ -z "$TMP_DIR" ] || [ ! -d "$TMP_DIR" ] || rm -rf -- "$TMP_DIR" + exit "$status" +} +trap map_cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM + +fail() { + printf 'fm-helm-project-map: %s\n' "$*" >&2 + exit 1 +} + +usage() { + sed -n '2,/^set -u$/p' "$0" | sed '$d' +} + +case "${1:-}" in + --help|-h) usage; exit 0 ;; +esac + +command -v jq >/dev/null 2>&1 || fail "jq is required" +command -v gh-axi >/dev/null 2>&1 || fail "gh-axi is required" +[ -d "$DATA_PATH" ] && [ ! -L "$DATA_PATH" ] || fail "data directory is unavailable" +[ -d "$STATE_PATH" ] && [ ! -L "$STATE_PATH" ] || fail "state directory is unavailable" +[ -f "$CONFIG_FILE" ] && [ ! -L "$CONFIG_FILE" ] || fail "Helm is not configured (config/helm.json is absent)" + +# shellcheck source=bin/fm-helm-lib.sh +. "$SCRIPT_DIR/fm-helm-lib.sh" +# shellcheck source=bin/fm-wake-lib.sh +. "$SCRIPT_DIR/fm-wake-lib.sh" + +CONFIG_JSON=$(sed -E '/^[[:space:]]*(\/\/|#)/d' "$CONFIG_FILE") \ + || fail "could not read config/helm.json" +DEFAULT_OWNER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.owner | strings | select(length > 0)' 2>/dev/null) \ + || fail "config/helm.json has no owner" +DEFAULT_NUMBER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.number | numbers | select(. > 0)' 2>/dev/null) \ + || fail "config/helm.json has no number" + +HOMES_TSV=$(fm_helm_discover_homes "$FM_HOME_PATH" "$DATA_PATH/secondmates.md" 2>/dev/null) \ + || fail "could not discover fleet homes" +HOME_PATHS=() +while IFS=$'\t' read -r _home_id home_path; do + [ -n "$home_path" ] || continue + HOME_PATHS+=("$home_path") +done <<EOF +$HOMES_TSV +EOF + +TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-project-map.XXXXXX") \ + || fail "could not create temporary workspace" + +map_load() { + if [ -f "$MAP_FILE" ]; then + [ ! -L "$MAP_FILE" ] || fail "refusing symlinked Helm routing state" + jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ + "$MAP_FILE" >/dev/null 2>&1 || fail "data/helm-project-map.json is invalid" + cp -- "$MAP_FILE" "$TMP_DIR/map.json" || fail "could not stage Helm routing state" + else + printf '%s\n' '{"version":1,"projects":{},"nudges":{}}' >"$TMP_DIR/map.json" + fi +} + +map_publish() { + local tmp + tmp=$(mktemp "$DATA_PATH/.helm-project-map.XXXXXX") || return 1 + chmod 0600 "$tmp" || return 1 + jq -S . "$TMP_DIR/map.json" >"$tmp" || return 1 + chmod 0600 "$tmp" && mv -f -- "$tmp" "$MAP_FILE" +} + +project_registered() { + local project=$1 + fm_helm_project_names "${HOME_PATHS[@]}" | awk -v p="$project" '$0 == p { found=1 } END { exit(found ? 0 : 1) }' +} + +map_entry() { + jq -c --arg p "$1" '.projects[$p] // null' "$TMP_DIR/map.json" +} + +parse_board_ref() { + local ref=$1 + case "$ref" in + */*[!0-9]) return 1 ;; + */*) BOARD_OWNER=${ref%/*}; BOARD_NUMBER=${ref##*/} ;; + *) return 1 ;; + esac + [ -n "$BOARD_OWNER" ] && [ "$BOARD_NUMBER" -gt 0 ] 2>/dev/null +} + +BOARD_OWNER= +BOARD_NUMBER= +BOARD_TITLE= +BOARD_URL= +BOARD_ID= +BOARD_OUTPUT= +PROJECT_SCHEMA_OPTION=other + +board_view() { + local owner=$1 number=$2 line + BOARD_OUTPUT=$(gh-axi project view "$number" --owner "$owner" 2>&1) \ + || return 1 + BOARD_OWNER=$owner + BOARD_NUMBER=$number + BOARD_TITLE=$(printf '%s\n' "$BOARD_OUTPUT" | sed -n 's/^title: //p' | head -1 | sed 's/^"//; s/"$//') + BOARD_URL=$(printf '%s\n' "$BOARD_OUTPUT" | sed -n 's/^url: //p' | head -1 | sed 's/^"//; s/"$//') + BOARD_ID=$(printf '%s\n' "$BOARD_OUTPUT" | sed -n 's/^id: //p' | head -1) + [ -n "$BOARD_ID" ] || return 1 + [ -n "$BOARD_TITLE" ] || BOARD_TITLE="$owner/$number" + for line in "$BOARD_ID" "$BOARD_TITLE"; do [ -n "$line" ] || return 1; done +} + +find_project_number() { + local owner=$1 title=$2 listing + listing=$(gh-axi project list --owner "$owner" --limit 1000 2>/dev/null) || return 1 + printf '%s\n' "$listing" | awk -v wanted="$title" ' + function trim(s) { sub(/^[[:space:]]+/, "", s); sub(/[[:space:]]+$/, "", s); return s } + /^[[:space:]]*[0-9]+,/ { + line = $0; sub(/^[[:space:]]+/, "", line) + n = line; sub(/,.*/, "", n) + rest = line; sub(/^[^,]*,/, "", rest) + if (substr(rest, 1, 1) == "\"") { + sub(/^"/, "", rest); comma = index(rest, "\",") + title = comma ? substr(rest, 1, comma - 1) : rest + } else { + comma = index(rest, ","); title = comma ? substr(rest, 1, comma - 1) : rest + } + if (trim(title) == wanted) print n + }' +} + +create_or_reuse_board() { # <owner> <title> [existing-owner/number] + local owner=$1 title=$2 existing=${3:-} number matches create_output + if [ -n "$existing" ]; then + parse_board_ref "$existing" || fail "invalid --existing board: $existing" + board_view "$BOARD_OWNER" "$BOARD_NUMBER" \ + || fail "GitHub Project $existing does not resolve" + return 0 + fi + matches=$(find_project_number "$owner" "$title" || true) + case "$matches" in + *$'\n'*) fail "more than one GitHub Project named '$title'; use --existing OWNER/NUMBER" ;; + '') + create_output=$(gh-axi project create --owner "$owner" --title "$title" 2>&1) \ + || fail "could not create GitHub Project '$title' for $owner" + number=$(printf '%s\n' "$create_output" | sed -n 's/.*\b\([0-9][0-9]*\)\b.*/\1/p' | tail -1) + [ -n "$number" ] || number=$(find_project_number "$owner" "$title" || true) + [ -n "$number" ] || fail "created GitHub Project '$title' but could not resolve its number" + board_view "$owner" "$number" || fail "created GitHub Project $owner/$number does not resolve" + ;; + *) board_view "$owner" "$matches" || fail "GitHub Project $owner/$matches does not resolve" ;; + esac +} + +field_has_options() { # <field-list> <field-name> <comma-separated-options> + local listing=$1 field=$2 options=$3 option + printf '%s\n' "$listing" | awk -v wanted="$field" ' + $1 == "name:" && substr($0, index($0, $2)) == wanted { found = 1 } + END { exit(found ? 0 : 1) }' >/dev/null + while IFS= read -r option; do + [ -n "$option" ] || continue + printf '%s\n' "$listing" | awk -v wanted="$field" -v option="$option" ' + $1 == "name:" { found = (substr($0, index($0, $2)) == wanted) } + found && $1 == "options:" { value = $0; if (index(value, option ":") > 0) ok=1; found=0 } + END { exit(ok ? 0 : 1) }' || return 1 + done < <(printf '%s' "$options" | tr ',' '\n') +} + +ensure_schema() { + local listing field options query option + listing=$(gh-axi project field-list "$BOARD_NUMBER" --owner "$BOARD_OWNER" --limit 100 2>&1) \ + || fail "could not read Helm fields from $BOARD_OWNER/$BOARD_NUMBER" + while IFS='|' read -r field options; do + field_has_options "$listing" "$field" "$options" && continue + if printf '%s\n' "$listing" | awk -v wanted="$field" '$1 == "name:" && substr($0, index($0, $2)) == wanted { found=1 } END { exit(found ? 0 : 1) }'; then + fail "GitHub Project $BOARD_OWNER/$BOARD_NUMBER has an incomplete $field field; add the missing options before linking" + fi + [ -n "$BOARD_ID" ] || fail "GitHub Project $BOARD_OWNER/$BOARD_NUMBER has no id" + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + query='mutation($projectId:ID!,$name:String!,$options:[String!]!){createProjectV2Field(input:{projectId:$projectId,dataType:SINGLE_SELECT,name:$name,singleSelectOptions:$options}){projectV2Field{id}}}' + set -- + while IFS= read -r option; do + [ -n "$option" ] || continue + set -- "$@" --field "options[]=$option" + done < <(printf '%s' "$options" | tr ',' '\n') + gh-axi api graphql --field "query=$query" --field "projectId=$BOARD_ID" --field "name=$field" "$@" >/dev/null 2>&1 \ + || fail "could not create Helm field $field on $BOARD_OWNER/$BOARD_NUMBER" + listing=$(gh-axi project field-list "$BOARD_NUMBER" --owner "$BOARD_OWNER" --limit 100 2>&1) \ + || fail "could not verify Helm field $field on $BOARD_OWNER/$BOARD_NUMBER" + field_has_options "$listing" "$field" "$options" \ + || fail "Helm field $field on $BOARD_OWNER/$BOARD_NUMBER is still incomplete" + done < <( + printf '%s\n' \ + 'Status|Queued,In flight,Waiting on you,Done' \ + 'Priority|P0,P1,P2,P3,P4' \ + 'Kind|ship,investigation,decision' + printf 'Project|other,%s\n' "$PROJECT_SCHEMA_OPTION" + ) +} + +map_link() { + local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' now + [ -n "$project" ] || fail "link requires a local project name" + project_registered "$project" || fail "local project '$project' is not registered in data/projects.md" + [ -n "$title" ] || title="$project" + shift 2 + while [ "$#" -gt 0 ]; do + case "$1" in + --owner) shift; [ "$#" -gt 0 ] || fail "--owner needs a login"; owner=$1 ;; + --existing) shift; [ "$#" -gt 0 ] || fail "--existing needs OWNER/NUMBER"; existing=$1 ;; + *) fail "unknown link argument: $1" ;; + esac + shift + done + PROJECT_SCHEMA_OPTION=$project + create_or_reuse_board "$owner" "$title" "$existing" + ensure_schema + now=$(date -u +%Y-%m-%dT%H:%M:%SZ) + jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" \ + --arg title "$BOARD_TITLE" --arg url "$BOARD_URL" --arg now "$now" \ + '.projects[$p] = {owner:$owner,number:$number,title:$title,url:$url,state:"active",linked_at:$now,move:null,orphan_hold_task:null} | .nudges |= del(.[$p])' \ + "$TMP_DIR/map.json" >"$TMP_DIR/map.next" || fail "could not prepare the Helm routing entry" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + map_publish || fail "could not publish data/helm-project-map.json" + printf 'linked: %s -> %s/%s "%s"\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" "$BOARD_TITLE" + printf 'link changes future cards only; use move %s --existing %s/%s --yes to relocate existing cards.\n' \ + "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + "$SCRIPT_DIR/fm-helm-sync.sh" || true +} + +backlog_task_ids() { # <project> <output-file> + local project=$1 output=$2 home_id home backlog_out + : >"$output" + while IFS=$'\t' read -r home_id home; do + backlog_out="$TMP_DIR/backlog-${home_id}.json" + fm_helm_parse_home_backlog "$home_id" "$home/data/backlog.md" "$backlog_out" 2>/dev/null \ + || fail "could not parse $home/data/backlog.md" + jq -r --arg p "$project" '.[] | select((.repo // "") == $p or (((.repo // "") | split("/"))[-1] == $p)) | .id' "$backlog_out" >>"$output" \ + || fail "could not find Helm tasks for $project" + done <<EOF +$HOMES_TSV +EOF + sort -u -o "$output" "$output" +} + +move_ledger_publish() { + local tmp=$TMP_DIR/moves.next + cp -- "$TMP_DIR/moves.tsv" "$tmp" || return 1 + chmod 0600 "$tmp" || return 1 + mv -f -- "$tmp" "$MOVES_FILE" +} + +move_ledger_update() { # <task> <new-line> + local task=$1 newline=$2 tmp + tmp=$(mktemp "$TMP_DIR/moves.XXXXXX") || return 1 + awk -F '\t' -v t="$task" '$1 != t' "$TMP_DIR/moves.tsv" >"$tmp" || return 1 + printf '%s\n' "$newline" >>"$tmp" || return 1 + sort -t $'\t' -k1,1 "$tmp" >"$TMP_DIR/moves.sorted" || return 1 + mv -f -- "$TMP_DIR/moves.sorted" "$TMP_DIR/moves.tsv" + move_ledger_publish +} + +move_card_add() { # <type> <content-node> <title> <body> + local type=$1 content=$2 title=$3 body=$4 query response + if [ "$type" = issue ]; then + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + query='mutation($projectId:ID!,$contentId:ID!){addProjectV2ItemById(input:{projectId:$projectId,contentId:$contentId}){item{id}}}' + response=$(gh-axi api graphql --field "query=$query" --field "projectId=$BOARD_ID" --field "contentId=$content" 2>/dev/null) \ + || return 1 + printf '%s\n' "$response" | jq -er '.data.addProjectV2ItemById.item.id' 2>/dev/null + else + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + query='mutation($projectId:ID!,$title:String!,$body:String!){addProjectV2DraftIssue(input:{projectId:$projectId,title:$title,body:$body}){projectItem{id}}}' + response=$(gh-axi api graphql --field "query=$query" --field "projectId=$BOARD_ID" --field "title=$title" --field "body=$body" 2>/dev/null) \ + || return 1 + printf '%s\n' "$response" | jq -er '.data.addProjectV2DraftIssue.projectItem.id' 2>/dev/null + fi +} + +move_card_delete() { # <owner> <number> <item-id> + gh-axi project item-delete "$2" --owner "$1" --id "$3" >/dev/null 2>&1 +} + +# move_card_read_source <source-item-id> reads the current source content +# before a draft move. The identity cache is a routing index, not a board +# snapshot, so captain edits must travel with the card when it is relocated. +move_card_read_source() { + local source_item=$1 source_json query + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + query='query($itemId:ID!){node(id:$itemId){... on ProjectV2Item {content {__typename ... on DraftIssue {title body} ... on Issue {title body}}}}}' + source_json=$(gh-axi api graphql --field "query=$query" --field "itemId=$source_item" 2>/dev/null) \ + || return 1 + MOVE_SOURCE_TYPE=$(jq -r '.data.node.content.__typename // empty' <<<"$source_json") + MOVE_SOURCE_TITLE=$(jq -r '.data.node.content.title // empty' <<<"$source_json") + MOVE_SOURCE_BODY=$(jq -r '.data.node.content.body // empty' <<<"$source_json") + [ "$MOVE_SOURCE_TYPE" = DraftIssue ] || [ "$MOVE_SOURCE_TYPE" = Issue ] || return 1 + [ -n "$MOVE_SOURCE_TITLE" ] && [ -n "$MOVE_SOURCE_BODY" ] +} + +# move_card_copy_fields <source-item-id> <destination-item-id> copies the +# current Helm single-select values before the source item is removed. Field +# and option ids are board-local, so resolve both sides by their stable names. +move_card_copy_fields() { + local source_item=$1 destination_item=$2 source_json destination_json writes field_id option_id query + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + local item_query='query($itemId:ID!){node(id:$itemId){... on ProjectV2Item {fieldValues(first:30){nodes{... on ProjectV2ItemFieldSingleSelectValue{name optionId field{... on ProjectV2SingleSelectField{name}}}}}}}}}' + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + local project_query='query($projectId:ID!){node(id:$projectId){... on ProjectV2 {fields(first:100){nodes{... on ProjectV2SingleSelectField{id name options{id name}}}}}}}' + source_json=$(gh-axi api graphql --field "query=$item_query" --field "itemId=$source_item" 2>/dev/null) \ + || return 1 + destination_json=$(gh-axi api graphql --field "query=$project_query" --field "projectId=$BOARD_ID" 2>/dev/null) \ + || return 1 + jq -e '.data.node != null and (.data.node.fieldValues.nodes | type == "array")' <<<"$source_json" >/dev/null 2>&1 \ + || return 1 + jq -e '.data.node != null and (.data.node.fields.nodes | type == "array")' <<<"$destination_json" >/dev/null 2>&1 \ + || return 1 + writes=$(jq -n --argjson source "$source_json" --argjson destination "$destination_json" ' + ($source.data.node.fieldValues.nodes // []) + | map(. as $value + | select(["Status", "Priority", "Kind", "Project"] | index($value.field.name) != null) + | select(($value.name // "") != "") + | {name:$value.field.name, value:$value.name}) as $values + | ($destination.data.node.fields.nodes // []) as $fields + | [ $values[] as $value + | ([ $fields[] + | select(.name == $value.name) + | .options[]? + | select(.name == $value.value) + | {optionId:.id} ][0]) as $option + | ([ $fields[] | select(.name == $value.name) ][0].id) as $field_id + | select($option != null and ($field_id // "") != "") + | {fieldId:$field_id, optionId:$option.optionId} ] as $writes + | if ($writes | length) == ($values | length) then $writes else error("destination Helm field option is unavailable") end + ' 2>/dev/null) || return 1 + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + query='mutation($projectId:ID!,$itemId:ID!,$fieldId:ID!,$optionId:String!){updateProjectV2ItemFieldValue(input:{projectId:$projectId,itemId:$itemId,fieldId:$fieldId,value:{singleSelectOptionId:$optionId}}){projectV2Item{id}}}' + while IFS=$'\t' read -r field_id option_id; do + [ -n "$field_id" ] && [ -n "$option_id" ] || continue + gh-axi api graphql --field "query=$query" --field "projectId=$BOARD_ID" \ + --field "itemId=$destination_item" --field "fieldId=$field_id" --field "optionId=$option_id" \ + >/dev/null 2>&1 || return 1 + done < <(jq -r '.[] | [.fieldId, .optionId] | @tsv' <<<"$writes") +} + +move_execute() { + local task from_owner from_number from_item to_owner to_number to_item phase updated row type node title body new_item move_line + local remaining=0 + while IFS= read -r move_line; do + # Bash treats tab as IFS whitespace and would collapse the intentionally + # empty destination-item field in a pending ledger row. Parse with a + # non-whitespace separator after preserving the TSV boundaries. + move_line=${move_line//$'\t'/$'\034'} + IFS=$'\034' read -r task from_owner from_number from_item to_owner to_number to_item phase updated <<<"$move_line" + [ -n "$task" ] || continue + case $'\n'$MOVE_TASKS$'\n' in + *$'\n'"$task"$'\n'*) ;; + *) continue ;; + esac + [ "$phase" = complete ] && continue + if [ "$phase" = pending ]; then + row=$(awk -F '\t' -v t="$task" '$1 == t { print; exit }' "$CARDS_FILE" 2>/dev/null) || row= + [ -n "$row" ] || { remaining=$((remaining + 1)); continue; } + if ! move_card_read_source "$from_item"; then + printf 'fm-helm-project-map: source card read failed for %s; it will resume\n' "$task" >&2 + remaining=$((remaining + 1)); continue + fi + type=$MOVE_SOURCE_TYPE + node=$(printf '%s\n' "$row" | awk -F '\t' '{print $3}') + title=$MOVE_SOURCE_TITLE + body=$MOVE_SOURCE_BODY + new_item=$(move_card_add "$type" "$node" "$title" "$body" || true) + if [ -z "$new_item" ]; then + printf 'fm-helm-project-map: destination add failed for %s; it will resume\n' "$task" >&2 + remaining=$((remaining + 1)); continue + fi + phase=dest-ready; to_item=$new_item; updated=$(date +%s) + move_ledger_update "$task" "$task"$'\t'"$from_owner"$'\t'"$from_number"$'\t'"$from_item"$'\t'"$to_owner"$'\t'"$to_number"$'\t'"$to_item"$'\t'"$phase"$'\t'"$updated" \ + || fail "could not publish move progress for $task" + fi + if [ "$phase" = dest-ready ]; then + if ! move_card_copy_fields "$from_item" "$to_item"; then + printf 'fm-helm-project-map: destination fields failed for %s; it will resume\n' "$task" >&2 + remaining=$((remaining + 1)); continue + fi + if ! move_card_delete "$from_owner" "$from_number" "$from_item"; then + printf 'fm-helm-project-map: source delete failed for %s; it will resume\n' "$task" >&2 + remaining=$((remaining + 1)); continue + fi + phase=complete; updated=$(date +%s) + move_ledger_update "$task" "$task"$'\t'"$from_owner"$'\t'"$from_number"$'\t'"$from_item"$'\t'"$to_owner"$'\t'"$to_number"$'\t'"$to_item"$'\t'"$phase"$'\t'"$updated" \ + || fail "could not publish move completion for $task" + if [ -f "$CARDS_FILE" ]; then + awk -F '\t' -v t="$task" -v owner="$to_owner" -v number="$to_number" -v item="$to_item" -v epoch="$updated" \ + 'BEGIN { OFS="\t" } $1 == t {$2=item; $9=epoch; $10=owner; $11=number} {print}' "$CARDS_FILE" >"$TMP_DIR/cards.next" \ + && chmod 0600 "$TMP_DIR/cards.next" && mv -f -- "$TMP_DIR/cards.next" "$CARDS_FILE" + fi + fi + done <"$TMP_DIR/moves.tsv" + printf '%s\n' "$remaining" +} + +map_move() { + local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' default_dest='' yes=0 entry source_owner source_number count now ids + [ -n "$project" ] || fail "move requires a local project name" + map_load + entry=$(map_entry "$project") + if [ "$(printf '%s\n' "$entry" | jq -r '.state // empty')" = migrating ]; then + destination_owner=$(printf '%s\n' "$entry" | jq -r '.owner') + destination_number=$(printf '%s\n' "$entry" | jq -r '.number') + source_owner=$(printf '%s\n' "$entry" | jq -r '.move.from.owner') + source_number=$(printf '%s\n' "$entry" | jq -r '.move.from.number') + BOARD_OWNER=$destination_owner; BOARD_NUMBER=$destination_number + board_view "$BOARD_OWNER" "$BOARD_NUMBER" || fail "migration destination $BOARD_OWNER/$BOARD_NUMBER does not resolve" + printf 'resuming confirmed move for %s -> %s/%s\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + yes=1 + else + project_registered "$project" || fail "local project '$project' is not registered in data/projects.md" + shift 2 + while [ "$#" -gt 0 ]; do + case "$1" in + --owner) shift; [ "$#" -gt 0 ] || fail "--owner needs a login"; owner=$1 ;; + --existing) shift; [ "$#" -gt 0 ] || fail "--existing needs OWNER/NUMBER"; existing=$1 ;; + --default) default_dest=1 ;; + --yes) yes=1 ;; + *) [ -z "$title" ] || fail "unknown move argument: $1"; title=$1 ;; + esac + shift + done + source_owner=$(printf '%s\n' "$entry" | jq -r '.owner // empty') + source_number=$(printf '%s\n' "$entry" | jq -r '.number // empty') + [ -n "$source_owner" ] || source_owner=$DEFAULT_OWNER + [ -n "$source_number" ] || source_number=$DEFAULT_NUMBER + if [ -n "$default_dest" ]; then + BOARD_OWNER=$DEFAULT_OWNER; BOARD_NUMBER=$DEFAULT_NUMBER + board_view "$BOARD_OWNER" "$BOARD_NUMBER" || fail "default Helm Project $BOARD_OWNER/$BOARD_NUMBER does not resolve" + else + [ -n "$title" ] || title="$project" + PROJECT_SCHEMA_OPTION=$project + create_or_reuse_board "$owner" "$title" "$existing" + ensure_schema + fi + fi + [ "$source_owner/$source_number" != "$BOARD_OWNER/$BOARD_NUMBER" ] || fail "source and destination boards are the same" + ids="$TMP_DIR/task-ids" + backlog_task_ids "$project" "$ids" + MOVE_TASKS=$(cat -- "$ids") + : >"$TMP_DIR/moves.tsv" + [ -f "$MOVES_FILE" ] && [ ! -L "$MOVES_FILE" ] && cp -- "$MOVES_FILE" "$TMP_DIR/moves.tsv" + awk -F '\t' -v ids_file="$ids" -v moves_file="$TMP_DIR/moves.tsv" \ + -v owner="$source_owner" -v number="$source_number" \ + -v dest_owner="$BOARD_OWNER" -v dest_number="$BOARD_NUMBER" -v epoch="$(date +%s)" \ + 'FILENAME == ids_file { ids[$1]=1; next } + FILENAME == moves_file { existing[$1]=1; next } + NF >= 11 && ids[$1] && !existing[$1] && $10 == owner && $11 == number { + print $1 "\t" owner "\t" number "\t" $2 "\t" dest_owner "\t" dest_number "\t\tpending\t" epoch + }' "$ids" "$CARDS_FILE" 2>/dev/null >>"$TMP_DIR/moves.tsv" || true + # The portable awk pipeline above intentionally keeps task selection local; + # normalize and de-duplicate any rows already present from an earlier run. + awk -F '\t' 'NF >= 8 { latest[$1]=$0 } END { for (id in latest) print latest[id] }' "$TMP_DIR/moves.tsv" \ + | sort -t $'\t' -k1,1 >"$TMP_DIR/moves.sorted" + mv -f -- "$TMP_DIR/moves.sorted" "$TMP_DIR/moves.tsv" + count=$(awk -F '\t' '$8 != "complete" { n++ } END { print n + 0 }' "$TMP_DIR/moves.tsv") + printf 'move plan: %s -> %s; %s card(s)\n' "$source_owner/$source_number" "$BOARD_OWNER/$BOARD_NUMBER" "$count" + if [ "$yes" -eq 0 ]; then + printf 're-run with --yes to execute.\n' + exit 0 + fi + if ! fm_lock_try_acquire "$LOCK_FILE"; then + fail "another Helm sync or move is already running" + fi + LOCK_HELD=1 + now=$(date -u +%Y-%m-%dT%H:%M:%SZ) + jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" \ + --arg from_owner "$source_owner" --argjson from_number "$source_number" --arg now "$now" \ + '.projects[$p] = ((.projects[$p] // {}) + {owner:$owner,number:$number,state:"migrating",move:{from:{owner:$from_owner,number:$from_number},to:{owner:$owner,number:$number},requested_at:$now,confirmed:true},orphan_hold_task:null})' \ + "$TMP_DIR/map.json" >"$TMP_DIR/map.next" || fail "could not prepare the migration mapping" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + map_publish || fail "could not publish the migration mapping" + if ! chmod 0600 "$TMP_DIR/moves.tsv" || ! move_ledger_publish; then + fail "could not publish state/helm-moves.tsv" + fi + remaining=$(move_execute) + if [ "$remaining" -eq 0 ]; then + jq --arg p "$project" '.projects[$p].state = "active" | .projects[$p].move = null' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + || fail "could not finish the migration mapping" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + map_publish || fail "could not publish the completed migration mapping" + else + printf 'move partial: %s card(s) remain; rerun move or sync to resume.\n' "$remaining" + fi + fm_lock_release "$LOCK_FILE" || fail "could not release the Helm move lock" + LOCK_HELD=0 + "$SCRIPT_DIR/fm-helm-sync.sh" || true +} + +map_unlink() { + local project=$1 + map_load + jq --arg p "$project" '.projects |= del(.[$p]) | .nudges |= del(.[$p])' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + || fail "could not prepare the unlink" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + map_publish || fail "could not publish data/helm-project-map.json" + printf 'unlinked: %s -> default Helm board %s/%s\n' "$project" "$DEFAULT_OWNER" "$DEFAULT_NUMBER" + printf 'unlink changes future cards only; use move %s --default --yes to relocate existing cards.\n' "$project" + "$SCRIPT_DIR/fm-helm-sync.sh" || true +} + +map_list() { + local counts=0 project entry owner number title count + [ "${1:-}" = --counts ] && counts=1 + map_load + while IFS= read -r project; do + [ -n "$project" ] || continue + entry=$(map_entry "$project") + if [ "$(printf '%s\n' "$entry" | jq -r '.orphan_hold_task // empty')" != "" ]; then + printf '%s -> [NEEDS DECISION: see task %s]\n' "$project" "$(printf '%s\n' "$entry" | jq -r '.orphan_hold_task')" + continue + fi + owner=$(printf '%s\n' "$entry" | jq -r '.owner // empty') + number=$(printf '%s\n' "$entry" | jq -r '.number // empty') + title=$(printf '%s\n' "$entry" | jq -r '.title // empty') + if [ -n "$owner" ]; then + printf '%s -> %s/%s "%s"' "$project" "$owner" "$number" "$title" + else + printf '%s -> default (%s/%s)' "$project" "$DEFAULT_OWNER" "$DEFAULT_NUMBER" + fi + if [ "$counts" -eq 1 ]; then + count=0 + if [ -n "$owner" ]; then + count=$(gh-axi project item-list "$number" --owner "$owner" --query "Project:$project" --limit 1000 2>/dev/null \ + | awk '/^[[:space:]]+[A-Za-z0-9_]+,/{n++} END{print n+0}') + else + count=$(gh-axi project item-list "$DEFAULT_NUMBER" --owner "$DEFAULT_OWNER" --query "Project:$project" --limit 1000 2>/dev/null \ + | awk '/^[[:space:]]+[A-Za-z0-9_]+,/{n++} END{print n+0}') + fi + printf ' [%s live cards]' "$count" + fi + printf '\n' + done < <(fm_helm_project_names "${HOME_PATHS[@]}") +} + +map_sync() { + local force=${1:-} + if [ "$force" = --force ]; then + "$SCRIPT_DIR/fm-helm-reconcile.sh" --force + else + "$SCRIPT_DIR/fm-helm-reconcile.sh" --force + fi +} + +[ "$#" -gt 0 ] || { usage >&2; exit 2; } +command=$1; shift +case "$command" in + list) map_list "$@" ;; + link) + [ "$#" -gt 0 ] || fail "link requires a local project name" + project=$1; shift; title= + case "${1:-}" in --*) ;; '') ;; *) title=$1; shift ;; esac + map_load + map_link "$project" "$title" "$@" + ;; + move) + [ "$#" -gt 0 ] || fail "move requires a local project name" + project=$1; shift; title= + case "${1:-}" in --*) ;; '') ;; *) title=$1; shift ;; esac + map_move "$project" "$title" "$@" + ;; + unlink) + [ "$#" -eq 1 ] || fail "unlink requires exactly one local project name" + map_unlink "$1" + ;; + sync) [ "$#" -eq 0 ] || fail "sync takes no arguments"; map_sync ;; + --help|-h) usage ;; + *) usage >&2; exit 2 ;; +esac diff --git a/bin/fm-helm-reconcile.sh b/bin/fm-helm-reconcile.sh new file mode 100755 index 00000000000..7c26c66798c --- /dev/null +++ b/bin/fm-helm-reconcile.sh @@ -0,0 +1,187 @@ +#!/usr/bin/env bash +# fm-helm-reconcile.sh - slowly reconcile Helm routing and board drift. +# +# Usage: bin/fm-helm-reconcile.sh [--force] +# +# This is the third, self-throttled Helm watcher check. It verifies every +# mapped board, records title drift, raises broken mappings through the existing +# captain-hold primitive, resumes confirmed moves, and then asks the ordinary +# multi-board sync to reconcile item drift across all boards. +set -u + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT_PATH="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME_PATH="${FM_HOME:-$FM_ROOT_PATH}" +CONFIG_PATH="${FM_CONFIG_OVERRIDE:-$FM_HOME_PATH/config}" +DATA_PATH="${FM_DATA_OVERRIDE:-$FM_HOME_PATH/data}" +STATE_PATH="${FM_STATE_OVERRIDE:-$FM_HOME_PATH/state}" +CONFIG_FILE="$CONFIG_PATH/helm.json" +MAP_FILE="$DATA_PATH/helm-project-map.json" +LAST_FILE="$STATE_PATH/.helm-reconcile-last" +TMP_DIR= + +reconcile_cleanup() { + local status=$? + [ -z "$TMP_DIR" ] || [ ! -d "$TMP_DIR" ] || rm -rf -- "$TMP_DIR" + exit "$status" +} +trap reconcile_cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM + +fail() { + printf 'fm-helm-reconcile: %s\n' "$*" >&2 + exit 0 +} + +FORCE=0 +while [ "$#" -gt 0 ]; do + case "$1" in + --force) FORCE=1 ;; + --help|-h) + sed -n '2,/^set -u$/p' "$0" | sed '$d' + exit 0 + ;; + *) fail "unknown argument: $1" ;; + esac + shift +done + +[ -f "$CONFIG_FILE" ] && [ ! -L "$CONFIG_FILE" ] || exit 0 +[ -d "$DATA_PATH" ] && [ ! -L "$DATA_PATH" ] || fail "data directory is unavailable" +[ -d "$STATE_PATH" ] && [ ! -L "$STATE_PATH" ] || fail "state directory is unavailable" +command -v jq >/dev/null 2>&1 || exit 0 +command -v gh-axi >/dev/null 2>&1 || exit 0 + +# shellcheck source=bin/fm-helm-lib.sh +. "$SCRIPT_DIR/fm-helm-lib.sh" +# shellcheck source=bin/fm-wake-lib.sh +. "$SCRIPT_DIR/fm-wake-lib.sh" + +CONFIG_JSON=$(sed -E '/^[[:space:]]*(\/\/|#)/d' "$CONFIG_FILE") || fail "could not read config/helm.json" +DEFAULT_OWNER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.owner | strings | select(length > 0)' 2>/dev/null) || exit 0 +DEFAULT_NUMBER=$(printf '%s\n' "$CONFIG_JSON" | jq -er '.number | numbers | select(. > 0)' 2>/dev/null) || exit 0 +INTERVAL_HOURS=$(printf '%s\n' "$CONFIG_JSON" | jq -r '.reconcile_interval_hours // 6' 2>/dev/null) +case "$INTERVAL_HOURS" in ''|*[!0-9]*) INTERVAL_HOURS=6 ;; esac +NOW=$(date +%s) +if [ "$FORCE" -eq 0 ] && [ -f "$LAST_FILE" ] && [ ! -L "$LAST_FILE" ]; then + LAST=$(sed -n '1p' "$LAST_FILE" 2>/dev/null) + case "$LAST" in + ''|*[!0-9]*) ;; + *) [ $((NOW - LAST)) -lt $((INTERVAL_HOURS * 3600)) ] && exit 0 ;; + esac +fi + +TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-reconcile.XXXXXX") || fail "could not create temporary workspace" +if [ -f "$MAP_FILE" ]; then + [ ! -L "$MAP_FILE" ] || fail "refusing symlinked Helm routing state" + jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ + "$MAP_FILE" >/dev/null 2>&1 || fail "data/helm-project-map.json is invalid" + cp -- "$MAP_FILE" "$TMP_DIR/map.json" || fail "could not stage Helm routing state" +else + printf '%s\n' '{"version":1,"projects":{},"nudges":{}}' >"$TMP_DIR/map.json" +fi + +publish_map() { + local tmp + tmp=$(mktemp "$DATA_PATH/.helm-project-map.XXXXXX") || return 1 + jq -S . "$TMP_DIR/map.json" >"$tmp" || return 1 + chmod 0600 "$tmp" && mv -f -- "$tmp" "$MAP_FILE" +} + +record_orphan() { # <project> <reason> + local project=$1 reason=$2 existing id + existing=$(jq -r --arg p "$project" '.projects[$p].orphan_hold_task // empty' "$TMP_DIR/map.json") + [ -n "$existing" ] && return 0 + id=$(fm_helm_raise_mapping_orphan "$FM_HOME_PATH" "$project" "$reason" 2>/dev/null) \ + || { printf 'fm-helm-reconcile: could not create captain hold for %s\n' "$project" >&2; return 1; } + jq --arg p "$project" --arg id "$id" '.projects[$p].orphan_hold_task = $id' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + || return 1 + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + return 0 +} + +BOARD_TITLE= +board_view() { # <owner> <number> + local output + output=$(gh-axi project view "$2" --owner "$1" 2>/dev/null) || return 1 + BOARD_TITLE=$(printf '%s\n' "$output" | sed -n 's/^title: //p' | head -1 | sed 's/^"//; s/"$//') + [ -n "$BOARD_TITLE" ] || BOARD_TITLE="$1/$2" +} + +# Verify local-project names against every local home registry. The mapping is +# intentionally main-home-owned, but its keys route the fleet-wide backlog union. +HOMES_TSV=$(fm_helm_discover_homes "$FM_HOME_PATH" "$DATA_PATH/secondmates.md" 2>/dev/null) || fail "could not discover fleet homes" +HOME_PATHS=() +while IFS=$'\t' read -r _home_id home_path; do + [ -n "$home_path" ] && HOME_PATHS+=("$home_path") +done <<EOF +$HOMES_TSV +EOF +REGISTERED=$(fm_helm_project_names "${HOME_PATHS[@]}" | sort -u) +MAP_DIRTY=0 + +while IFS=$'\t' read -r project owner number title state _hold; do + [ -n "$project" ] || continue + if ! printf '%s\n' "$REGISTERED" | awk -v p="$project" '$0 == p { ok=1 } END { exit(ok ? 0 : 1) }'; then + reason="local project '$project' no longer appears in the project registry (renamed or removed); its Helm routing to $owner/$number is orphaned - point the new project name at this board, send it to the Helm default, or confirm it should be dropped" + record_orphan "$project" "$reason" && MAP_DIRTY=1 + continue + fi + if ! board_view "$owner" "$number"; then + reason="the GitHub Project $owner/$number backing local project '$project' no longer resolves (renamed away from that number, or deleted by hand) - point '$project' at a different existing board, let me create a new one, or send it back to the Helm default" + record_orphan "$project" "$reason" && MAP_DIRTY=1 + continue + fi + if [ "$title" != "$BOARD_TITLE" ]; then + jq --arg p "$project" --arg board_title "$BOARD_TITLE" '.projects[$p].title = $board_title' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + || fail "could not record title drift for $project" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + MAP_DIRTY=1 + fi +done < <(jq -r '.projects // {} | to_entries[] | [.key,.value.owner,(.value.number|tostring),(.value.title // ""),(.value.state // "active"),(.value.orphan_hold_task // "")] | @tsv' "$TMP_DIR/map.json") + +if ! board_view "$DEFAULT_OWNER" "$DEFAULT_NUMBER"; then + fm_wake_append check helm-default-board "check: Helm default board $DEFAULT_OWNER/$DEFAULT_NUMBER could not be resolved" >/dev/null 2>&1 || true +else + # A default-board read is also the cheap source for the optional live-card + # nudge. The full cross-board item reconciliation remains in fm-helm-sync.sh. + while IFS= read -r project; do + [ -n "$project" ] || continue + jq -e --arg p "$project" '.projects[$p] == null' "$TMP_DIR/map.json" >/dev/null 2>&1 || continue + live_count=$(gh-axi project item-list "$DEFAULT_NUMBER" --owner "$DEFAULT_OWNER" --query "Project:$project" --limit 1000 2>/dev/null \ + | awk '/^[[:space:]]+[A-Za-z0-9_]+,/{n++} END{print n+0}') + old_enough=0 + added=$(grep -F -- "- $project " "$FM_HOME_PATH/data/projects.md" 2>/dev/null | sed -n 's/.*(added \([0-9-]*\)).*/\1/p' | head -1) + if [ -n "$added" ] && added_epoch=$(date -d "$added" +%s 2>/dev/null); then + [ $((NOW - added_epoch)) -ge 2592000 ] && old_enough=1 + fi + [ "$live_count" -ge 5 ] || [ "$old_enough" -eq 1 ] || continue + last_nudge=$(jq -r --arg p "$project" '.nudges[$p].last_nudged_at // 0' "$TMP_DIR/map.json") + [ "$last_nudge" -gt 0 ] 2>/dev/null && [ $((NOW - last_nudge)) -lt 2592000 ] && continue + fm_wake_append check "helm-nudge:$project" "check: local project '$project' has $live_count live cards in the Helm default board; consider giving it its own GitHub Project (fm-helm-project-map.sh link $project)" >/dev/null 2>&1 || true + jq --arg p "$project" --argjson now "$NOW" '.nudges[$p] = {last_nudged_at:$now,nudge_count:((.nudges[$p].nudge_count // 0) + 1)}' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + || fail "could not record the Helm nudge for $project" + mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" + MAP_DIRTY=1 + done < <(printf '%s\n' "$REGISTERED") +fi + +if [ "$MAP_DIRTY" -ne 0 ] && ! publish_map; then + fail "could not publish Helm reconciliation state" +fi + +# A confirmed migration is an already-approved operation. Continue it without +# asking for a second confirmation, then let the ordinary sync reconcile item +# existence on every board (including cards that drifted or vanished by hand). +while IFS= read -r project; do + [ -n "$project" ] || continue + "$SCRIPT_DIR/fm-helm-project-map.sh" move "$project" --yes >/dev/null 2>&1 \ + || printf 'fm-helm-reconcile: move resume failed for %s\n' "$project" >&2 +done < <(jq -r '.projects // {} | to_entries[] | select(.value.state == "migrating" and .value.move.confirmed == true) | .key' "$TMP_DIR/map.json") + +printf '%s\n' "$NOW" >"$TMP_DIR/last" +if ! chmod 0600 "$TMP_DIR/last" || ! mv -f -- "$TMP_DIR/last" "$LAST_FILE"; then + fail "could not publish reconcile cadence" +fi +"$SCRIPT_DIR/fm-helm-sync.sh" --force >/dev/null 2>&1 || true diff --git a/bin/fm-helm-sync.sh b/bin/fm-helm-sync.sh index a35e6acde01..8ed6cb4b47e 100755 --- a/bin/fm-helm-sync.sh +++ b/bin/fm-helm-sync.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# fm-helm-sync.sh - reconcile the optional Helm GitHub Project board with the +# fm-helm-sync.sh - reconcile the optional Helm GitHub Project boards with the # whole fleet's backlog. # # The local config/helm.json file is the opt-in. When it is absent this script @@ -7,7 +7,7 @@ # call. # # ## Fleet-aware -# The sync runs from the main home only and is the single writer of the board. +# The sync runs from the main home only and is the single writer of the boards. # It discovers every LOCAL secondmate home from data/secondmates.md, parses each # home's data/backlog.md, and reconciles the union against the board. A card is # "missing" (and closed to Done) only when its task id is in NO home's backlog. @@ -42,13 +42,15 @@ # # ## Debounce and run budget # The watcher-check path debounces all GitHub work on one SHA-256 hash over every -# discovered home's data/backlog.md, stored in +# discovered home's data/backlog.md plus data/helm-project-map.json, stored in # state/.helm-sync-backlog.sha256. Use --force for an explicit board read when # the captain has edited a card without changing any backlog; --force still # never calls bin/fm-spawn.sh and never deletes a card. -# One 25-second deadline covers the whole run: the paginated board read and every -# card write. The script uses `timeout` when available and otherwise stops each -# request with a watchdog at the same deadline. When the deadline arrives with +# Each board gets the existing 25-second work budget within a total cap of 120 +# seconds. The paginated board reads and card writes share the total deadline, +# while each board receives its own slice in stable default-first order. The +# script uses `timeout` when available and otherwise stops each request with a +# watchdog at the same deadline. When the deadline arrives with # card writes still planned, or a card's own write failed, the run stops cleanly, # keeps everything it already landed (see "Durable progress"), prints one # `fm-helm-sync: partial: N cards remain` line, and exits 0; the next run plans @@ -66,7 +68,7 @@ # but blank Status and Priority baselines. After the field write lands, the # sync publishes the complete per-field baseline. The debounce hash and the # deletion tombstones are published only by a complete run. -# The board poll signature (state/.helm-board-poll) is republished at every +# The per-board poll signatures (state/.helm-board-poll) are republished at every # non-fail-open exit from the board as read plus the "landed patches" recorded # for each successful write, so the sync's own writes never read back as a # captain edit; bin/fm-helm-lib.sh's fm_helm_landed_patch_program owns the patch @@ -76,6 +78,7 @@ # state/helm-cards.tsv (mode 0600) maps every synced card: # <task-id> <item-id> <content-node-id> <draft|issue> <status-option> # <priority-option> <title-base64> <body-base64> <last-seen-epoch> +# <board-owner> <board-number> # Version 1 rows with one opaque fingerprint are migrated by adopting the # board-current values as their baseline, so migration cannot fabricate a # divergence. A missing baseline is rebuilt the same way without a wake. The @@ -113,6 +116,7 @@ STATE_PATH="${FM_STATE_OVERRIDE:-$FM_HOME_PATH/state}" BACKLOG_PATH="${FM_BACKLOG_OVERRIDE:-$DATA_PATH/backlog.md}" SECONDMATES_PATH="$DATA_PATH/secondmates.md" CONFIG_FILE="$CONFIG_PATH/helm.json" +ROUTING_FILE="$DATA_PATH/helm-project-map.json" HASH_FILE="$STATE_PATH/.helm-sync-backlog.sha256" DISPATCH_FILE="$STATE_PATH/.helm-dispatch-requests" DIVERGENCE_FILES=( @@ -141,6 +145,7 @@ LOCK_HELD=false FORCE=0 LANDED= SYNC_DEADLINE_SECONDS=25 +TOTAL_DEADLINE_SECONDS=120 helm_cleanup() { local status=$? @@ -209,6 +214,7 @@ command -v jq >/dev/null 2>&1 || helm_fail_open "jq is unavailable" command -v gh >/dev/null 2>&1 || helm_fail_open "gh is unavailable" if [ -L "$HASH_FILE" ] || [ -L "$DISPATCH_FILE" ] || [ -L "$CARDS_FILE" ] || [ -L "$DELETED_FILE" ] \ + || [ -L "$ROUTING_FILE" ] \ || [ -L "$POLL_FILE" ] || [ -L "$RESUME_FILE" ]; then helm_fail_open "refusing symlinked Helm state" fi @@ -234,19 +240,27 @@ DISPATCH_STATUS=$(printf '%s\n' "$CONFIG_JSON" | jq -r '.dispatch_status // "In || helm_fail_open "config/helm.json is not valid JSON" [ -n "$DISPATCH_STATUS" ] || helm_fail_open "config/helm.json has an empty dispatch_status" +if [ -f "$ROUTING_FILE" ]; then + jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ + "$ROUTING_FILE" >/dev/null 2>&1 \ + || helm_fail_open "data/helm-project-map.json is not valid Helm routing JSON" +fi + # Discover every local home and build the combined debounce hash. HOMES_TSV="$(fm_helm_discover_homes "$FM_HOME_PATH" "$SECONDMATES_PATH")" \ || helm_fail_open "could not discover fleet homes" BACKLOG_PATHS=() +HOME_PATHS=() while IFS=$'\t' read -r home_id home_path; do [ -n "$home_path" ] || continue + HOME_PATHS+=("$home_path") BACKLOG_PATHS+=("$home_path/data/backlog.md") done <<EOF $HOMES_TSV EOF [ "${#BACKLOG_PATHS[@]}" -gt 0 ] || helm_fail_open "no fleet home resolved" -BACKLOG_HASH=$(fm_helm_combined_hash "${BACKLOG_PATHS[@]}") \ +BACKLOG_HASH=$(fm_helm_combined_hash "${BACKLOG_PATHS[@]}" "$ROUTING_FILE") \ || helm_fail_open "no SHA-256 utility is available" if [ "$FORCE" -eq 0 ] && [ -f "$HASH_FILE" ] && [ "$(sed -n '1p' "$HASH_FILE" 2>/dev/null)" = "$BACKLOG_HASH" ]; then exit 0 @@ -259,7 +273,8 @@ fi TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-sync.XXXXXX") \ || helm_fail_open "could not create temporary workspace" -BOARD_JSON="$TMP_DIR/board.json" +BOARDS_DIR="$TMP_DIR/boards" +mkdir -p "$BOARDS_DIR" || helm_fail_open "could not create Helm board workspace" GH_ERROR="$TMP_DIR/gh-error" # shellcheck disable=SC2016 # GraphQL variables must remain literal for gh api. @@ -300,6 +315,44 @@ GRAPHQL_QUERY='query($owner:String!, $number:Int!, $cursor:String) { } }' +# shellcheck disable=SC2016 # GraphQL variables must remain literal for gh api. +ORG_GRAPHQL_QUERY='query($owner:String!, $number:Int!, $cursor:String) { + organization(login:$owner) { + projectV2(number:$number) { + id + fields(first:100) { + pageInfo { hasNextPage } + nodes { + __typename + ... on ProjectV2FieldCommon { id name } + ... on ProjectV2SingleSelectField { id name options { id name } } + } + } + items(first:100, after:$cursor) { + pageInfo { hasNextPage endCursor } + nodes { + id + content { + __typename + ... on DraftIssue { id title body } + ... on Issue { id title body number url } + } + fieldValues(first:30) { + nodes { + __typename + ... on ProjectV2ItemFieldSingleSelectValue { + name + optionId + field { ... on ProjectV2SingleSelectField { name } } + } + } + } + } + } + } + } +}' + # shellcheck disable=SC2016 # GraphQL variables must remain literal for gh api. ITEM_GRAPHQL_QUERY='query($itemId:ID!) { node(id:$itemId) { @@ -324,47 +377,57 @@ ITEM_GRAPHQL_QUERY='query($itemId:ID!) { } }' -SYNC_DEADLINE=$(( $(date +%s) + SYNC_DEADLINE_SECONDS )) -PAGE_COUNT=0 -CURSOR= -while :; do - PAGE_COUNT=$((PAGE_COUNT + 1)) - [ "$PAGE_COUNT" -le 50 ] || helm_fail_open "GitHub project has more than 5000 items" - PAGE_JSON="$TMP_DIR/board-page-$PAGE_COUNT.json" - GH_ARGS=( - --field "query=$GRAPHQL_QUERY" - --field "owner=$OWNER" - --field "number=$PROJECT_NUMBER" - ) - [ -z "$CURSOR" ] || GH_ARGS+=(--field "cursor=$CURSOR") - REMAINING=$(( SYNC_DEADLINE - $(date +%s) )) - [ "$REMAINING" -gt 0 ] || helm_fail_open "GitHub project pagination timed out" - if ! run_gh_bounded "$REMAINING" gh api graphql "${GH_ARGS[@]}" >"$PAGE_JSON" 2>"$GH_ERROR"; then - helm_fail_open "GitHub project read failed" - fi - if jq -e '(.errors // []) | length > 0' "$PAGE_JSON" >/dev/null 2>&1; then - helm_fail_open "GitHub project read returned an error" - fi - if ! jq -e '.data.user.projectV2.id and (.data.user.projectV2.fields.pageInfo.hasNextPage == false)' "$PAGE_JSON" >/dev/null 2>&1; then - helm_fail_open "GitHub project fields exceeded the safe page bound" - fi - if [ "$PAGE_COUNT" -eq 1 ]; then - mv -f -- "$PAGE_JSON" "$BOARD_JSON" - else - jq -s '.[0] as $all | .[1] as $page - | $all - | .data.user.projectV2.items.nodes += $page.data.user.projectV2.items.nodes - | .data.user.projectV2.items.pageInfo = $page.data.user.projectV2.items.pageInfo' \ - "$BOARD_JSON" "$PAGE_JSON" >"$BOARD_JSON.next" \ - || helm_fail_open "could not combine GitHub project pages" - mv -f -- "$BOARD_JSON.next" "$BOARD_JSON" - fi - if [ "$(jq -r '.data.user.projectV2.items.pageInfo.hasNextPage' "$BOARD_JSON")" = false ]; then - break - fi - CURSOR=$(jq -r '.data.user.projectV2.items.pageInfo.endCursor // empty' "$BOARD_JSON") - [ -n "$CURSOR" ] || helm_fail_open "GitHub project item page is missing its cursor" -done +TOTAL_DEADLINE= + +# read_board <owner> <number> <output> +# Read one board into the normalized shape consumed by the planner. User-owned +# boards use the first query; organization-owned boards use its fallback. +read_board() { + local owner=$1 number=$2 output=$3 query=$GRAPHQL_QUERY page_count=0 cursor='' page_json remaining + local -a gh_args + while :; do + page_count=$((page_count + 1)) + [ "$page_count" -le 50 ] || return 1 + page_json="$TMP_DIR/board-page-$BOARD_READ_SEQUENCE-$page_count.json" + gh_args=(--field "query=$query" --field "owner=$owner" --field "number=$number") + [ -z "$cursor" ] || gh_args+=(--field "cursor=$cursor") + remaining=$(( TOTAL_DEADLINE - $(date +%s) )) + [ "$remaining" -gt 0 ] || return 1 + run_gh_bounded "$remaining" gh api graphql "${gh_args[@]}" >"$page_json" 2>"$GH_ERROR" || return 1 + if jq -e '(.errors // []) | length > 0' "$page_json" >/dev/null 2>&1; then + return 1 + fi + if [ "$query" = "$ORG_GRAPHQL_QUERY" ]; then + jq '.data.user.projectV2 = (.data.organization.projectV2 // null)' "$page_json" >"$page_json.normalized" \ + || return 1 + mv -f -- "$page_json.normalized" "$page_json" || return 1 + fi + if ! jq -e '.data.user.projectV2.id and (.data.user.projectV2.fields.pageInfo.hasNextPage == false)' "$page_json" >/dev/null 2>&1; then + if [ "$query" = "$GRAPHQL_QUERY" ]; then + query=$ORG_GRAPHQL_QUERY + page_count=0 + cursor= + continue + fi + return 1 + fi + if [ "$page_count" -eq 1 ]; then + cp -- "$page_json" "$output" || return 1 + else + jq -s '.[0] as $all | .[1] as $page + | $all + | .data.user.projectV2.items.nodes += $page.data.user.projectV2.items.nodes + | .data.user.projectV2.items.pageInfo = $page.data.user.projectV2.items.pageInfo' \ + "$output" "$page_json" >"$output.next" || return 1 + mv -f -- "$output.next" "$output" || return 1 + fi + if [ "$(jq -r '.data.user.projectV2.items.pageInfo.hasNextPage' "$output")" = false ]; then + return 0 + fi + cursor=$(jq -r '.data.user.projectV2.items.pageInfo.endCursor // empty' "$output") + [ -n "$cursor" ] || return 1 + done +} # Parse every discovered home's backlog into one tagged union. BACKLOG_JSON="$TMP_DIR/backlog.json" @@ -385,8 +448,6 @@ if ! jq -e 'map(.id) | group_by(.) | all(length == 1)' "$BACKLOG_JSON" >/dev/nul helm_fail_open "the same task id appears in more than one home's backlog" fi -PROJECT_ID=$(jq -r '.data.user.projectV2.id' "$BOARD_JSON") - field_id() { jq -r --arg wanted "$1" \ '[.data.user.projectV2.fields.nodes[] | select(.name == $wanted and .__typename == "ProjectV2SingleSelectField") | .id][0] // empty' \ @@ -399,21 +460,6 @@ option_id() { "$BOARD_JSON" } -STATUS_FIELD_ID=$(field_id Status) -PROJECT_FIELD_ID=$(field_id Project) -KIND_FIELD_ID=$(field_id Kind) -PRIORITY_FIELD_ID=$(field_id Priority) -[ -n "$STATUS_FIELD_ID" ] && [ -n "$PROJECT_FIELD_ID" ] && [ -n "$KIND_FIELD_ID" ] && [ -n "$PRIORITY_FIELD_ID" ] \ - || helm_fail_open "required Helm fields are unavailable" - -STATUS_QUEUED_ID=$(option_id Status "Queued") -STATUS_IN_FLIGHT_ID=$(option_id Status "In flight") -STATUS_WAITING_ID=$(option_id Status "Waiting on you") -STATUS_DONE_ID=$(option_id Status "Done") -DISPATCH_OPTION_ID=$(option_id Status "$DISPATCH_STATUS") -[ -n "$STATUS_QUEUED_ID" ] && [ -n "$STATUS_IN_FLIGHT_ID" ] && [ -n "$STATUS_WAITING_ID" ] && [ -n "$STATUS_DONE_ID" ] && [ -n "$DISPATCH_OPTION_ID" ] \ - || helm_fail_open "required Helm Status options are unavailable" - TMP_RESPONSE="$TMP_DIR/response.json" TMP_RESPONSE_ERROR="$TMP_DIR/response.error" CREATED_ITEM_ID= @@ -607,20 +653,38 @@ publish_file() { } # Board patches recorded for every landed write ("<item>\t<json>" lines), folded -# into the poll signature so the sync's own writes never read back as edits. -LANDED="$TMP_DIR/landed.tsv" -: >"$LANDED" +# into that board's poll signature so the sync's own writes never read back as +# edits. +LANDED= +BOARD_JSON= +POLL_WORK="$TMP_DIR/poll-work.tsv" +: >"$POLL_WORK" +if [ -f "$POLL_FILE" ]; then + while IFS= read -r poll_line || [ -n "$poll_line" ]; do + case "$poll_line" in + *$'\t'*) printf '%s\n' "$poll_line" >>"$POLL_WORK" ;; + '') ;; + *) printf '%s/%s\t%s\n' "$OWNER" "$PROJECT_NUMBER" "$poll_line" >>"$POLL_WORK" ;; + esac + done <"$POLL_FILE" +fi -# publish_progress - republish the poll signature from the board as read plus -# the landed patches, and keep a stopped forced run forced on its next run. +# publish_progress - republish the current board signature from the board as +# read plus its landed patches, and keep a stopped forced run forced on its +# next run. publish_progress() { - local signature + local signature board_key poll_tmp + [ -n "$BOARD_JSON" ] && [ -n "$LANDED" ] || return 0 signature=$(jq -r --rawfile landed "$LANDED" \ "$(fm_helm_landed_patch_program) | $(fm_helm_board_signature_program)" "$BOARD_JSON" \ | fm_helm_sha256_stdin) || return 1 [ -n "$signature" ] || return 1 - printf '%s\n' "$signature" >"$TMP_DIR/poll" || return 1 - publish_file "$TMP_DIR/poll" "$POLL_FILE" || return 1 + board_key="$BOARD_OWNER/$BOARD_NUMBER" + poll_tmp=$(mktemp "$TMP_DIR/poll.XXXXXX") || return 1 + awk -F '\t' -v k="$board_key" '$1 != k' "$POLL_WORK" >"$poll_tmp" || return 1 + printf '%s\t%s\n' "$board_key" "$signature" >>"$poll_tmp" || return 1 + mv -f -- "$poll_tmp" "$POLL_WORK" || return 1 + publish_file "$POLL_WORK" "$POLL_FILE" || return 1 if [ "$FORCE" -eq 1 ]; then : >"$TMP_DIR/resume" && publish_file "$TMP_DIR/resume" "$RESUME_FILE" || return 1 fi @@ -638,9 +702,6 @@ fi # The working copy of the cache: prior rows, replaced row by row as writes land. CACHE_WORK="$TMP_DIR/cache-work.tsv" cp -- "$OLD_CARDS" "$CACHE_WORK" || helm_fail_open "could not stage the Helm identity cache" -# Rows a complete run publishes as the whole new cache. -NEW_CARDS="$TMP_DIR/new-cards.tsv" -: >"$NEW_CARDS" OLD_DELETED="$TMP_DIR/old-deleted.tsv" : >"$OLD_DELETED" if [ -f "$DELETED_FILE" ]; then @@ -673,6 +734,16 @@ cache_publish_row() { publish_file "$CACHE_WORK" "$CARDS_FILE" } +# cache_remove_row <task-id> - remove one card identity after the board confirms +# it was deleted or intentionally suppressed. +cache_remove_row() { + local task_id=$1 tmp + tmp=$(mktemp "$TMP_DIR/cache.XXXXXX") || return 1 + awk -F '\t' -v t="$task_id" '$1 != t' "$CACHE_WORK" >"$tmp" || return 1 + mv -f -- "$tmp" "$CACHE_WORK" || return 1 + publish_file "$CACHE_WORK" "$CARDS_FILE" +} + # Wake keys already queued, plus every key this run raises: one wake per key. RAISED_KEYS=$'\n'"$(fm_wake_queued_keys check)"$'\n' @@ -721,7 +792,45 @@ if [ "${#report_ids[@]}" -gt 0 ]; then || helm_fail_open "could not index task reports" fi DESIRED_JSON="$TMP_DIR/desired.json" -jq --argjson report_ids "$REPORT_IDS" "$(fm_helm_desired_program)" "$BACKLOG_JSON" >"$DESIRED_JSON" \ +ROUTING_JSON="$TMP_DIR/routing.json" +if [ -f "$ROUTING_FILE" ]; then + cp -- "$ROUTING_FILE" "$ROUTING_JSON" || helm_fail_open "could not stage Helm routing state" +else + printf '%s\n' '{"version":1,"projects":{},"nudges":{}}' >"$ROUTING_JSON" \ + || helm_fail_open "could not stage Helm routing defaults" +fi +REGISTERED_PROJECTS=$(fm_helm_project_names "${HOME_PATHS[@]}" \ + | jq -Rsc 'split("\n") | map(select(length > 0))') \ + || helm_fail_open "could not read the project registry" + +# A mapping key that no longer appears in any local registry is a broken +# boundary, not an instruction to silently route future cards to the default. +# Use the existing captain-hold primitive and persist its back-reference so the +# slower reconciliation check does not mint the same hold repeatedly. +ROUTING_DIRTY=0 +while IFS= read -r orphan_project; do + [ -n "$orphan_project" ] || continue + orphan_hold=$(jq -r --arg p "$orphan_project" '.projects[$p].orphan_hold_task // empty' "$ROUTING_JSON") + [ -n "$orphan_hold" ] && continue + orphan_owner=$(jq -r --arg p "$orphan_project" '.projects[$p].owner // ""' "$ROUTING_JSON") + orphan_number=$(jq -r --arg p "$orphan_project" '.projects[$p].number // 0' "$ROUTING_JSON") + orphan_reason="local project '$orphan_project' no longer appears in the project registry (renamed or removed); its Helm routing to $orphan_owner/$orphan_number is orphaned - point the new project name at this board, send it to the Helm default, or confirm it should be dropped" + orphan_hold=$(fm_helm_raise_mapping_orphan "$FM_HOME_PATH" "$orphan_project" "$orphan_reason" 2>/dev/null) \ + || continue + jq --arg p "$orphan_project" --arg id "$orphan_hold" \ + '.projects[$p].orphan_hold_task = $id' "$ROUTING_JSON" >"$TMP_DIR/routing.next" \ + || helm_fail_open "could not record the orphaned Helm mapping for $orphan_project" + mv -f -- "$TMP_DIR/routing.next" "$ROUTING_JSON" + ROUTING_DIRTY=1 +done < <(jq -r --argjson registered "$REGISTERED_PROJECTS" \ + '.projects // {} | to_entries[] as $e | select(($registered | index($e.key)) == null) | $e.key' "$ROUTING_JSON") +if [ "$ROUTING_DIRTY" -eq 1 ]; then + publish_file "$ROUTING_JSON" "$ROUTING_FILE" \ + || helm_fail_open "could not publish Helm routing state" +fi +jq --slurpfile routing "$ROUTING_JSON" --argjson registered "$REGISTERED_PROJECTS" \ + --arg default_owner "$OWNER" --argjson default_number "$PROJECT_NUMBER" \ + --argjson report_ids "$REPORT_IDS" "$(fm_helm_desired_program)" "$BACKLOG_JSON" >"$DESIRED_JSON" \ || helm_fail_open "could not render the Helm cards" # Fingerprint = sha256(status, priority, project, kind, title, sha256(body)), @@ -753,22 +862,113 @@ if [ "$record_count" -gt 0 ]; then [ -s "$FPS" ] || helm_fail_open "could not fingerprint the Helm cards" fi -PLAN="$TMP_DIR/plan.nul" -jq -j --slurpfile desired "$DESIRED_JSON" \ - --rawfile cards "$OLD_CARDS" --rawfile deleted "$OLD_DELETED" --rawfile markers "$OLD_MARKERS" \ - --rawfile divergences "$OLD_DIVERGENCES" \ - --rawfile fps "$FPS" \ - --arg force "$FORCE" --arg dispatch_status "$DISPATCH_STATUS" --arg now "$NOW_EPOCH" \ - --arg tsv_existed "$TSV_EXISTED" \ - "$(fm_helm_plan_program)" "$BOARD_JSON" >"$PLAN" \ - || helm_fail_open "could not plan the Helm reconciliation" - -# Execute the plan. Each entry is FM_HELM_PLAN_FIELDS NUL-terminated fields. +BOARD_INDEX=0 +BOARD_READ_SEQUENCE=0 +BOARD_FAILURES=() +BOARDS_SUCCEEDED=0 +BOARD_OWNER= +BOARD_NUMBER= FAILED=0 REMAINING=0 EXHAUSTED=0 GUARD_CONFLICT=0 CONFLICT_WAKE=0 + +board_failure() { + local key=$1 reason=$2 + local existing + for existing in "${BOARD_FAILURES[@]}"; do + [ "$existing" = "$key" ] && return 0 + done + BOARD_FAILURES+=("$key") + # Keep board-read failures fail-open. The final diagnostic is surfaced by + # the watcher adapter, while a transient network failure must not create a + # durable wake or partial baseline. + : "$reason" +} + +publish_board_failure_wakes() { + local key + [ "$BOARDS_SUCCEEDED" -gt 0 ] || return 0 + for key in "${BOARD_FAILURES[@]}"; do + fm_wake_append check "helm-board-failure:$key" \ + "check: Helm board $key could not be reconciled; retry the board-specific sync" \ + || helm_fail_open "could not enqueue Helm board failure for $key" + done +} + +process_board() { + local owner=$1 number=$2 key="$1/$2" board_file group_desired + local board_now board_budget schema_ok + BOARD_WRITE_FAILURE=0 + BOARD_OWNER=$owner + BOARD_NUMBER=$number + BOARD_INDEX=$((BOARD_INDEX + 1)) + BOARD_READ_SEQUENCE=$((BOARD_READ_SEQUENCE + 1)) + board_file="$BOARDS_DIR/board-$BOARD_INDEX.json" + BOARD_JSON= + LANDED= + if ! read_board "$owner" "$number" "$board_file"; then + board_failure "$key" "GitHub project read failed" + return 0 + fi + BOARDS_SUCCEEDED=$((BOARDS_SUCCEEDED + 1)) + BOARD_JSON="$board_file" + LANDED="$TMP_DIR/landed-$BOARD_INDEX.tsv" + : >"$LANDED" + board_now=$(date +%s) + board_budget=$((board_now + SYNC_DEADLINE_SECONDS)) + [ "$board_budget" -le "$TOTAL_DEADLINE" ] || board_budget=$TOTAL_DEADLINE + SYNC_DEADLINE=$board_budget + PROJECT_ID=$(jq -r '.data.user.projectV2.id' "$BOARD_JSON") + STATUS_FIELD_ID=$(field_id Status) + PROJECT_FIELD_ID=$(field_id Project) + KIND_FIELD_ID=$(field_id Kind) + PRIORITY_FIELD_ID=$(field_id Priority) + if [ -z "$STATUS_FIELD_ID" ] || [ -z "$PROJECT_FIELD_ID" ] || [ -z "$KIND_FIELD_ID" ] || [ -z "$PRIORITY_FIELD_ID" ]; then + board_failure "$key" "required Helm fields are unavailable" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 + fi + DISPATCH_OPTION_ID=$(option_id Status "$DISPATCH_STATUS") + group_desired="$TMP_DIR/desired-$BOARD_INDEX.json" + jq --arg owner "$owner" --argjson number "$number" --rawfile cards "$OLD_CARDS" \ + '($cards | split("\n") | map(select(. != "") | split("\t")) + | map(select(length >= 11 and .[9] == $owner and (.[10] | tonumber) == $number) | .[0])) as $historical + | map(. as $record + | select(($record.desired.board.owner == $owner and $record.desired.board.number == $number) + or (($historical | index($record.id)) != null)))' \ + "$DESIRED_JSON" >"$group_desired" \ + || helm_fail_open "could not group desired Helm cards" + schema_ok=$(jq -e --argjson records "$(cat "$group_desired")" --arg dispatch "$DISPATCH_STATUS" ' + .data.user.projectV2.fields.nodes as $fields + | (def has_option($field; $name): any($fields[]; .name == $field and .__typename == "ProjectV2SingleSelectField" and any(.options[]?; .name == $name)); + any($fields[]; .name == "Status" and .__typename == "ProjectV2SingleSelectField") + and any($fields[]; .name == "Project" and .__typename == "ProjectV2SingleSelectField") + and any($fields[]; .name == "Kind" and .__typename == "ProjectV2SingleSelectField") + and any($fields[]; .name == "Priority" and .__typename == "ProjectV2SingleSelectField") + and all((["Queued", "In flight", "Waiting on you", "Done", $dispatch] | unique)[]; has_option("Status"; .)) + and all(["P0", "P1", "P2", "P3", "P4"][]; has_option("Priority"; .)) + and all(["ship", "investigation", "decision"][]; has_option("Kind"; .)) + and all($records[]; has_option("Project"; .desired.project))) + ' "$BOARD_JSON" 2>/dev/null) || schema_ok= + if [ "$schema_ok" != true ]; then + board_failure "$key" "required Helm fields or options are unavailable" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 + fi + PLAN="$TMP_DIR/plan-$BOARD_INDEX.nul" + jq -j --slurpfile desired "$group_desired" \ + --rawfile cards "$OLD_CARDS" --rawfile deleted "$OLD_DELETED" --rawfile markers "$OLD_MARKERS" \ + --rawfile divergences "$OLD_DIVERGENCES" \ + --rawfile fps "$FPS" \ + --arg force "$FORCE" --arg dispatch_status "$DISPATCH_STATUS" --arg now "$NOW_EPOCH" \ + --arg tsv_existed "$TSV_EXISTED" --arg board_owner "$owner" --argjson board_number "$number" \ + --arg default_owner "$OWNER" --argjson default_number "$PROJECT_NUMBER" \ + "$(fm_helm_plan_program)" "$BOARD_JSON" >"$PLAN" \ + || { board_failure "$key" "could not plan the board reconciliation"; publish_progress || helm_fail_open "could not publish Helm board acknowledgement"; return 0; } + +# Execute the plan. Each entry is FM_HELM_PLAN_FIELDS NUL-terminated fields. E=() read_entry() { @@ -795,7 +995,9 @@ while read_entry; do ack_create=${E[18]} ack_write=${E[19]} fingerprint=${E[20]} note=${E[21]} [ -z "$note" ] || printf '%s\n' "$note" >&2 if [ "$phase" = error ]; then - helm_fail_open "$action" + board_failure "$key" "$action" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 fi if [ "$EXHAUSTED" -eq 1 ]; then case "$action" in create|update|close) REMAINING=$((REMAINING + 1)) ;; esac @@ -808,6 +1010,7 @@ while read_entry; do ;; record:skip) [ -z "$tombstone" ] || printf '%s\n' "$tombstone" >>"$NEW_DELETED" + cache_remove_row "$task_id" || helm_fail_open "could not remove the deleted Helm identity" apply_divergence_ops "$divergence_ops" || helm_fail_open "could not update Helm divergence memory" continue ;; @@ -839,12 +1042,14 @@ while read_entry; do ;; deleted:retain) printf '%s\n' "$tombstone" >>"$NEW_DELETED" + cache_remove_row "$task_id" || helm_fail_open "could not remove the retained Helm identity" continue ;; deleted:hold) backlog_hold_for_captain "$home_path" "$task_id" "$hold_reason" \ || helm_fail_open "could not hold deleted Helm task $task_id" printf '%s\n' "$tombstone" >>"$NEW_DELETED" + cache_remove_row "$task_id" || helm_fail_open "could not remove the deleted Helm identity" raise_wakes "$wakes" \ || printf 'fm-helm-sync: could not enqueue the Helm card deletion for %s\n' "$task_id" >&2 apply_divergence_ops "$divergence_ops" \ @@ -857,7 +1062,8 @@ while read_entry; do esac if [ "$action" = none ]; then - printf '%s\n' "$cache_row" >>"$NEW_CARDS" + cache_publish_row "$task_id" "$cache_row" \ + || helm_fail_open "could not publish the Helm identity cache" continue fi if ! budget_left; then @@ -869,6 +1075,7 @@ while read_entry; do if ! create_draft "$title" "$body"; then printf 'fm-helm-sync: could not create the Helm card for %s\n' "$task_id" >&2 FAILED=$((FAILED + 1)) + BOARD_WRITE_FAILURE=1 continue fi item_id=$CREATED_ITEM_ID @@ -886,6 +1093,7 @@ while read_entry; do fi if [ "$guard_status" -ne 0 ] || ! write_card "$item_id" "$draft_id" "$title" "$body" "$field_writes"; then FAILED=$((FAILED + 1)) + BOARD_WRITE_FAILURE=1 if [ "$action" = close ]; then printf 'fm-helm-sync: could not close the missing Helm task %s\n' "$task_id" >&2 elif [ -n "$field_writes" ]; then @@ -897,6 +1105,7 @@ while read_entry; do fi land "$item_id" "$ack_write" if [ "$action" = close ]; then + cache_remove_row "$task_id" || helm_fail_open "could not remove the closed Helm identity" [ "$marker" != remove ] || marker_remove "$task_id" \ || helm_fail_open "could not clear the Helm dispatch marker for $task_id" continue @@ -905,9 +1114,42 @@ while read_entry; do cache_row=$complete_cache_row fi cache_publish_row "$task_id" "$cache_row" || helm_fail_open "could not publish the Helm identity cache" - printf '%s\n' "$cache_row" >>"$NEW_CARDS" done exec 3<&- +publish_progress || helm_fail_open "could not publish Helm board acknowledgement" +if [ "$BOARD_WRITE_FAILURE" -eq 1 ]; then + board_failure "$key" "one or more board writes failed" +fi +} + +# Stable board order keeps the configured default responsive, then processes +# linked boards by owner and number. Old cache rows keep an empty desired group +# alive long enough to read the board that still owns their cards. +BOARD_KEYS_RAW="$TMP_DIR/board-keys.raw" +BOARD_KEYS_FILE="$TMP_DIR/board-keys.tsv" +{ + printf '%s\t%s\n' "$OWNER" "$PROJECT_NUMBER" + jq -r '.[] | [.desired.board.owner, (.desired.board.number | tostring)] | @tsv' "$DESIRED_JSON" + awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" \ + 'NF >= 11 && $10 != "" && $11 != "" { print $10 "\t" $11 }' "$OLD_CARDS" +} | awk -F '\t' '!seen[$1 SUBSEP $2]++' >"$BOARD_KEYS_RAW" \ + || helm_fail_open "could not build the Helm board groups" +{ + awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 == owner && $2 == number' "$BOARD_KEYS_RAW" + awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 != owner || $2 != number' "$BOARD_KEYS_RAW" \ + | sort -t $'\t' -k1,1 -k2,2n +} >"$BOARD_KEYS_FILE" || helm_fail_open "could not order the Helm board groups" +BOARD_COUNT=$(wc -l <"$BOARD_KEYS_FILE") +TOTAL_BUDGET=$((SYNC_DEADLINE_SECONDS * BOARD_COUNT)) +[ "$TOTAL_BUDGET" -le "$TOTAL_DEADLINE_SECONDS" ] || TOTAL_BUDGET=$TOTAL_DEADLINE_SECONDS +TOTAL_DEADLINE=$(( $(date +%s) + TOTAL_BUDGET )) + +while IFS=$'\t' read -r board_owner board_number; do + [ -n "$board_owner" ] && [ -n "$board_number" ] || continue + process_board "$board_owner" "$board_number" +done <"$BOARD_KEYS_FILE" + +publish_board_failure_wakes if [ "$CONFLICT_WAKE" -eq 1 ]; then printf 'check: Helm board and backlog both changed; reconcile the affected card(s)\n' @@ -915,20 +1157,29 @@ if [ "$CONFLICT_WAKE" -eq 1 ]; then fi if [ "$GUARD_CONFLICT" -eq 1 ]; then - publish_progress || helm_fail_open "could not publish Helm board acknowledgement" printf 'check: Helm board and backlog both changed; run bin/fm-helm-sync.sh --force to reconcile\n' exit 0 fi +if [ "${#BOARD_FAILURES[@]}" -gt 0 ]; then +board_failure_list=$(IFS=', '; printf '%s' "${BOARD_FAILURES[*]}") + printf 'fm-helm-sync: %s board(s) could not be reconciled: %s\n' "${#BOARD_FAILURES[@]}" "$board_failure_list" +fi if [ $((FAILED + REMAINING)) -gt 0 ]; then - publish_progress || helm_fail_open "could not publish Helm board acknowledgement" printf 'fm-helm-sync: partial: %s cards remain\n' $((FAILED + REMAINING)) exit 0 fi +if [ "${#BOARD_FAILURES[@]}" -gt 0 ]; then + exit 0 +fi # A complete run: publish the whole refreshed identity cache, the deletion # tombstones, the debounce hash, and the poll signature. -if [ -s "$NEW_CARDS" ] || [ "$TSV_EXISTED" = true ]; then - sort -u "$NEW_CARDS" >"$TMP_DIR/cards.sorted" || helm_fail_open "could not stage the Helm identity cache" +if [ -s "$CACHE_WORK" ] || [ "$TSV_EXISTED" = true ]; then + jq -r '.[].id' "$BACKLOG_JSON" >"$TMP_DIR/backlog-ids" \ + || helm_fail_open "could not stage the Helm task identities" + awk -F '\t' 'NR == FNR { ids[$1] = 1; next } ids[$1]' \ + "$TMP_DIR/backlog-ids" "$CACHE_WORK" | sort -u >"$TMP_DIR/cards.sorted" \ + || helm_fail_open "could not stage the Helm identity cache" publish_file "$TMP_DIR/cards.sorted" "$CARDS_FILE" || helm_fail_open "could not publish the Helm identity cache" fi @@ -943,7 +1194,6 @@ printf '%s\n' "$BACKLOG_HASH" >"$TMP_DIR/hash" || helm_fail_open "could not writ publish_file "$TMP_DIR/hash" "$HASH_FILE" || helm_fail_open "could not publish Helm sync state" FORCE=0 -publish_progress || helm_fail_open "could not publish Helm board acknowledgement" rm -f -- "$RESUME_FILE" printf 'fm-helm-sync: synchronized\n' diff --git a/docs/1.architecture/_dir.yml b/docs/1.architecture/_dir.yml new file mode 100644 index 00000000000..d5ddc3ef408 --- /dev/null +++ b/docs/1.architecture/_dir.yml @@ -0,0 +1,2 @@ +title: Architecture +icon: i-heroicons-cpu-chip diff --git a/docs/1.architecture/helm-project-routing.md b/docs/1.architecture/helm-project-routing.md new file mode 100644 index 00000000000..d6170f4c455 --- /dev/null +++ b/docs/1.architecture/helm-project-routing.md @@ -0,0 +1,49 @@ +--- +title: Helm Project Routing +description: Routes each registered local project to its own or a shared GitHub Project board. +icon: 'i-heroicons-map' +tags: ['helm', 'github-projects', 'routing'] +--- + +## Overview + +Helm keeps `config/helm.json` as the default board and consults the main home's +`data/helm-project-map.json` for project-specific destinations. The sync parses +the fleet backlog once, groups desired cards by `(owner, number)`, and runs the +existing planner independently for each board. Multiple local projects may +intentionally share a mapped board. + +## Usage + +The mapping is managed by `bin/fm-helm-project-map.sh`: + +```bash +bin/fm-helm-project-map.sh list +bin/fm-helm-project-map.sh link firetabs --existing geojitsu/5 +bin/fm-helm-project-map.sh move firetabs --default --yes +bin/fm-helm-project-map.sh unlink firetabs +bin/fm-helm-project-map.sh sync +``` + +## API Reference + +### `bin/fm-helm-sync.sh` + +Reconciles all discovered local-home backlogs against their routed boards. + +### `bin/fm-helm-poll.sh` + +Stores and compares one signature per routed board. + +### `bin/fm-helm-reconcile.sh` + +Checks mapped-board existence and title drift, resumes confirmed moves, and +routes broken mappings through the existing captain-hold mechanism. + +## Notes + +`state/helm-cards.tsv` records the current board owner and number. An explicit +move uses `state/helm-moves.tsv` and adds or creates the destination card before +deleting the source card, so a crash leaves a resumable phase rather than an +ambiguous relocation. Routing uses project identity from `data/projects.md`; it +does not use checkout paths. diff --git a/docs/2.api/_dir.yml b/docs/2.api/_dir.yml new file mode 100644 index 00000000000..406cefbbf93 --- /dev/null +++ b/docs/2.api/_dir.yml @@ -0,0 +1,2 @@ +title: API Reference +icon: i-heroicons-code-bracket diff --git a/docs/2.api/guides/_dir.yml b/docs/2.api/guides/_dir.yml new file mode 100644 index 00000000000..a8bca2ef40d --- /dev/null +++ b/docs/2.api/guides/_dir.yml @@ -0,0 +1,2 @@ +title: Guides +icon: i-heroicons-book-open diff --git a/docs/2.api/guides/helm-project-routing.md b/docs/2.api/guides/helm-project-routing.md new file mode 100644 index 00000000000..e1b675a23fa --- /dev/null +++ b/docs/2.api/guides/helm-project-routing.md @@ -0,0 +1,41 @@ +--- +title: Configure Helm Project Routing +description: Worked example for linking local projects to GitHub Project boards. +icon: 'i-heroicons-map' +tags: ['helm', 'setup'] +--- + +## Overview + +Helm routing is opt-in. Configure a default board, register local projects in +`data/projects.md`, and use the mapping command when a project needs a separate +board or should share another board intentionally. + +## Usage + +```bash +cp docs/examples/helm.json.example config/helm.json +bin/fm-helm-project-map.sh link firetabs --owner geojitsu +bin/fm-helm-project-map.sh list +``` + +To relocate existing cards, resolve the destination first and then confirm the +operation explicitly: + +```bash +bin/fm-helm-project-map.sh move firetabs --existing geojitsu/5 --yes +``` + +## API Reference + +### `data/helm-project-map.json` + +Version 1 stores `owner`, `number`, `title`, `url`, `state`, `linked_at`, +`move`, and `orphan_hold_task` for each project, plus nudge cooldowns under +`nudges`. The file is main-home-only and mode `0600`. + +## Notes + +`link` and `unlink` affect future cards only. `move` is the only operation that +relocates existing cards, and it can resume from `state/helm-moves.tsv` after a +partial GitHub operation. diff --git a/docs/2.api/helm-project-map.md b/docs/2.api/helm-project-map.md new file mode 100644 index 00000000000..325d65dde92 --- /dev/null +++ b/docs/2.api/helm-project-map.md @@ -0,0 +1,53 @@ +--- +title: Helm Project Map Command +description: Command reference for managing per-project Helm GitHub Project routing. +icon: 'i-heroicons-command-line' +tags: ['helm', 'cli'] +--- + +## Overview + +`fm-helm-project-map.sh` is the direct CLI for inspecting and changing the +main-home Helm routing map. It uses `gh-axi` for board lookup, creation, schema +provisioning, and card relocation. + +## Usage + +```bash +bin/fm-helm-project-map.sh list [--counts] +bin/fm-helm-project-map.sh link <project> [<title>] [--owner <login>] [--existing <owner>/<number>] +bin/fm-helm-project-map.sh move <project> [<title>] [--owner <login>] [--existing <owner>/<number>] [--default] [--yes] +bin/fm-helm-project-map.sh unlink <project> +bin/fm-helm-project-map.sh sync +``` + +## API Reference + +### `list [--counts]` + +Prints every registered project and its default or mapped destination. `--counts` +adds a live Project item count. + +### `link <project>` + +Creates or reuses and provisions a board, then changes future routing. It does +not relocate existing cards. + +### `move <project>` + +Requires `--yes` for a non-interactive relocation. `--default` selects the +configured default board. Confirmed moves are resumable. + +### `unlink <project>` + +Removes the mapping and sends future cards to the default board without moving +existing cards. + +### `sync` + +Runs the routing reconciliation immediately. + +## Notes + +Mapping keys must be registered project names. A missing key means the default +board; a missing or malformed configuration refuses direct management commands. diff --git a/docs/4.configuration.md b/docs/4.configuration.md new file mode 100644 index 00000000000..5a1f303838e --- /dev/null +++ b/docs/4.configuration.md @@ -0,0 +1,34 @@ +--- +title: Configuration +description: Configuration reference for Helm project routing. +icon: 'i-heroicons-adjustments-horizontal' +tags: ['configuration', 'helm'] +--- + +## Overview + +Helm is disabled until the main home contains `config/helm.json`. That file +selects the default GitHub Project board; `data/helm-project-map.json` is the +optional main-home map for per-project destinations. + +## Usage + +Copy `docs/examples/helm.json.example` to `config/helm.json`, then set +`owner` and `number`. Use `bin/fm-helm-project-map.sh` to manage the map. + +## API Reference + +### `config/helm.json` + +`owner` and positive numeric `number` are required. `dispatch_status` defaults +to `In flight`; `reconcile_interval_hours` defaults to `6`. + +### `data/helm-project-map.json` + +Version 1 contains `projects` and `nudges`. A project entry's `state` is +`active`, `provisioning`, or `migrating`; absent entries use the default board. + +## Notes + +The map is keyed by the exact project identity from `data/projects.md`, not by a +checkout path. Both the map and move ledger are mode `0600` local state. diff --git a/docs/architecture.md b/docs/architecture.md index c83144f704d..4461009daaa 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -318,8 +318,8 @@ Teardown is fail-closed for ship worktrees: dirty worktrees refuse, and committe ## Helm board synchronization -The optional `bin/fm-helm-sync.sh` gives the captain one GitHub Project v2 view over the whole fleet's durable backlog without making the board a second task database. -It runs from the main home only and is the single writer of the board. +The optional `bin/fm-helm-sync.sh` gives the captain grouped GitHub Project v2 views over the whole fleet's durable backlog without making boards a second task database. +It runs from the main home only and is the single writer of those boards. It discovers every local secondmate home from `data/secondmates.md`, parses each home's `data/backlog.md` through the one parser in `bin/fm-helm-lib.sh`, and reconciles the union against the board. A card is closed to Done only when its task id is in no home's backlog. Remote secondmate homes are out of scope for now; `bin/fm-helm-lib.sh` records the planned owner-marker path for when the first one appears. @@ -328,6 +328,14 @@ The sync creates missing draft cards and keeps the backlog-owned fields in step It also tolerates a card the captain converted to a real repo issue: such a card keeps full field sync but its title and body are never rewritten. It never archives or deletes a card, and a malformed backlog parse fails open before any board mutation. +`data/helm-project-map.json` routes the exact project identity from the local +registries to an `owner`/`number` board. The configured `config/helm.json` +board remains the default bucket, and several project identities may share any +one board. `fm-helm-project-map.sh` owns linking, explicit card moves, and +unlinking; `state/helm-moves.tsv` makes the two-step add/create-then-delete move +resumable. `fm-helm-reconcile.sh` is the slower cross-board drift check and +uses `fm-captain-hold.sh` for broken mappings or vanished boards. + An explicit `--force` read accepts approved captain board edits and routes unresolved changes through ordinary firstmate intake. The [Helm board sync configuration](configuration.md#helm-board-sync-confighelmjson) owns watcher registration, field authority, wake behavior, and the accepted pre-write containment limit. The script header owns the card identity cache and deletion-tombstone contract. diff --git a/docs/configuration.md b/docs/configuration.md index 69ccb7f6fab..e4073d689d9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -122,17 +122,17 @@ The file format is unchanged in both modes; tasks-axi and manual edits produce t ## Helm board sync (config/helm.json) -The optional `bin/fm-helm-sync.sh` reconciles the whole fleet's backlog with a user-owned GitHub Project v2 board. -Configure it in the main home only: the sync runs from there and is the single writer of the board. +The optional `bin/fm-helm-sync.sh` reconciles the whole fleet's backlog with one or more user-owned GitHub Project v2 boards. +Configure it in the main home only: the sync runs from there and is the single writer of the boards. Copy [`docs/examples/helm.json.example`](examples/helm.json.example) to `config/helm.json` and set the board owner and number before enabling the sync. The tracked example lives under `docs/examples/` so the whole `config/` directory can stay gitignored, while the real configuration remains local and untracked. -The accepted configuration fields are `owner`, `number`, and optional `dispatch_status`, which defaults to `In flight`. +The accepted configuration fields are `owner`, `number`, optional `dispatch_status` (default `In flight`), and optional `reconcile_interval_hours` (default `6`). The configured GitHub account needs the `project` scope, which provides the project read and write access used by the sync. An absent `config/helm.json` makes the script exit 0 without reading any backlog or contacting GitHub. -The sync discovers every local secondmate home from `data/secondmates.md` and reconciles the union of every home's `data/backlog.md` against the board, so a secondmate's cards are managed too and a card is closed to Done only when its task id is in no home's backlog. +The sync discovers every local secondmate home from `data/secondmates.md` and reconciles the union of every home's `data/backlog.md` against grouped boards. A project named in `data/projects.md` can route its cards to its mapped `owner`/`number`; unmapped projects use the configured default board. Multiple projects may intentionally share one mapped board. Remote secondmate homes are not handled yet; see the "Remote homes" note in `bin/fm-helm-lib.sh`. -Bootstrap automatically registers `state/helm-sync.check.sh` and `state/helm-board.check.sh` with the main home's watcher while this configuration exists. +Bootstrap automatically registers `state/helm-sync.check.sh`, `state/helm-board.check.sh`, and `state/helm-reconcile.check.sh` with the main home's watcher while this configuration exists. The watcher runs the sync and authenticated board poll at its ordinary check cadence, and the combined backlog hash avoids a GitHub call when no local home changed. That one main-home writer covers every discovered local secondmate backlog, so secondmates do not need copied Helm configuration or competing sync processes. If a fail-open sync skip would leave a backlog item unsynchronized, its diagnostic becomes a durable watcher `check` wake instead of being silent. @@ -160,6 +160,24 @@ The sync records each request and unresolved field divergence with durable marke `state/helm-cards.tsv` (mode 0600) maps cards to tasks and stores the per-field board baselines used for reconciliation. The sync rebuilds the cache silently from the board when absent, and the `bin/fm-helm-sync.sh` header owns its row format and the per-card publication rule. +`data/helm-project-map.json` is the optional main-home routing map. Its +version-1 `projects` entries use the exact local project name as the key and +contain `owner`, `number`, `title`, `url`, `state`, `linked_at`, `move`, and +`orphan_hold_task`; an absent entry means the configured default board. The +`nudges` object remembers the once-per-30-days suggestion for an unlinked +project. The file is mode `0600` and is included in the sync debounce hash. + +Use `bin/fm-helm-project-map.sh list [--counts]` to inspect routing, +`link <project> [<title>] [--owner <login>] [--existing <owner>/<number>]` to +create or reuse and provision a board, `unlink <project>` to route future cards +back to the default, and `sync` to run routing reconciliation immediately. +`move <project> [<title>] [--owner <login>] [--existing <owner>/<number>] [--default] [--yes]` +is the only command that relocates existing cards. It records add/create and +delete progress in `state/helm-moves.tsv` and resumes a confirmed migration. +`bin/fm-helm-reconcile.sh` checks mapped-board existence and title drift, +continues confirmed moves, and sends broken local or board mappings through the +existing captain-hold path. + `bin/fm-helm-poll.sh` is the automatically registered watcher check (through `state/helm-board.check.sh`, bound with `bin/fm-check-register.sh helm-board`) that wakes firstmate to run `--force` when the captain edits a card, including when a backlog change is pending. It is inert until `config/helm.json` exists. diff --git a/docs/examples/helm.json.example b/docs/examples/helm.json.example index 57887988e8d..2420c4b77aa 100644 --- a/docs/examples/helm.json.example +++ b/docs/examples/helm.json.example @@ -6,12 +6,14 @@ // The real config/helm.json is local and gitignored (the whole config/ // directory is excluded). // An absent config/helm.json disables Helm synchronization completely. - // Bootstrap registers main-home sync and board-poll watcher checks - // automatically while this config exists. They cover every discovered local - // secondmate backlog too. + // Bootstrap registers main-home sync, board-poll, and routing-reconcile + // watcher checks automatically while this config exists. They cover every + // discovered local secondmate backlog too. // "dispatch_status" is the Status option a captain moves a card into to ask // firstmate to pick that task up; it defaults to "In flight". "owner": "geojitsu", "number": 2, - "dispatch_status": "In flight" + "dispatch_status": "In flight", + // Optional self-throttle for routing reconciliation; defaults to 6 hours. + "reconcile_interval_hours": 6 } diff --git a/docs/scripts.md b/docs/scripts.md index d83d6562741..574c99108f6 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -31,10 +31,12 @@ The shared no-mistakes gate refusal for fleet lifecycle entrypoints is summarize | `fm-remote-doctor.sh` | Check, and with `--fix` repair, one remote account's second-mate readiness (remote job worker, Herdr, Aqua launch agents, PATH, and required tools) | | `fm-backlog-handoff.sh` | Move queued backlog items into a secondmate home and durably wake its recorded receiver | | `fm-backlog-receive.sh` | Idempotently ingest one confined remote handoff outbox through tasks-axi | -| `fm-helm-sync.sh` | Reconcile the opt-in GitHub Project board against every local home's `data/backlog.md`, accept captain board edits back on `--force`, and queue captain dispatch and delete-confirmation wakes | +| `fm-helm-sync.sh` | Reconcile the opt-in grouped GitHub Project boards against every local home's `data/backlog.md`, accept captain board edits back on `--force`, and queue captain dispatch and delete-confirmation wakes | | `fm-helm-watch.sh` | Watcher-check adapter for Helm sync: silence successful/debounced runs and expose fail-open sync diagnostics as durable check wakes | -| `fm-helm-poll.sh` | One cheap read-only poll for a captain edit to the Helm board; prints a forced-reconciliation wake for idle or concurrent backlog changes | -| `fm-helm-lib.sh` | Shared Helm helpers: local fleet home discovery, the one `data/backlog.md` parser, the combined debounce hash, the shared board signature, the card renderer and one-pass reconciliation planner, and sha256 helpers | +| `fm-helm-poll.sh` | One cheap read-only poll for captain edits to every routed Helm board; prints a forced-reconciliation wake for idle or concurrent backlog changes | +| `fm-helm-project-map.sh` | List, link, move, unlink, or sync per-project Helm board routing; moves are resumable | +| `fm-helm-reconcile.sh` | Self-throttled cross-board routing, board-drift, and confirmed-move reconciliation through captain-hold | +| `fm-helm-lib.sh` | Shared Helm helpers: local fleet home discovery, the one `data/backlog.md` parser, project registry lookup, the combined debounce hash, per-board signatures, the card renderer and one-pass reconciliation planner, and sha256 helpers | | `fm-captain-hold.sh` | Hold tasks for the captain, record the captain's answers, gate investigation completion, and report record divergence between the status log and the backlog | | `fm-decision-hold.sh` | One-release compatibility shim mapping the retired decision commands onto fm-captain-hold.sh | | `fm-brief.sh` | Scaffold ship (explicit `--mode`), scout, secondmate-charter, and Herdr-lab briefs, with Captain's intent and Firstmate spec subsections on ship/scout | diff --git a/tests/fm-helm-poll.test.sh b/tests/fm-helm-poll.test.sh index dff0599ee0e..9afb64bbae0 100755 --- a/tests/fm-helm-poll.test.sh +++ b/tests/fm-helm-poll.test.sh @@ -9,7 +9,6 @@ set -u . "$(dirname "${BASH_SOURCE[0]}")/lib.sh" POLL="$ROOT/bin/fm-helm-poll.sh" -SYNC="$ROOT/bin/fm-helm-sync.sh" TMP_ROOT=$(fm_test_tmproot fm-helm-poll) command -v jq >/dev/null 2>&1 || { echo "skip: jq not found"; exit 0; } @@ -23,7 +22,7 @@ board_fixture() { # <status> <priority-name> <priority-id> {id:"queued-status",name:"Queued"},{id:"flight-status",name:"In flight"}, {id:"waiting-status",name:"Waiting on you"},{id:"done-status",name:"Done"}]}, {__typename:"ProjectV2SingleSelectField",id:"project-field",name:"Project",options:[ - {id:"firstmate-project",name:"firstmate"},{id:"other-project",name:"other"}]}, + {id:"fixture-firstmate-project",name:"fixture-firstmate"},{id:"other-project",name:"other"}]}, {__typename:"ProjectV2SingleSelectField",id:"kind-field",name:"Kind",options:[ {id:"ship-kind",name:"ship"},{id:"investigation-kind",name:"investigation"}, {id:"decision-kind",name:"decision"}]}, @@ -65,7 +64,11 @@ SH } run_poll() { # <case-dir> <fakebin> - FM_HOME="$1/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$1/board.json" \ + mkdir -p "$1/gh-config" + [ "$(PATH="$2:$PATH" command -v gh)" = "$2/gh" ] \ + || fail "poll test did not install its fail-closed gh fake" + GH_CONFIG_DIR="$1/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$1/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$1/board.json" \ FM_FAKE_BOARD_PAGE_2="${FM_FAKE_BOARD_PAGE_2:-}" \ PATH="$2:$PATH" "$POLL" } @@ -78,7 +81,11 @@ run_poll_without_timeout() { # <case-dir> <fakebin> ln -sf "$(command -v "$command")" "$portable_bin/$command" done ln -sf "$2/gh" "$portable_bin/gh" - FM_HOME="$1/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$1/board.json" \ + [ "$(PATH="$portable_bin" command -v gh)" = "$portable_bin/gh" ] \ + || fail "portable poll test did not install its fail-closed gh fake" + mkdir -p "$1/gh-config" + GH_CONFIG_DIR="$1/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$1/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$1/board.json" \ FM_FAKE_BOARD_PAGE_2="$1/page-two.json" PATH="$portable_bin" "$POLL" } @@ -92,19 +99,23 @@ out=$(run_poll "$case_dir" "$fb" 2>&1) || fail "poll without config exited nonze [ -e "$case_dir/home/state/.helm-board-poll" ] && fail "poll without config wrote state" pass "the board poll is inert until config/helm.json exists" -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] t - T (repo: firstmate) (kind: ship) (since: 2026-09-05) +- [ ] t - T (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) ## Done EOF board_fixture Queued P3 p3-priority > "$case_dir/board.json" -# Make the backlog "quiescent": record its combined hash the way the sync does. -FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$case_dir/board.json" \ - PATH="$fb:$PATH" "$SYNC" >/dev/null 2>&1 || fail "seed sync failed" +# Make the backlog "quiescent": record the combined hash through the same +# helper the sync uses. This poll fixture intentionally does not model write +# mutations, so it should not need a successful board sync just to baseline. +FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" bash -c \ + '. "$1/bin/fm-helm-lib.sh"; fm_helm_combined_hash "$2/data/backlog.md" "$2/data/helm-project-map.json"' \ + _ "$ROOT" "$case_dir/home" >"$case_dir/home/state/.helm-sync-backlog.sha256" \ + || fail "could not seed the sync hash" # First poll baselines silently. out=$(run_poll "$case_dir" "$fb" 2>&1) || fail "first poll exited nonzero: $out" @@ -114,7 +125,7 @@ pass "the first board poll baselines the signature without waking" # No change -> silent. out=$(run_poll "$case_dir" "$fb" 2>&1) -[ -z "$out" ] || fail "an unchanged board woke firstmate: $out" +[ -z "$out" ] || fail "an unchanged board woke fixture-firstmate: $out" pass "an unchanged board produces no wake" # Captain edits the board while the backlog is quiescent -> one wake line. @@ -126,21 +137,23 @@ pass "a board edit on a quiescent backlog prints exactly one wake line" # Same edit again -> already baselined -> silent. out=$(run_poll "$case_dir" "$fb" 2>&1) -[ -z "$out" ] || fail "the same board edit woke firstmate twice: $out" -pass "a board edit wakes firstmate only once" +[ -z "$out" ] || fail "the same board edit woke fixture-firstmate twice: $out" +pass "a board edit wakes fixture-firstmate only once" # Board change while a backlog change is pending -> reconciliation wake. board_fixture Done P4 p4-priority > "$case_dir/board.json" cat >> "$case_dir/home/data/backlog.md" <<'EOF' -- [ ] t2 - Another (repo: firstmate) (kind: ship) (since: 2026-09-06) +- [ ] t2 - Another (repo: fixture-firstmate) (kind: ship) (since: 2026-09-06) EOF out=$(run_poll "$case_dir" "$fb" 2>&1) printf '%s\n' "$out" | grep -F 'Helm board and backlog both changed' >/dev/null \ || fail "a concurrent board and backlog change did not request reconciliation: $out" pass "a board change with a pending backlog change requests reconciliation" -FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_FAKE_BOARD="$case_dir/board.json" \ - PATH="$fb:$PATH" "$SYNC" >/dev/null 2>&1 || fail "pagination sync baseline failed" +FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" bash -c \ + '. "$1/bin/fm-helm-lib.sh"; fm_helm_combined_hash "$2/data/backlog.md" "$2/data/helm-project-map.json"' \ + _ "$ROOT" "$case_dir/home" >"$case_dir/home/state/.helm-sync-backlog.sha256" \ + || fail "could not seed the pagination sync hash" rm -f "$case_dir/home/state/.helm-board-poll" board_fixture Done P4 p4-priority | jq '.data.user.projectV2.items.pageInfo={hasNextPage:true,endCursor:"page-2"}' > "$case_dir/board.json" board_fixture Queued P3 p3-priority > "$case_dir/page-two.json" @@ -151,7 +164,7 @@ board_fixture "In flight" P0 p0-priority > "$case_dir/page-two.json" FM_FAKE_BOARD_PAGE_2="$case_dir/page-two.json" out=$(run_poll "$case_dir" "$fb" 2>&1) \ || fail "second paginated poll exited nonzero: $out" printf '%s\n' "$out" | grep -F 'fm-helm-sync.sh --force' >/dev/null \ - || fail "a page-two board edit did not wake firstmate: $out" + || fail "a page-two board edit did not wake fixture-firstmate: $out" pass "the board poll detects an edit on a second project page" rm -f "$case_dir/home/state/.helm-board-poll" diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh new file mode 100755 index 00000000000..f6dea3319c2 --- /dev/null +++ b/tests/fm-helm-project-map.test.sh @@ -0,0 +1,340 @@ +#!/usr/bin/env bash +# Behavior tests for Helm routing, the resumable move ledger, and reconciliation. +# Every GitHub fixture below uses an obviously fake owner and board number. +set -u + +# shellcheck source=tests/lib.sh +# shellcheck disable=SC1091 +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +LIB="$ROOT/bin/fm-helm-lib.sh" +MAP="$ROOT/bin/fm-helm-project-map.sh" +RECONCILE="$ROOT/bin/fm-helm-reconcile.sh" +TMP_ROOT=$(fm_test_tmproot fm-helm-project-map) + +seed_home() { # <case-dir> + local case_dir=$1 + mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" + printf '%s\n' '{"owner":"fixture-owner","number":999}' >"$case_dir/home/config/helm.json" + cat >"$case_dir/home/data/projects.md" <<'EOF' +# Projects +- alpha [no-mistakes] - Alpha (added 2026-09-01) +- beta [no-mistakes] - Beta (added 2026-09-01) +EOF + cat >"$case_dir/home/data/backlog.md" <<'EOF' +# Backlog + +## In flight +## Queued +- [ ] alpha-task - Alpha task (repo: alpha) (kind: ship) (since: 2026-09-10) +## Done +EOF +} + +write_empty_board() { # <path> + jq -n ' + {data:{user:{projectV2:{id:"project-board",fields:{pageInfo:{hasNextPage:false},nodes:[ + {__typename:"ProjectV2SingleSelectField",id:"status-field",name:"Status",options:[ + {id:"queued-status",name:"Queued"},{id:"flight-status",name:"In flight"}, + {id:"waiting-status",name:"Waiting on you"},{id:"done-status",name:"Done"}]}, + {__typename:"ProjectV2SingleSelectField",id:"project-field",name:"Project",options:[ + {id:"alpha-project",name:"alpha"},{id:"beta-project",name:"beta"},{id:"other-project",name:"other"}]}, + {__typename:"ProjectV2SingleSelectField",id:"kind-field",name:"Kind",options:[ + {id:"ship-kind",name:"ship"},{id:"investigation-kind",name:"investigation"},{id:"decision-kind",name:"decision"}]}, + {__typename:"ProjectV2SingleSelectField",id:"priority-field",name:"Priority",options:[ + {id:"p0-priority",name:"P0"},{id:"p1-priority",name:"P1"},{id:"p2-priority",name:"P2"}, + {id:"p3-priority",name:"P3"},{id:"p4-priority",name:"P4"}]}]}, + items:{pageInfo:{hasNextPage:false},nodes:[]}}}}}' >"$1" +} + +install_sync_gh() { # <case-dir> + local case_dir=$1 fb + fb=$(fm_fakebin "$case_dir") + cat >"$fb/gh" <<'SH' +#!/usr/bin/env bash +set -u +printf 'fake-gh ' >>"$FM_HELM_GH_LOG" +printf '%s' "$*" | tr '\n' ' ' >>"$FM_HELM_GH_LOG" +printf '\n' >>"$FM_HELM_GH_LOG" +if [ "${1:-}" = auth ] && [ "${2:-}" = status ]; then + printf '%s\n' "Token scopes: 'project', 'repo'" + exit 0 +fi +if [ "${1:-}" = api ]; then + case "$*" in + *addProjectV2DraftIssue*) + jq -n '{data:{addProjectV2DraftIssue:{projectItem:{id:"sync-created-item",content:{__typename:"DraftIssue",id:"sync-created-draft",title:"created",body:"created"}}}}}' ;; + *updateProjectV2DraftIssue*|*updateProjectV2ItemFieldValue*) + jq -n '{data:{updateProjectV2DraftIssue:{draftIssue:{id:"sync-created-draft"}},updateProjectV2ItemFieldValue:{projectV2Item:{id:"sync-created-item"}}}}' ;; + *'node(id:'*) + jq -n '{data:{node:{id:"sync-created-item",content:{__typename:"DraftIssue",id:"sync-created-draft",title:"created",body:"created"},fieldValues:{nodes:[]}}}}' ;; + *) cat "$FM_HELM_BOARD_JSON" ;; + esac + exit 0 +fi +exit 1 +SH + chmod +x "$fb/gh" + printf '%s\n' "$fb" +} + +assert_fake_tools() { # <fake-bin> + local fb=$1 actual + actual=$(PATH="$fb:$PATH" command -v gh) + [ "$actual" = "$fb/gh" ] || fail "gh is not shadowed by the fail-closed test fake: $actual" + actual=$(PATH="$fb:$PATH" command -v gh-axi) + [ "$actual" = "$fb/gh-axi" ] || fail "gh-axi is not shadowed by the fail-closed test fake: $actual" +} + +run_sync_case() { # <case-dir> <fakebin> <board-json> + mkdir -p "$1/gh-config" + GH_CONFIG_DIR="$1/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$1/home" FM_ROOT_OVERRIDE="$ROOT" FM_HELM_GH_LOG="$1/gh.log" \ + FM_HELM_BOARD_JSON="$3" PATH="$2:$PATH" "$ROOT/bin/fm-helm-sync.sh" --force +} + +assert_fake_log() { # <case-dir> + local log line + for log in "$1"/gh.log "$1"/gh-axi.log; do + [ -f "$log" ] || continue + while IFS= read -r line; do + case "$line" in + fake-gh\ *|fake-gh-axi\ *) ;; + *) fail "unexpected non-fake GitHub log entry: $line" ;; + esac + done <"$log" + done +} + +# A registered project and a second registered project may share the default +# board; only an unknown repo falls into the existing `other` field bucket. +case_dir="$TMP_ROOT/routing" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,state:"active"}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +jq -n '[ + {id:"alpha-task",title:"Alpha",repo:"alpha",state:"queued",kind:"ship",priority:"3",body_lines:[],home_backlog:"/tmp/home/data/backlog.md"}, + {id:"beta-task",title:"Beta",repo:"beta",state:"queued",kind:"ship",priority:"3",body_lines:[],home_backlog:"/tmp/home/data/backlog.md"}, + {id:"unknown-task",title:"Unknown",repo:"unregistered",state:"queued",kind:"ship",priority:"3",body_lines:[],home_backlog:"/tmp/home/data/backlog.md"} +]' >"$case_dir/records.json" +desired=$(FM_ROOT_OVERRIDE="$ROOT" bash -c \ + '. "$1"; jq --slurpfile routing "$2/home/data/helm-project-map.json" \ + --argjson registered "[\"alpha\",\"beta\"]" --arg default_owner fixture-owner --argjson default_number 999 \ + --argjson report_ids "[]" "$(fm_helm_desired_program)" "$3"' \ + _ "$LIB" "$case_dir" "$case_dir/records.json") +printf '%s\n' "$desired" | jq -e ' + .[0].desired.project == "alpha" and .[0].desired.board == {owner:"fixture-org",number:998} and + .[1].desired.project == "beta" and .[1].desired.board == {owner:"fixture-owner",number:999} and + .[2].desired.project == "other" and .[2].desired.board == {owner:"fixture-owner",number:999} +' >/dev/null || fail "dynamic project registry lookup or board routing is wrong" +pass "registered projects route to their mapped boards and unknown repos use other" + +# Exercise the grouped sync itself: it must read the configured default and the +# mapped board in one run, even though only the mapped project has a card. +fb=$(install_sync_gh "$case_dir") +PATH="$fb:$PATH" command -v gh | grep -Fx "$fb/gh" >/dev/null \ + || fail "sync test did not install its fail-closed gh fake" +: >"$case_dir/gh.log" +run_sync_case "$case_dir" "$fb" "$case_dir/board.json" >/dev/null 2>&1 \ + || fail "multi-board sync exited nonzero" +grep -F 'owner=fixture-owner' "$case_dir/gh.log" >/dev/null \ + || fail "multi-board sync did not read the configured default board" +grep -F 'owner=fixture-org' "$case_dir/gh.log" >/dev/null \ + || fail "multi-board sync did not read the mapped board" +pass "sync reads and processes the default and mapped board groups" + +install_gh_axi() { # <case-dir> <missing-board:yes|no> + local case_dir=$1 missing=${2:-no} fb + fb=$(fm_fakebin "$case_dir") + cat >"$fb/gh-axi" <<SH +#!/usr/bin/env bash +set -u +printf 'fake-gh-axi ' >>"\$FM_HELM_GH_AXI_LOG" +printf '%s' "\$*" | tr '\n' ' ' >>"\$FM_HELM_GH_AXI_LOG" +printf '\n' >>"\$FM_HELM_GH_AXI_LOG" +if [ "\${1:-}" = project ] && [ "\${2:-}" = view ]; then + if [ "$missing" = yes ] && [ "\${5:-}" = fixture-org ] && [ "\${3:-}" = 998 ]; then exit 1; fi + if [ "\${5:-}" = fixture-org ] && [ "\${3:-}" = 997 ]; then + printf '%s\n' 'id: beta-board' 'title: Beta Board Renamed' 'url: https://github.com/orgs/fixture-org/projects/997' + else + printf '%s\n' 'id: helm-board' 'title: Helm Default' 'url: https://github.com/users/fixture-owner/projects/999' + fi + exit 0 +fi +if [ "\${1:-}" = project ] && [ "\${2:-}" = field-list ]; then + printf '%s\n' \ + 'id: status-field' 'name: Status' 'options: "Queued:queued,In flight:flight,Waiting on you:waiting,Done:done"' \ + 'id: priority-field' 'name: Priority' 'options: "P0:p0,P1:p1,P2:p2,P3:p3,P4:p4"' \ + 'id: kind-field' 'name: Kind' 'options: "ship:ship,investigation:investigation,decision:decision"' \ + 'id: project-field' 'name: Project' 'options: "other:other,alpha:alpha,beta:beta"' + exit 0 +fi +if [ "\${1:-}" = project ] && [ "\${2:-}" = item-list ]; then exit 0; fi +if [ "\${1:-}" = project ] && [ "\${2:-}" = item-delete ]; then + [ -z "\${FM_HELM_FAIL_DELETE:-}" ] || exit 1 + exit 0 +fi +if [ "\${1:-}" = api ]; then + case "\$*" in + *'itemId=old-item'* ) + jq -n '{data:{node:{content:{__typename:"DraftIssue",title:"Captain-edited Alpha",body:"\`alpha-task\`\\n\\nCaptain body"},fieldValues:{nodes:[ + {name:"Queued",optionId:"queued-status",field:{name:"Status"}}, + {name:"P3",optionId:"p3-priority",field:{name:"Priority"}}, + {name:"ship",optionId:"ship-kind",field:{name:"Kind"}}, + {name:"alpha",optionId:"alpha-project",field:{name:"Project"}} + ]}}}}' ;; + *'fields(first:100)'* ) + jq -n '{data:{node:{fields:{nodes:[ + {__typename:"ProjectV2SingleSelectField",id:"dest-status-field",name:"Status",options:[{id:"dest-queued-status",name:"Queued"}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-priority-field",name:"Priority",options:[{id:"dest-p3-priority",name:"P3"}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-kind-field",name:"Kind",options:[{id:"dest-ship-kind",name:"ship"}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-project-field",name:"Project",options:[{id:"dest-alpha-project",name:"alpha"}]} + ]}}}}' ;; + *addProjectV2DraftIssue*) jq -n '{data:{addProjectV2DraftIssue:{projectItem:{id:"moved-item"}}}}' ;; + *) jq -n '{data:{}}' ;; + esac +exit 0 +fi +exit 1 +SH + cat >"$fb/gh" <<'SH' +#!/usr/bin/env bash +set -u + printf 'fake-gh ' >>"$FM_HELM_GH_LOG" + printf '%s' "$*" | tr '\n' ' ' >>"$FM_HELM_GH_LOG" + printf '\n' >>"$FM_HELM_GH_LOG" +if [ "${1:-}" = auth ] && [ "${2:-}" = status ]; then + printf '%s\n' "Token scopes: 'project', 'repo'" + exit 0 +fi +if [ "${1:-}" = api ]; then + cat "$FM_HELM_BOARD_JSON" + exit 0 +fi +exit 1 +SH + chmod +x "$fb/gh-axi" + chmod +x "$fb/gh" + printf '%s\n' "$fb" +} + +# A confirmed move records add/delete boundaries and finishes with the cache on +# the destination board. The final sync is faked only at the GitHub boundary. +case_dir="$TMP_ROOT/move" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +title_b64=$(printf '%s' 'Alpha task' | base64 | tr -d '\n') +# shellcheck disable=SC2016 # backticks are card-body literals, not expansions. +body_b64=$(printf '%s' '`alpha-task`\n\nBody' | base64 | tr -d '\n') +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-task old-item old-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ + >"$case_dir/home/state/helm-cards.tsv" +jq -n '{version:1,projects:{alpha:{owner:"fixture-owner",number:999,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_FAIL_DELETE=1 FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >/dev/null 2>&1 \ + || fail "confirmed move exited nonzero" +grep -F 'api graphql' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move did not add the destination item" +grep -F 'title=Captain-edited Alpha' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move did not copy the source card's current title" +# shellcheck disable=SC2016 # backticks are card-body literals, not expansions. +grep -F 'body=`alpha-task`' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move did not copy the source card's current body" +grep -F 'updateProjectV2ItemFieldValue' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move did not copy current field values to the destination item" +grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item\tdest-ready' \ + "$case_dir/home/state/helm-moves.tsv" >/dev/null \ + || fail "a failed source delete did not leave a resumable destination-ready ledger row" +jq -e '.projects.alpha.state == "migrating" and .projects.alpha.move.confirmed == true and .projects.alpha.number == 998' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "a partial move did not retain its confirmed migration mapping" + +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --yes >/dev/null 2>&1 \ + || fail "resuming the confirmed move exited nonzero" +grep -F 'item-delete 999 --owner fixture-owner --id old-item' "$case_dir/gh-axi.log" >/dev/null \ + || fail "resuming the move did not delete the source item" +jq -e '.projects.alpha.state == "active" and .projects.alpha.move == null and .projects.alpha.number == 998' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "move did not finish the routing mapping" +grep -F $'alpha-task\tmoved-item\told-draft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "move did not update the cache to the destination item" +grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item\tcomplete' \ + "$case_dir/home/state/helm-moves.tsv" >/dev/null \ + || fail "move ledger did not retain its complete add/delete history" +pass "confirmed moves preserve card history in the resumable ledger" + +# Reconciliation covers two sides of a broken mapping through the existing +# captain-hold path: a board that disappeared and a local project that left the +# registry. It also refreshes title drift on another board in the same pass. +case_dir="$TMP_ROOT/reconcile" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,title:"Alpha Board",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null},beta:{owner:"fixture-org",number:997,title:"Old Beta Title",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null},ghost:{owner:"fixture-owner",number:999,title:"Ghost Board",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(install_gh_axi "$case_dir" yes) +assert_fake_tools "$fb" +mkdir -p "$case_dir/holds" +mkdir -p "$case_dir/gh-config" +cat >"$fb/tasks-axi" <<'SH' +#!/usr/bin/env bash +set -u +case "${1:-}" in + --version) printf '%s\n' '0.2.5' ;; + update) [ "${2:-}" = --help ] && printf '%s\n' '--archive-body' ;; + mv) [ "${2:-}" = --help ] && printf '%s\n' 'tasks-axi mv <id> [<id>...]' ;; + hold) + if [ "${2:-}" = --help ]; then + printf '%s\n' '--kind captain' + else + : >"$FM_HELM_HOLD_DIR/${2:-unknown}" + fi + ;; + add) + file= + previous= + for arg in "$@"; do + [ "$previous" = --file ] && file=$arg + previous=$arg + done + [ -z "$file" ] || printf '%s\n' "- [ ] ${2:-unknown} - ${3:-hold} (repo: fixture-firstmate) (kind: decision) (since: 2026-09-16)" >>"$file" + : >"$FM_HELM_HOLD_DIR/${2:-unknown}" + ;; + show) + if [ -e "$FM_HELM_HOLD_DIR/${2:-unknown}" ]; then + printf '%s\n' ' state: queued' ' held: yes' ' hold_kind: captain' ' title: Helm routing hold' ' body: -' + else + exit 1 + fi + ;; + *) exit 0 ;; +esac +SH +chmod +x "$fb/tasks-axi" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_HOLD_DIR="$case_dir/holds" FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" \ + FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" "$RECONCILE" --force >"$case_dir/reconcile.out" 2>&1 \ + || fail "cross-board reconciliation exited nonzero" +jq -e ' + .projects.alpha.orphan_hold_task == "helm-map-alpha" and + .projects.ghost.orphan_hold_task == "helm-map-ghost" and + .projects.beta.title == "Beta Board Renamed" +' "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "cross-board orphan or title drift was not reconciled" +grep -F 'helm-map-alpha' "$case_dir/home/data/backlog.md" >/dev/null \ + || fail "missing mapped board did not use the existing captain-hold path" +grep -F 'helm-map-ghost' "$case_dir/home/data/backlog.md" >/dev/null \ + || fail "unregistered mapped project did not use the existing captain-hold path" +pass "cross-board orphan and drift reconciliation uses captain holds" +for case_dir in "$TMP_ROOT"/*; do + [ -d "$case_dir" ] || continue + assert_fake_log "$case_dir" +done +pass "all Helm routing tests used only the fail-closed GitHub fakes" diff --git a/tests/fm-helm-sync.test.sh b/tests/fm-helm-sync.test.sh index f0cc40290da..6cba9d41688 100644 --- a/tests/fm-helm-sync.test.sh +++ b/tests/fm-helm-sync.test.sh @@ -18,7 +18,7 @@ TMP_ROOT=$(fm_test_tmproot fm-helm-sync) command -v jq >/dev/null 2>&1 || { echo "skip: jq not found"; exit 0; } # board_json <extra-item-nodes-json> - a project fixture with the P0..P4 and the -# nocout/cryptoseacurrents options the shipped board carries. +# fixture-nocout/fixture-csc options the shipped board carries. board_json() { local items=$1 spool spool=$(mktemp "$TMP_ROOT/items.XXXXXX") || fail "could not spool board items" @@ -35,11 +35,11 @@ board_json() { {id:"done-status",name:"Done"} ]}, {__typename:"ProjectV2SingleSelectField",id:"project-field",name:"Project",options:[ - {id:"firetabs-project",name:"firetabs"}, - {id:"bbt-project",name:"BetterBlueToo"}, - {id:"firstmate-project",name:"firstmate"}, - {id:"nocout-project",name:"nocout"}, - {id:"csc-project",name:"cryptoseacurrents"}, + {id:"fixture-firetabs-project",name:"fixture-firetabs"}, + {id:"bbt-project",name:"fixture-blue"}, + {id:"fixture-firstmate-project",name:"fixture-firstmate"}, + {id:"fixture-nocout-project",name:"fixture-nocout"}, + {id:"csc-project",name:"fixture-csc"}, {id:"other-project",name:"other"} ]}, {__typename:"ProjectV2SingleSelectField",id:"kind-field",name:"Kind",options:[ @@ -207,7 +207,19 @@ SH run_sync() { # <case-dir> <fakebin> [--force] local case_dir=$1 fb=$2 arg=${3:-} local -a a=() + mkdir -p "$case_dir/gh-config" + [ "$(PATH="$fb:$PATH" command -v gh)" = "$fb/gh" ] \ + || fail "sync test did not install its fail-closed gh fake" [ -z "$arg" ] || a+=("$arg") + if [ ! -f "$case_dir/home/data/projects.md" ]; then + cat > "$case_dir/home/data/projects.md" <<'EOF' +# Projects + +- fixture-firetabs [direct-PR] - test project (added 2026-09-01) +- fixture-firstmate [no-mistakes] - test project (added 2026-09-01) +- fixture-nocout [no-mistakes] - test project (added 2026-09-01) +EOF + fi cp "$case_dir/board.json" "$case_dir/board-state.json" if [ -n "${FM_FAKE_BOARD_PAGE_2:-}" ]; then if jq -s '.[0] as $first | .[1] as $second @@ -220,7 +232,8 @@ run_sync() { # <case-dir> <fakebin> [--force] fail "could not stage paginated fake board state" fi fi - FM_HOME="$case_dir/home" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$case_dir/home" \ FM_ROOT_OVERRIDE="$ROOT" \ FM_FAKE_BOARD="$case_dir/board.json" \ FM_FAKE_BOARD_STATE="$case_dir/board-state.json" \ @@ -240,8 +253,12 @@ run_sync() { # <case-dir> <fakebin> [--force] run_poll() { # <case-dir> <fakebin> local case_dir=$1 fb=$2 + mkdir -p "$case_dir/gh-config" + [ "$(PATH="$fb:$PATH" command -v gh)" = "$fb/gh" ] \ + || fail "poll test did not install its fail-closed gh fake" cp "$case_dir/board.json" "$case_dir/board-state.json" - FM_HOME="$case_dir/home" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$case_dir/home" \ FM_ROOT_OVERRIDE="$ROOT" \ FM_FAKE_BOARD="$case_dir/board.json" \ FM_FAKE_BOARD_STATE="$case_dir/board-state.json" \ @@ -253,8 +270,12 @@ run_poll() { # <case-dir> <fakebin> run_watch() { # <case-dir> <fakebin> local case_dir=$1 fb=$2 + mkdir -p "$case_dir/gh-config" + [ "$(PATH="$fb:$PATH" command -v gh)" = "$fb/gh" ] \ + || fail "watch test did not install its fail-closed gh fake" cp "$case_dir/board.json" "$case_dir/board-state.json" - FM_HOME="$case_dir/home" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 \ + FM_HOME="$case_dir/home" \ FM_ROOT_OVERRIDE="$ROOT" \ FM_FAKE_BOARD="$case_dir/board.json" \ FM_FAKE_BOARD_STATE="$case_dir/board-state.json" \ @@ -274,41 +295,41 @@ run_watch() { # <case-dir> <fakebin> case_dir="$TMP_ROOT/pr-link-shapes" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] github-pr-task - GitHub PR task (repo: firstmate) (kind: ship) (since: 2026-09-05) https://github.com/geojitsu/firstmate/pull/123 -- [ ] gitlab-mr-task - GitLab MR task (repo: nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/dc-noc/nocout/-/merge_requests/123 -- [ ] gitlab-bare-mr-task - GitLab bare MR task (repo: nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/dc-noc/nocout/merge_requests/456 -- [ ] gitlab-zero-mr-task - GitLab zero MR task (repo: nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/dc-noc/nocout/-/merge_requests/0 -- [ ] gitlab-suffixed-mr-task - GitLab suffixed MR task (repo: nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/dc-noc/nocout/-/merge_requests/12x +- [ ] github-pr-task - GitHub PR task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) https://github.com/fixture-owner/fixture-firstmate/pull/123 +- [ ] gitlab-mr-task - GitLab MR task (repo: fixture-nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/123 +- [ ] gitlab-bare-mr-task - GitLab bare MR task (repo: fixture-nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/fixture-org/fixture-nocout/merge_requests/456 +- [ ] gitlab-zero-mr-task - GitLab zero MR task (repo: fixture-nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/0 +- [ ] gitlab-suffixed-mr-task - GitLab suffixed MR task (repo: fixture-nocout) (kind: ship) (since: 2026-09-05) https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/12x ## Done EOF board_json '[]' > "$case_dir/board.json" run_sync "$case_dir" "$fb" >/dev/null 2>&1 || fail "PR-link shape sync failed" -jq -e --arg url 'https://github.com/geojitsu/firstmate/pull/123' ' +jq -e --arg url 'https://github.com/fixture-owner/fixture-firstmate/pull/123' ' [.data.user.projectV2.items.nodes[].content.body] | any(.[]; contains("- **PR:** " + $url)) ' "$case_dir/board-state.json" >/dev/null \ || fail "a GitHub pull-request URL did not render in the Facts section" -jq -e --arg url 'https://gitlab.com/dc-noc/nocout/-/merge_requests/123' ' +jq -e --arg url 'https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/123' ' [.data.user.projectV2.items.nodes[].content.body] | any(.[]; contains("- **PR:** " + $url)) ' "$case_dir/board-state.json" >/dev/null \ || fail "a GitLab merge-request URL did not render in the Facts section" -jq -e --arg url 'https://gitlab.com/dc-noc/nocout/merge_requests/456' ' +jq -e --arg url 'https://gitlab.com/fixture-org/fixture-nocout/merge_requests/456' ' [.data.user.projectV2.items.nodes[].content.body] | all(.[]; contains("- **PR:** " + $url) | not) ' "$case_dir/board-state.json" >/dev/null \ || fail "a bare GitLab merge-request URL rendered in the Facts section" -jq -e --arg url 'https://gitlab.com/dc-noc/nocout/-/merge_requests/0' ' +jq -e --arg url 'https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/0' ' [.data.user.projectV2.items.nodes[].content.body] | all(.[]; contains("- **PR:** " + $url) | not) ' "$case_dir/board-state.json" >/dev/null \ || fail "a zero GitLab merge-request URL rendered in the Facts section" -jq -e --arg url 'https://gitlab.com/dc-noc/nocout/-/merge_requests/12x' ' +jq -e --arg url 'https://gitlab.com/fixture-org/fixture-nocout/-/merge_requests/12x' ' [.data.user.projectV2.items.nodes[].content.body] | all(.[]; contains("- **PR:** " + $url) | not) ' "$case_dir/board-state.json" >/dev/null \ @@ -322,12 +343,12 @@ case_dir="$TMP_ROOT/union" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" \ "$case_dir/sm/data" "$case_dir/sm/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2,"dispatch_status":"In flight"}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999,"dispatch_status":"In flight"}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] main-task - Main queued (repo: firstmate) (kind: ship) (priority: 0) (since: 2026-09-05) +- [ ] main-task - Main queued (repo: fixture-firstmate) (kind: ship) (priority: 0) (since: 2026-09-05) ## Done EOF cat > "$case_dir/home/data/secondmates.md" <<EOF @@ -337,7 +358,7 @@ cat > "$case_dir/sm/data/backlog.md" <<'EOF' # Backlog ## In flight -- [ ] sm-task - Secondmate flight (repo: firetabs) (kind: ship) (priority: 2) (since: 2026-09-05) +- [ ] sm-task - Secondmate flight (repo: fixture-firetabs) (kind: ship) (priority: 2) (since: 2026-09-05) ## Done EOF board_json "$(jq -n \ @@ -367,12 +388,12 @@ pass "fleet union reconciles every home and closes only cards in no home's backl pagination_dir="$TMP_ROOT/pagination" mkdir -p "$pagination_dir/home/config" "$pagination_dir/home/data" "$pagination_dir/home/state" pagination_fb=$(install_fakes "$pagination_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$pagination_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$pagination_dir/home/config/helm.json" cat > "$pagination_dir/home/data/backlog.md" <<'EOF' # Backlog ## In flight -- [ ] page-two-task - Second page task (repo: firstmate) (kind: ship) (since: 2026-09-05) +- [ ] page-two-task - Second page task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) ## Done EOF board_json '[]' | jq '.data.user.projectV2.items.pageInfo={hasNextPage:true,endCursor:"page-2"}' > "$pagination_dir/board.json" @@ -399,17 +420,17 @@ pass "identity cache rebuilds from the board when absent" # --------------------------------------------------------------------------- # Delete detection: a previously synced card gone from the board holds its -# still-live task for the captain and wakes firstmate; it is not a new card. +# still-live task for the captain and wakes fixture-firstmate; it is not a new card. # --------------------------------------------------------------------------- case_dir="$TMP_ROOT/delete" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" acknowledged_body=$(cat <<'EOF' ## Facts -- **Repo:** firstmate +- **Repo:** fixture-firstmate - **Type:** ship - produces a change and a PR - **Priority:** P3 - **Filed:** 2026-09-09 @@ -418,15 +439,15 @@ acknowledged_body=$(cat <<'EOF' --- -_Source of truth: `data/backlog.md` in the owning firstmate home._ +_Source of truth: `data/backlog.md` in the owning local home._ EOF ) cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] keep-task - Keep me (repo: firstmate) (kind: ship) (since: 2026-09-05) -- [ ] del-task - Delete my card (repo: firstmate) (kind: ship) (since: 2026-09-05) +- [ ] keep-task - Keep me (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) +- [ ] del-task - Delete my card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) ## Done EOF board_json "$(jq -n \ @@ -457,7 +478,7 @@ fi grep -F 'hold del-task --kind captain' "$case_dir/tasks-axi.log" >/dev/null \ || fail "deleted card did not hold its live task for the captain: $(cat "$case_dir/tasks-axi.log")" grep -F $'\tcheck\thelm-card-deleted:del-task\t' "$case_dir/home/state/.wake-queue" >/dev/null \ - || fail "deleted card did not wake firstmate" + || fail "deleted card did not wake fixture-firstmate" if grep -F 'helm-new-card:del-task' "$case_dir/home/state/.wake-queue" >/dev/null; then fail "a deleted card was misread as a brand-new captain card" fi @@ -467,8 +488,8 @@ cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] keep-task - Keep me (repo: firstmate) (kind: ship) (since: 2026-09-05) -- [ ] del-task - Delete my card (repo: firstmate) (kind: ship) (since: 2026-09-05) (hold: captain review) (hold-kind: captain) +- [ ] keep-task - Keep me (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) +- [ ] del-task - Delete my card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) (hold: captain review) (hold-kind: captain) ## Done EOF : > "$case_dir/gh.log" @@ -495,12 +516,12 @@ pass "a deleted card holds its live task for the captain and is not treated as n held_delete_dir="$TMP_ROOT/delete-already-held" mkdir -p "$held_delete_dir/home/config" "$held_delete_dir/home/data" "$held_delete_dir/home/state" held_delete_fb=$(install_fakes "$held_delete_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$held_delete_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$held_delete_dir/home/config/helm.json" cat > "$held_delete_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] captain-held-task - Captain-held task (repo: firstmate) (kind: ship) (since: 2026-09-05) (hold: captain review) (hold-kind: captain) +- [ ] captain-held-task - Captain-held task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) (hold: captain review) (hold-kind: captain) ## Done EOF board_json "$(jq -n --argjson item "$(draft_item captain-held-item captain-held-draft captain-held-task 'Captain-held task' 'x' Queued queued-status P3 p3-priority)" '[$item]')" > "$held_delete_dir/board.json" @@ -526,7 +547,7 @@ board_json "$(jq -n \ : > "$case_dir/gh.log" run_sync "$case_dir" "$fb" --force >/dev/null 2>&1 || fail "new-card run failed" grep -F $'\tcheck\thelm-new-card:brand-new-idea\t' "$case_dir/home/state/.wake-queue" >/dev/null \ - || fail "a brand-new captain card did not wake firstmate for intake" + || fail "a brand-new captain card did not wake fixture-firstmate for intake" if grep -F 'itemId=new-item' "$case_dir/gh.log" | grep -F 'done-status' >/dev/null; then fail "a brand-new captain card was wrongly closed to Done" fi @@ -554,12 +575,12 @@ pass "a completed task's Done card is left untouched with no wake" case_dir="$TMP_ROOT/issue" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] issue-task - Backlog authoritative title (repo: firstmate) (kind: ship) (priority: 1) (since: 2026-09-05) +- [ ] issue-task - Backlog authoritative title (repo: fixture-firstmate) (kind: ship) (priority: 1) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson it "$(jq -n ' @@ -585,12 +606,12 @@ pass "a real repo issue keeps field-only sync and its own title and body" case_dir="$TMP_ROOT/prio" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] prio-task - Priority task (repo: firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) +- [ ] prio-task - Priority task (repo: fixture-firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item prio-item prio-draft prio-task 'Priority task' 'x' Queued queued-status P2 p2-priority)" '[$a]')" > "$case_dir/board.json" @@ -609,12 +630,12 @@ pass "a forced read accepts a captain board Priority edit into the owning backlo case_dir="$TMP_ROOT/prio-failure" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] failed-prio - Priority task (repo: firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) +- [ ] failed-prio - Priority task (repo: fixture-firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item failed-prio-item failed-prio-draft failed-prio 'Priority task' 'x' Queued queued-status P2 p2-priority)" '[$a]')" > "$case_dir/board.json" @@ -638,12 +659,12 @@ pass "a failed Priority write-back queues reconciliation and remains retryable" case_dir="$TMP_ROOT/new-card-forward-progress" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] fresh-task - Fresh task (repo: firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) +- [ ] fresh-task - Fresh task (repo: fixture-firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -665,12 +686,12 @@ pass "new cards retain a completed baseline after creation" case_dir="$TMP_ROOT/three-way" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## In flight -- [ ] merge-task - Merge task (repo: firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) +- [ ] merge-task - Merge task (repo: fixture-firstmate) (kind: ship) (priority: 2) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item merge-item merge-draft merge-task 'Merge task' 'original body' 'In flight' flight-status P2 p2-priority)" '[$a]')" > "$case_dir/board.json" @@ -732,7 +753,7 @@ cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Done -- [ ] merge-task - Merge task (repo: firstmate) (kind: ship) (priority: 4) (since: 2026-09-06) +- [ ] merge-task - Merge task (repo: fixture-firstmate) (kind: ship) (priority: 4) (since: 2026-09-06) EOF cp "$case_dir/converged-board.json" "$case_dir/board.json" : > "$case_dir/gh.log"; : > "$case_dir/tasks-axi.log"; : > "$case_dir/home/state/.wake-queue" @@ -759,12 +780,12 @@ pass "no-baseline cards rebuild silently" case_dir="$TMP_ROOT/debounce" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] d-task - D (repo: firstmate) (kind: ship) (since: 2026-09-05) +- [ ] d-task - D (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item d-item d-draft d-task 'D' 'x' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -777,12 +798,12 @@ pass "an unchanged fleet backlog is debounced without a GitHub call" case_dir="$TMP_ROOT/poll-acknowledgement" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] acknowledged-task - Acknowledged task (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] acknowledged-task - Acknowledged task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item ack-item ack-draft acknowledged-task 'Acknowledged task' "$acknowledged_body" Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -802,19 +823,19 @@ pass "a backlog-driven board sync acknowledges the poll signature" case_dir="$TMP_ROOT/prewrite-conflict" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] conflict-task - Backlog title (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] conflict-task - Backlog title (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF conflict_body=$(cat <<'EOF' ## Facts -- **Repo:** firstmate +- **Repo:** fixture-firstmate - **Type:** ship - produces a change and a PR - **Priority:** P3 - **Filed:** 2026-09-09 @@ -823,7 +844,7 @@ conflict_body=$(cat <<'EOF' --- -_Source of truth: `data/backlog.md` in the owning firstmate home._ +_Source of truth: `data/backlog.md` in the owning local home._ EOF ) board_json "$(jq -n --argjson a "$(draft_item conflict-item conflict-draft conflict-task 'Backlog title' "$conflict_body" Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -854,12 +875,12 @@ pass "a board/backlog conflict wakes once" case_dir="$TMP_ROOT/conflict-progress" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] conflict-progress - Original title (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] conflict-progress - Original title (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item conflict-progress-item conflict-progress-draft conflict-progress 'Original title' 'body' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -881,12 +902,12 @@ pass "independent progress does not repeat a title conflict" case_dir="$TMP_ROOT/title-body-merge" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] text-task - Original title (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] text-task - Original title (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item text-item text-draft text-task 'Original title' 'body' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -910,12 +931,12 @@ pass "body progress preserves a captain title edit" case_dir="$TMP_ROOT/partial-text-conflict" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] partial-text - Original title (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] partial-text - Original title (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item partial-item partial-draft partial-text 'Original title' 'original body' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -941,12 +962,12 @@ pass "partial text reconciliation preserves conflict acknowledgement" case_dir="$TMP_ROOT/waiting-status" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] waiting-task - Waiting task (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] waiting-task - Waiting task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item waiting-item waiting-draft waiting-task 'Waiting task' 'body' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -969,12 +990,12 @@ pass "Waiting on you yields to later completion" case_dir="$TMP_ROOT/status-conflict" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] status-task - Status task (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] status-task - Status task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item status-item status-draft status-task 'Status task' 'body' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -997,12 +1018,12 @@ pass "a status-only conflict wakes once" case_dir="$TMP_ROOT/late-prewrite-conflict" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] late-conflict-task - Backlog title (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] late-conflict-task - Backlog title (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item late-conflict-item late-conflict-draft late-conflict-task 'Backlog title' "$conflict_body" Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -1034,12 +1055,12 @@ pass "a late pre-write board conflict preserves the captain edit" case_dir="$TMP_ROOT/multi-field-card" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## In flight -- [ ] multi-task - Revised backlog title (repo: firstmate) (kind: ship) (priority: 0) (since: 2026-09-09) +- [ ] multi-task - Revised backlog title (repo: fixture-firstmate) (kind: ship) (priority: 0) (since: 2026-09-09) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -1053,7 +1074,7 @@ jq -e ' | .content.title == "Revised backlog title" and any(.fieldValues.nodes[]; .field.name == "Status" and .name == "In flight") and any(.fieldValues.nodes[]; .field.name == "Priority" and .name == "P0") - and any(.fieldValues.nodes[]; .field.name == "Project" and .name == "firstmate") + and any(.fieldValues.nodes[]; .field.name == "Project" and .name == "fixture-firstmate") and any(.fieldValues.nodes[]; .field.name == "Kind" and .name == "ship") ' "$case_dir/board-state.json" >/dev/null \ || fail "the multi-field card did not land every change: $(jq -c '.data.user.projectV2.items.nodes[] | select(.id == "created-item")' "$case_dir/board-state.json")" @@ -1081,11 +1102,11 @@ scale_body() { # <task-id> <Pn> <filed> [note-line...] local id=$1 pn=$2 filed=$3 shift 3 # shellcheck disable=SC2016 # backticks are card-body literals, not expansions. - printf '`%s`\n\n## Facts\n\n- **Repo:** firstmate\n- **Type:** ship - produces a change and a PR\n- **Priority:** %s\n- **Filed:** %s\n\n## Notes\n\n' "$id" "$pn" "$filed" + printf '`%s`\n\n## Facts\n\n- **Repo:** fixture-firstmate\n- **Type:** ship - produces a change and a PR\n- **Priority:** %s\n- **Filed:** %s\n\n## Notes\n\n' "$id" "$pn" "$filed" local line for line in "$@"; do printf '%s\n' "$line"; done # shellcheck disable=SC2016 # backticks are card-body literals, not expansions. - printf '\n---\n_Source of truth: `data/backlog.md` in the owning firstmate home._' + printf '\n---\n_Source of truth: `data/backlog.md` in the owning local home._' } scale_item() { # <task-id> <title> <body> <status-name> <status-id> <Pn> <pn-id> @@ -1094,7 +1115,7 @@ scale_item() { # <task-id> <title> <body> <status-name> <status-id> <Pn> <pn-id fieldValues:{nodes:[ {__typename:"ProjectV2ItemFieldSingleSelectValue",field:{name:"Status"},name:$so,optionId:$si}, {__typename:"ProjectV2ItemFieldSingleSelectValue",field:{name:"Priority"},name:$pn,optionId:$pi}, - {__typename:"ProjectV2ItemFieldSingleSelectValue",field:{name:"Project"},name:"firstmate",optionId:"firstmate-project"}, + {__typename:"ProjectV2ItemFieldSingleSelectValue",field:{name:"Project"},name:"fixture-firstmate",optionId:"fixture-firstmate-project"}, {__typename:"ProjectV2ItemFieldSingleSelectValue",field:{name:"Kind"},name:"ship",optionId:"ship-kind"} ]}}' } @@ -1104,16 +1125,16 @@ write_scale_fixtures() { # <case-dir> { printf '# Backlog\n\n## In flight\n' for i in 1 2 3 4 5; do - printf -- '- [ ] scale-flight-%s - Flight task %s (repo: firstmate) (kind: ship) (priority: 1) (since: 2026-09-01)\n' "$i" "$i" + printf -- '- [ ] scale-flight-%s - Flight task %s (repo: fixture-firstmate) (kind: ship) (priority: 1) (since: 2026-09-01)\n' "$i" "$i" done printf '## Queued\n' for i in $(seq 1 85); do - printf -- '- [ ] scale-queued-%s - Queued task %s (repo: firstmate) (kind: ship) (priority: 3) (since: 2026-09-02)\n' "$i" "$i" + printf -- '- [ ] scale-queued-%s - Queued task %s (repo: fixture-firstmate) (kind: ship) (priority: 3) (since: 2026-09-02)\n' "$i" "$i" printf ' note for task %s\n' "$i" done printf '## Done\n' for i in $(seq 1 10); do - printf -- '- [x] scale-done-%s - Done task %s (repo: firstmate) (kind: ship) (done: 2026-09-10)\n' "$i" "$i" + printf -- '- [x] scale-done-%s - Done task %s (repo: fixture-firstmate) (kind: ship) (done: 2026-09-10)\n' "$i" "$i" done } > "$dir/home/data/backlog.md" items="$dir/items.jsonl" @@ -1149,7 +1170,7 @@ write_scale_fixtures() { # <case-dir> case_dir="$TMP_ROOT/fleet-scale" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" write_scale_fixtures "$case_dir" [ "$(jq '.data.user.projectV2.items.nodes | length' "$case_dir/board.json")" -eq 110 ] \ || fail "scale board fixture is not 110 cards" @@ -1209,14 +1230,14 @@ pass "a fleet-scale run finishes well inside the deadline and is idempotent" case_dir="$TMP_ROOT/resume-after-cutoff" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] cut-a - First card (repo: firstmate) (kind: ship) (since: 2026-09-09) -- [ ] cut-b - Second card (repo: firstmate) (kind: ship) (since: 2026-09-09) -- [ ] cut-c - Third card (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] cut-a - First card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) +- [ ] cut-b - Second card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) +- [ ] cut-c - Third card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -1251,12 +1272,12 @@ pass "a run cut off after its first creation resumes from the recorded card" case_dir="$TMP_ROOT/resume-after-kill" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" "$case_dir/tmp" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] kill-a - Killed run card (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] kill-a - Killed run card (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -1285,12 +1306,12 @@ pass "a run killed mid-write keeps its created card and resumes cleanly" case_dir="$TMP_ROOT/bounded-mutation" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## In flight -- [ ] bounded-task - Bounded mutation (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] bounded-task - Bounded mutation (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item bounded-item bounded-draft bounded-task 'Bounded mutation' 'x' Queued queued-status P3 p3-priority)" '[$a]')" > "$case_dir/board.json" @@ -1312,12 +1333,12 @@ pass "a delayed board mutation is bounded and remains retryable" case_dir="$TMP_ROOT/watcher-trigger" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] watcher-task - Reaches the board automatically (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] watcher-task - Reaches the board automatically (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -1339,13 +1360,18 @@ FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_BOOTSTRAP_NETWORK=skip \ || fail "bootstrap did not install the Helm board poll check" [ -s "$case_dir/home/state/helm-board.check-trust" ] \ || fail "bootstrap did not authenticate the Helm board poll check" +[ -x "$case_dir/home/state/helm-reconcile.check.sh" ] \ + || fail "bootstrap did not install the Helm reconciliation check" +[ -s "$case_dir/home/state/helm-reconcile.check-trust" ] \ + || fail "bootstrap did not authenticate the Helm reconciliation check" pass "bootstrap arms the authenticated Helm watcher checks" rm -f "$case_dir/home/config/helm.json" override_state="$case_dir/override-state" mkdir -p "$override_state" mv "$case_dir/home/state/helm-sync.check.sh" "$case_dir/home/state/helm-sync.check-trust" \ - "$case_dir/home/state/helm-board.check.sh" "$case_dir/home/state/helm-board.check-trust" "$override_state/" + "$case_dir/home/state/helm-board.check.sh" "$case_dir/home/state/helm-board.check-trust" \ + "$case_dir/home/state/helm-reconcile.check.sh" "$case_dir/home/state/helm-reconcile.check-trust" "$override_state/" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$override_state" FM_BOOTSTRAP_NETWORK=skip \ PATH="$fb:$PATH" "$ROOT/bin/fm-bootstrap.sh" >/dev/null 2>&1 \ || fail "bootstrap could not retire overridden Helm watcher checks" @@ -1353,12 +1379,14 @@ FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$override_s || fail "bootstrap did not retire the overridden Helm sync check" [ ! -e "$override_state/helm-board.check.sh" ] && [ ! -e "$override_state/helm-board.check-trust" ] \ || fail "bootstrap did not retire the overridden Helm board check" +[ ! -e "$override_state/helm-reconcile.check.sh" ] && [ ! -e "$override_state/helm-reconcile.check-trust" ] \ + || fail "bootstrap did not retire the overridden Helm reconciliation check" pass "bootstrap retires Helm checks from the overridden state directory" case_dir="$TMP_ROOT/watcher-diagnostic" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog @@ -1381,7 +1409,7 @@ pass "watcher adapter retains unsupported projects in the other bucket without w case_dir="$TMP_ROOT/unsupported-repo-debounce" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog @@ -1408,7 +1436,7 @@ cat > "$case_dir/home/data/backlog.md" <<'EOF' ## Queued - [ ] debounce-task - Must not disappear (repo: repo-prefix repo-suffix) (kind: ship) (since: 2026-09-09) -- [ ] unrelated-task - Elsewhere entirely (repo: firstmate) (kind: ship) (since: 2026-09-10) +- [ ] unrelated-task - Elsewhere entirely (repo: fixture-firstmate) (kind: ship) (since: 2026-09-10) ## Done EOF out=$(run_sync "$case_dir" "$fb" 2>&1) || fail "unrelated-task replan exited nonzero: $out" @@ -1425,7 +1453,7 @@ cat > "$case_dir/home/data/backlog.md" <<'EOF' ## Queued - [ ] debounce-task - Must not disappear (repo: another-unrecognised-project) (kind: ship) (since: 2026-09-09) -- [ ] unrelated-task - Elsewhere entirely (repo: firstmate) (kind: ship) (since: 2026-09-10) +- [ ] unrelated-task - Elsewhere entirely (repo: fixture-firstmate) (kind: ship) (since: 2026-09-10) ## Done EOF out=$(run_sync "$case_dir" "$fb" 2>&1) || fail "changed repo: value run exited nonzero: $out" @@ -1442,12 +1470,12 @@ pass "a task's repo: value changing to a new unsupported value re-fires the note case_dir="$TMP_ROOT/watcher-lock-contention" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") -printf '{"owner":"geojitsu","number":2}\n' > "$case_dir/home/config/helm.json" +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" cat > "$case_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] lock-task - Reaches the board while another sync holds the lock (repo: firstmate) (kind: ship) (since: 2026-09-09) +- [ ] lock-task - Reaches the board while another sync holds the lock (repo: fixture-firstmate) (kind: ship) (since: 2026-09-09) ## Done EOF board_json '[]' > "$case_dir/board.json" @@ -1473,12 +1501,12 @@ for mode in no-config noauth scope network; do cd_dir="$TMP_ROOT/fail-$mode" mkdir -p "$cd_dir/home/config" "$cd_dir/home/data" "$cd_dir/home/state" fbx=$(install_fakes "$cd_dir") - [ "$mode" = no-config ] || printf '{"owner":"geojitsu","number":2}\n' > "$cd_dir/home/config/helm.json" + [ "$mode" = no-config ] || printf '{"owner":"fixture-owner","number":999}\n' > "$cd_dir/home/config/helm.json" cat > "$cd_dir/home/data/backlog.md" <<'EOF' # Backlog ## Queued -- [ ] f-task - F (repo: firstmate) (kind: ship) (since: 2026-09-05) +- [ ] f-task - F (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) ## Done EOF board_json "$(jq -n --argjson a "$(draft_item f-item f-draft f-task 'F' 'x' Queued queued-status P3 p3-priority)" '[$a]')" > "$cd_dir/board.json" From e895842b26c752c9f5551d6b24730c4c753fe737 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 03:41:53 +0000 Subject: [PATCH 02/23] fix(helm): provision a board's missing Project option instead of dropping the card A registered-but-unmapped project's card landed with its own name in the default board's Project bucket, but the sync only checked that the option already existed there - it never created it, so the card silently never landed. Linking a second project onto an already-linked board hit the same gap: ensure_schema refused instead of adding the option (spec decision 6 makes sharing a board across projects normal, not an error). Both paths now call a shared fm_helm_ensure_field_options helper that reads a GitHub Project field's current options and adds whatever is missing, carrying every existing option's id/color/description forward so cards already set to it are undisturbed. The prior ensure_schema mutation was also broken against the real GitHub API (wrong GraphQL variable type, an invalid union selection) and never actually provisioned anything; the new helper is dry-validated against GitHub's live schema. --- bin/fm-helm-lib.sh | 87 +++++++++++++++ bin/fm-helm-project-map.sh | 48 +++------ bin/fm-helm-sync.sh | 62 ++++++++++- docs/1.architecture/helm-project-routing.md | 7 ++ tests/fm-helm-project-map.test.sh | 111 +++++++++++++++++++- tests/fm-helm-sync.test.sh | 67 ++++++++++++ 6 files changed, 340 insertions(+), 42 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index 29f32cf7985..2bb1be5f06e 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -138,6 +138,93 @@ fm_helm_project_names() { done | awk '!seen[$0]++' } +# fm_helm_ensure_field_options <run-fn> <project-id> <field-name> <required-csv> +# +# Ensures GitHub Project <project-id>'s single-select field <field-name> has +# every option named in the comma-separated <required-csv>. Creates the field +# with the full required set when it does not exist yet; otherwise adds only +# the options that are missing, carrying every already-present option's id, +# color, and description forward unchanged so cards already set to it, and +# its own display color, are undisturbed. Idempotent: a field that already +# has every required option makes no GitHub call. +# +# <run-fn> is the name of a function the caller defines that runs +# `gh-axi api graphql "$@"` (or an equivalently bounded wrapper) and prints +# the raw JSON response; the singleSelectOptions GraphQL input requires a +# non-null name/color/description per option, so every entry this function +# sends carries all three even when only carrying an existing option forward. +fm_helm_ensure_field_options() { + local run_fn=$1 project_id=$2 field=$3 required=$4 + local detail response field_id existing name missing_csv + local -a have=() missing=() names=() ids=() colors=() descs=() args=() + # shellcheck disable=SC2016 # GraphQL variables must remain literal. + detail='query($projectId:ID!){node(id:$projectId){... on ProjectV2{fields(first:100){nodes{__typename ... on ProjectV2FieldCommon{id name} ... on ProjectV2SingleSelectField{id name options{id name color description}}}}}}}' + response=$("$run_fn" --field "query=$detail" --field "projectId=$project_id") || return 1 + jq -e '(.errors // []) | length == 0' <<<"$response" >/dev/null 2>&1 || return 1 + field_id=$(jq -r --arg f "$field" \ + '[.data.node.fields.nodes[]? | select(.name == $f and .__typename == "ProjectV2SingleSelectField") | .id][0] // empty' \ + <<<"$response") + existing=$(jq -c --arg f "$field" \ + '[.data.node.fields.nodes[]? | select(.name == $f and .__typename == "ProjectV2SingleSelectField") | .options[]?]' \ + <<<"$response") + while IFS= read -r name; do + [ -n "$name" ] || continue + have+=("$name") + done < <(jq -r '.[].name' <<<"$existing") + while IFS= read -r name; do + [ -n "$name" ] || continue + if [ "${#have[@]}" -gt 0 ] && printf '%s\n' "${have[@]}" | grep -qxF "$name"; then + continue + fi + missing+=("$name") + done < <(printf '%s\n' "$required" | tr ',' '\n') + [ "${#missing[@]}" -gt 0 ] || return 0 + if [ -z "$field_id" ]; then + # The field does not exist yet: create it with the full required set. + while IFS= read -r name; do + [ -n "$name" ] || continue + names+=("$name"); ids+=(""); colors+=("GRAY"); descs+=("") + done < <(printf '%s\n' "$required" | tr ',' '\n') + else + # The field exists: carry every current option forward, then append the + # gap. The unit separator (not a tab) keeps `read` from collapsing an + # empty color or description field, which tab does since it is + # whitespace-class in IFS word splitting. + while IFS=$'\037' read -r name color desc id; do + [ -n "$name" ] || continue + names+=("$name"); ids+=("$id"); colors+=("${color:-GRAY}"); descs+=("$desc") + done < <(jq -r '.[] | [(.name // ""), (.color // ""), (.description // ""), (.id // "")] | join("")' <<<"$existing") + for name in "${missing[@]}"; do + names+=("$name"); ids+=(""); colors+=("GRAY"); descs+=("") + done + fi + local vars='' literals='' i + for ((i = 0; i < ${#names[@]}; i++)); do + vars="$vars, \$name$i:String!, \$color$i:ProjectV2SingleSelectFieldOptionColor!, \$desc$i:String!" + args+=(--field "name$i=${names[$i]}" --field "color$i=${colors[$i]}" --field "desc$i=${descs[$i]}") + if [ -n "${ids[$i]}" ]; then + vars="$vars, \$id$i:String!" + args+=(--field "id$i=${ids[$i]}") + literals="${literals:+$literals,}{id:\$id$i,name:\$name$i,color:\$color$i,description:\$desc$i}" + else + literals="${literals:+$literals,}{name:\$name$i,color:\$color$i,description:\$desc$i}" + fi + done + local query + if [ -z "$field_id" ]; then + query="mutation(\$projectId:ID!, \$fname:String!$vars){createProjectV2Field(input:{projectId:\$projectId,dataType:SINGLE_SELECT,name:\$fname,singleSelectOptions:[$literals]}){projectV2Field{... on ProjectV2SingleSelectField{options{name}}}}}" + response=$("$run_fn" --field "query=$query" --field "projectId=$project_id" --field "fname=$field" "${args[@]}") || return 1 + else + query="mutation(\$fieldId:ID!$vars){updateProjectV2Field(input:{fieldId:\$fieldId,singleSelectOptions:[$literals]}){projectV2Field{... on ProjectV2SingleSelectField{options{name}}}}}" + response=$("$run_fn" --field "query=$query" --field "fieldId=$field_id" "${args[@]}") || return 1 + fi + jq -e '(.errors // []) | length == 0' <<<"$response" >/dev/null 2>&1 || return 1 + missing_csv=$(jq -r '(.data.createProjectV2Field // .data.updateProjectV2Field).projectV2Field.options[]?.name' <<<"$response" 2>/dev/null) + for name in "${missing[@]}"; do + printf '%s\n' "$missing_csv" | grep -qxF "$name" || return 1 + done +} + # fm_helm_mapping_orphan_id <project-name> - derive the stable captain-hold id # used for a broken local-project or GitHub-board mapping. fm_helm_mapping_orphan_id() { diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index a619a6ba88b..2c8e8083892 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -187,43 +187,25 @@ create_or_reuse_board() { # <owner> <title> [existing-owner/number] esac } -field_has_options() { # <field-list> <field-name> <comma-separated-options> - local listing=$1 field=$2 options=$3 option - printf '%s\n' "$listing" | awk -v wanted="$field" ' - $1 == "name:" && substr($0, index($0, $2)) == wanted { found = 1 } - END { exit(found ? 0 : 1) }' >/dev/null - while IFS= read -r option; do - [ -n "$option" ] || continue - printf '%s\n' "$listing" | awk -v wanted="$field" -v option="$option" ' - $1 == "name:" { found = (substr($0, index($0, $2)) == wanted) } - found && $1 == "options:" { value = $0; if (index(value, option ":") > 0) ok=1; found=0 } - END { exit(ok ? 0 : 1) }' || return 1 - done < <(printf '%s' "$options" | tr ',' '\n') +# helm_graphql_call <field-args...> - the gh-axi invocation +# fm_helm_ensure_field_options uses to read and provision Helm field schema. +helm_graphql_call() { + gh-axi api graphql "$@" } +# ensure_schema provisions every field Helm's cards need on the board this +# script is currently linking or moving to ($BOARD_ID/$BOARD_OWNER/ +# $BOARD_NUMBER), including the Project option for $PROJECT_SCHEMA_OPTION. +# A board a second project links onto already has a Project field with other +# projects' options; fm_helm_ensure_field_options adds the missing option +# there without disturbing the ones already in use (spec decision 6: sharing +# one board across projects is normal, not an error to refuse). ensure_schema() { - local listing field options query option - listing=$(gh-axi project field-list "$BOARD_NUMBER" --owner "$BOARD_OWNER" --limit 100 2>&1) \ - || fail "could not read Helm fields from $BOARD_OWNER/$BOARD_NUMBER" + local field options + [ -n "$BOARD_ID" ] || fail "GitHub Project $BOARD_OWNER/$BOARD_NUMBER has no id" while IFS='|' read -r field options; do - field_has_options "$listing" "$field" "$options" && continue - if printf '%s\n' "$listing" | awk -v wanted="$field" '$1 == "name:" && substr($0, index($0, $2)) == wanted { found=1 } END { exit(found ? 0 : 1) }'; then - fail "GitHub Project $BOARD_OWNER/$BOARD_NUMBER has an incomplete $field field; add the missing options before linking" - fi - [ -n "$BOARD_ID" ] || fail "GitHub Project $BOARD_OWNER/$BOARD_NUMBER has no id" - # shellcheck disable=SC2016 # GraphQL variables must remain literal. - query='mutation($projectId:ID!,$name:String!,$options:[String!]!){createProjectV2Field(input:{projectId:$projectId,dataType:SINGLE_SELECT,name:$name,singleSelectOptions:$options}){projectV2Field{id}}}' - set -- - while IFS= read -r option; do - [ -n "$option" ] || continue - set -- "$@" --field "options[]=$option" - done < <(printf '%s' "$options" | tr ',' '\n') - gh-axi api graphql --field "query=$query" --field "projectId=$BOARD_ID" --field "name=$field" "$@" >/dev/null 2>&1 \ - || fail "could not create Helm field $field on $BOARD_OWNER/$BOARD_NUMBER" - listing=$(gh-axi project field-list "$BOARD_NUMBER" --owner "$BOARD_OWNER" --limit 100 2>&1) \ - || fail "could not verify Helm field $field on $BOARD_OWNER/$BOARD_NUMBER" - field_has_options "$listing" "$field" "$options" \ - || fail "Helm field $field on $BOARD_OWNER/$BOARD_NUMBER is still incomplete" + fm_helm_ensure_field_options helm_graphql_call "$BOARD_ID" "$field" "$options" \ + || fail "could not provision Helm field $field on $BOARD_OWNER/$BOARD_NUMBER" done < <( printf '%s\n' \ 'Status|Queued,In flight,Waiting on you,Done' \ diff --git a/bin/fm-helm-sync.sh b/bin/fm-helm-sync.sh index 8ed6cb4b47e..86ab691e091 100755 --- a/bin/fm-helm-sync.sh +++ b/bin/fm-helm-sync.sh @@ -503,6 +503,26 @@ graphql_mutation() { jq -e '(.errors // []) | length == 0' "$TMP_RESPONSE" >/dev/null 2>&1 } +# helm_graphql_call <field-args...> - the bounded `gh api graphql` call +# fm_helm_ensure_field_options uses to provision a missing Project option on +# the current board. Requires SYNC_DEADLINE to already be set for that board. +helm_graphql_call() { + local remaining + remaining=$(( SYNC_DEADLINE - $(date +%s) )) + [ "$remaining" -gt 0 ] || return 1 + run_gh_bounded "$remaining" gh api graphql "$@" +} + +# missing_project_options <board-json> <desired-json> - print the comma +# joined desired Project names this board's Project field has no option for. +missing_project_options() { + jq -r --slurpfile records "$2" ' + ($records[0] | map(.desired.project) | unique) as $wanted + | [.data.user.projectV2.fields.nodes[]? | select(.name == "Project" and .__typename == "ProjectV2SingleSelectField") | .options[]?.name] as $have + | ($wanted - $have) | join(",") + ' "$1" +} + US=$'\037' RS=$'\036' SPLIT=() @@ -899,7 +919,7 @@ publish_board_failure_wakes() { process_board() { local owner=$1 number=$2 key="$1/$2" board_file group_desired - local board_now board_budget schema_ok + local board_now board_budget schema_ok missing_projects BOARD_WRITE_FAILURE=0 BOARD_OWNER=$owner BOARD_NUMBER=$number @@ -940,7 +960,13 @@ process_board() { or (($historical | index($record.id)) != null)))' \ "$DESIRED_JSON" >"$group_desired" \ || helm_fail_open "could not group desired Helm cards" - schema_ok=$(jq -e --argjson records "$(cat "$group_desired")" --arg dispatch "$DISPATCH_STATUS" ' + # One jq pass computes both the base-schema verdict and any Project options + # a registered-but-unmapped project's card needs (fm_helm_desired_program's + # project_of puts its own name in the Project bucket, and the board's + # Project field may not have that option yet) so the per-board hot path + # stays a single jq call, same as before this check grew a second part. + schema_ok=false missing_projects= + IFS=$'\t' read -r schema_ok missing_projects < <(jq -r --argjson records "$(cat "$group_desired")" --arg dispatch "$DISPATCH_STATUS" ' .data.user.projectV2.fields.nodes as $fields | (def has_option($field; $name): any($fields[]; .name == $field and .__typename == "ProjectV2SingleSelectField" and any(.options[]?; .name == $name)); any($fields[]; .name == "Status" and .__typename == "ProjectV2SingleSelectField") @@ -949,14 +975,40 @@ process_board() { and any($fields[]; .name == "Priority" and .__typename == "ProjectV2SingleSelectField") and all((["Queued", "In flight", "Waiting on you", "Done", $dispatch] | unique)[]; has_option("Status"; .)) and all(["P0", "P1", "P2", "P3", "P4"][]; has_option("Priority"; .)) - and all(["ship", "investigation", "decision"][]; has_option("Kind"; .)) - and all($records[]; has_option("Project"; .desired.project))) - ' "$BOARD_JSON" 2>/dev/null) || schema_ok= + and all(["ship", "investigation", "decision"][]; has_option("Kind"; .))) as $ok + | ([$fields[] | select(.name == "Project" and .__typename == "ProjectV2SingleSelectField") | .options[]?.name]) as $have + | [$ok, (($records | map(.desired.project) | unique) - $have | join(","))] | @tsv + ' "$BOARD_JSON" 2>/dev/null) if [ "$schema_ok" != true ]; then board_failure "$key" "required Helm fields or options are unavailable" publish_progress || helm_fail_open "could not publish Helm board acknowledgement" return 0 fi + if [ -n "$missing_projects" ]; then + if ! fm_helm_ensure_field_options helm_graphql_call "$PROJECT_ID" Project "$missing_projects"; then + board_failure "$key" "could not provision the Project field option for $missing_projects" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 + fi + BOARD_READ_SEQUENCE=$((BOARD_READ_SEQUENCE + 1)) + if ! read_board "$owner" "$number" "$board_file"; then + board_failure "$key" "GitHub project re-read after provisioning failed" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 + fi + BOARD_JSON="$board_file" + PROJECT_ID=$(jq -r '.data.user.projectV2.id' "$BOARD_JSON") + STATUS_FIELD_ID=$(field_id Status) + PROJECT_FIELD_ID=$(field_id Project) + KIND_FIELD_ID=$(field_id Kind) + PRIORITY_FIELD_ID=$(field_id Priority) + DISPATCH_OPTION_ID=$(option_id Status "$DISPATCH_STATUS") + if [ -n "$(missing_project_options "$BOARD_JSON" "$group_desired")" ]; then + board_failure "$key" "Project field option for $missing_projects is still missing after provisioning" + publish_progress || helm_fail_open "could not publish Helm board acknowledgement" + return 0 + fi + fi PLAN="$TMP_DIR/plan-$BOARD_INDEX.nul" jq -j --slurpfile desired "$group_desired" \ --rawfile cards "$OLD_CARDS" --rawfile deleted "$OLD_DELETED" --rawfile markers "$OLD_MARKERS" \ diff --git a/docs/1.architecture/helm-project-routing.md b/docs/1.architecture/helm-project-routing.md index d6170f4c455..ccdb466a42d 100644 --- a/docs/1.architecture/helm-project-routing.md +++ b/docs/1.architecture/helm-project-routing.md @@ -47,3 +47,10 @@ move uses `state/helm-moves.tsv` and adds or creates the destination card before deleting the source card, so a crash leaves a resumable phase rather than an ambiguous relocation. Routing uses project identity from `data/projects.md`; it does not use checkout paths. + +When a board's Project field is missing an option a routed project needs - +whether that is the shared default board gaining a newly registered project or +a board another project already linked - `fm_helm_ensure_field_options` +(`bin/fm-helm-lib.sh`) adds the missing option in place, carrying every +existing option's id, color, and description forward unchanged so cards +already set to it are undisturbed. diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index f6dea3319c2..9c5eb9abd3c 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -186,10 +186,24 @@ if [ "\${1:-}" = api ]; then ]}}}}' ;; *'fields(first:100)'* ) jq -n '{data:{node:{fields:{nodes:[ - {__typename:"ProjectV2SingleSelectField",id:"dest-status-field",name:"Status",options:[{id:"dest-queued-status",name:"Queued"}]}, - {__typename:"ProjectV2SingleSelectField",id:"dest-priority-field",name:"Priority",options:[{id:"dest-p3-priority",name:"P3"}]}, - {__typename:"ProjectV2SingleSelectField",id:"dest-kind-field",name:"Kind",options:[{id:"dest-ship-kind",name:"ship"}]}, - {__typename:"ProjectV2SingleSelectField",id:"dest-project-field",name:"Project",options:[{id:"dest-alpha-project",name:"alpha"}]} + {__typename:"ProjectV2SingleSelectField",id:"dest-status-field",name:"Status",options:[ + {id:"dest-queued-status",name:"Queued",color:"GRAY",description:""}, + {id:"dest-flight-status",name:"In flight",color:"GRAY",description:""}, + {id:"dest-waiting-status",name:"Waiting on you",color:"GRAY",description:""}, + {id:"dest-done-status",name:"Done",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-priority-field",name:"Priority",options:[ + {id:"dest-p0-priority",name:"P0",color:"GRAY",description:""}, + {id:"dest-p1-priority",name:"P1",color:"GRAY",description:""}, + {id:"dest-p2-priority",name:"P2",color:"GRAY",description:""}, + {id:"dest-p3-priority",name:"P3",color:"GRAY",description:""}, + {id:"dest-p4-priority",name:"P4",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-kind-field",name:"Kind",options:[ + {id:"dest-ship-kind",name:"ship",color:"GRAY",description:""}, + {id:"dest-investigation-kind",name:"investigation",color:"GRAY",description:""}, + {id:"dest-decision-kind",name:"decision",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"dest-project-field",name:"Project",options:[ + {id:"dest-other-project",name:"other",color:"GRAY",description:""}, + {id:"dest-alpha-project",name:"alpha",color:"GRAY",description:""}]} ]}}}}' ;; *addProjectV2DraftIssue*) jq -n '{data:{addProjectV2DraftIssue:{projectItem:{id:"moved-item"}}}}' ;; *) jq -n '{data:{}}' ;; @@ -271,6 +285,95 @@ grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item || fail "move ledger did not retain its complete add/delete history" pass "confirmed moves preserve card history in the resumable ledger" +# Linking a second project onto a board another project already linked must +# add the missing Project option, not refuse the board as "incomplete" +# (spec decision 6: several mapping entries sharing one board is normal). +case_dir="$TMP_ROOT/shared-board-link" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,title:"Shared Board",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(fm_fakebin "$case_dir") +cat >"$fb/gh-axi" <<'SH' +#!/usr/bin/env bash +set -u +printf 'fake-gh-axi ' >>"$FM_HELM_GH_AXI_LOG" +printf '%s' "$*" | tr '\n' ' ' >>"$FM_HELM_GH_AXI_LOG" +printf '\n' >>"$FM_HELM_GH_AXI_LOG" +if [ "${1:-}" = project ] && [ "${2:-}" = view ]; then + printf '%s\n' 'id: shared-board' 'title: Shared Board' 'url: https://github.com/orgs/fixture-org/projects/998' + exit 0 +fi +if [ "${1:-}" = project ] && [ "${2:-}" = item-list ]; then exit 0; fi +if [ "${1:-}" = api ]; then + case "$*" in + *'node(id:$projectId)'*) + jq -n '{data:{node:{fields:{nodes:[ + {__typename:"ProjectV2SingleSelectField",id:"status-field",name:"Status",options:[ + {id:"queued-status",name:"Queued",color:"GRAY",description:""}, + {id:"flight-status",name:"In flight",color:"GRAY",description:""}, + {id:"waiting-status",name:"Waiting on you",color:"GRAY",description:""}, + {id:"done-status",name:"Done",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"priority-field",name:"Priority",options:[ + {id:"p0-priority",name:"P0",color:"GRAY",description:""},{id:"p1-priority",name:"P1",color:"GRAY",description:""}, + {id:"p2-priority",name:"P2",color:"GRAY",description:""},{id:"p3-priority",name:"P3",color:"GRAY",description:""}, + {id:"p4-priority",name:"P4",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"kind-field",name:"Kind",options:[ + {id:"ship-kind",name:"ship",color:"GRAY",description:""},{id:"investigation-kind",name:"investigation",color:"GRAY",description:""}, + {id:"decision-kind",name:"decision",color:"GRAY",description:""}]}, + {__typename:"ProjectV2SingleSelectField",id:"project-field",name:"Project",options:[ + {id:"other-project",name:"other",color:"GRAY",description:""}, + {id:"alpha-project",name:"alpha",color:"GRAY",description:""}]} + ]}}}}' ;; + *createProjectV2Field*|*updateProjectV2Field*) + declare -A opt_name=() + for arg in "$@"; do + case "$arg" in + name[0-9]*=*) idx=${arg%%=*}; idx=${idx#name}; opt_name[$idx]=${arg#*=} ;; + esac + done + options='[]' + for idx in "${!opt_name[@]}"; do + options=$(jq -c --arg name "${opt_name[$idx]}" '. + [{name:$name}]' <<<"$options") + done + jq -n --argjson options "$options" '{data:{updateProjectV2Field:{projectV2Field:{options:$options}}}}' ;; + *) jq -n '{data:{}}' ;; + esac + exit 0 +fi +exit 1 +SH +cat >"$fb/gh" <<'SH' +#!/usr/bin/env bash +set -u +printf 'fake-gh ' >>"$FM_HELM_GH_LOG" +printf '%s' "$*" | tr '\n' ' ' >>"$FM_HELM_GH_LOG" +printf '\n' >>"$FM_HELM_GH_LOG" +if [ "${1:-}" = auth ] && [ "${2:-}" = status ]; then + printf "%s\n" "Token scopes: 'project', 'repo'" + exit 0 +fi +if [ "${1:-}" = api ]; then cat "$FM_HELM_BOARD_JSON"; exit 0; fi +exit 1 +SH +chmod +x "$fb/gh-axi" "$fb/gh" +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +: >"$case_dir/gh-axi.log"; : >"$case_dir/gh.log" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" \ + FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" link beta --existing fixture-org/998 >/dev/null 2>&1 \ + || fail "linking a second project onto an already-linked board exited nonzero" +grep -F 'updateProjectV2Field' "$case_dir/gh-axi.log" >/dev/null \ + || fail "link did not provision the missing Project option on the shared board" +grep -F 'name0=other' "$case_dir/gh-axi.log" >/dev/null \ + || fail "provisioning the shared board's Project field dropped its existing 'other' option" +jq -e '.projects.alpha.number == 998 and .projects.beta.number == 998 and .projects.beta.state == "active"' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "a second project did not join the first project's board in the mapping" +pass "linking a second project onto an already-linked board provisions its option instead of refusing" + # Reconciliation covers two sides of a broken mapping through the existing # captain-hold path: a board that disappeared and a local project that left the # registry. It also refreshes title drift on another board in the same pass. diff --git a/tests/fm-helm-sync.test.sh b/tests/fm-helm-sync.test.sh index 6cba9d41688..523c55284cd 100644 --- a/tests/fm-helm-sync.test.sh +++ b/tests/fm-helm-sync.test.sh @@ -95,6 +95,36 @@ if [ "${1:-}" = api ]; then fi [ -z "${FM_FAKE_GH_LATENCY:-}" ] || sleep "$FM_FAKE_GH_LATENCY" case "$*" in + *'node(id:$projectId)'*) + jq '{data:{node:{fields:{nodes:.data.user.projectV2.fields.nodes}}}}' "$FM_FAKE_BOARD_STATE" ;; + *createProjectV2Field*|*updateProjectV2Field*) + declare -A opt_name=() opt_color=() + fname=; field_id_arg= + for arg in "$@"; do + case "$arg" in + fname=*) fname=${arg#fname=} ;; + fieldId=*) field_id_arg=${arg#fieldId=} ;; + name[0-9]*=*) idx=${arg%%=*}; idx=${idx#name}; opt_name[$idx]=${arg#*=} ;; + color[0-9]*=*) idx=${arg%%=*}; idx=${idx#color}; opt_color[$idx]=${arg#*=} ;; + esac + done + options_json='[]' + for idx in "${!opt_name[@]}"; do + name=${opt_name[$idx]}; color=${opt_color[$idx]:-GRAY} + options_json=$(jq -c --arg id "${name}-project" --arg name "$name" --arg color "$color" \ + '. + [{id:$id,name:$name,color:$color,description:""}]' <<<"$options_json") + done + if [ -n "$field_id_arg" ]; then + jq --arg fid "$field_id_arg" --argjson options "$options_json" \ + '.data.user.projectV2.fields.nodes |= map(if .id == $fid then .options = $options else . end)' \ + "$FM_FAKE_BOARD_STATE" > "$FM_FAKE_BOARD_STATE.next" && mv "$FM_FAKE_BOARD_STATE.next" "$FM_FAKE_BOARD_STATE" + else + jq --arg name "$fname" --argjson options "$options_json" \ + '.data.user.projectV2.fields.nodes += [{__typename:"ProjectV2SingleSelectField",id:($name+"-field"),name:$name,options:$options}]' \ + "$FM_FAKE_BOARD_STATE" > "$FM_FAKE_BOARD_STATE.next" && mv "$FM_FAKE_BOARD_STATE.next" "$FM_FAKE_BOARD_STATE" + fi + jq -n --argjson options "$options_json" \ + '{data:{updateProjectV2Field:{projectV2Field:{options:$options}},createProjectV2Field:{projectV2Field:{options:$options}}}}' ;; *addProjectV2DraftIssue*) for arg in "$@"; do case "$arg" in @@ -1383,6 +1413,43 @@ FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$override_s || fail "bootstrap did not retire the overridden Helm reconciliation check" pass "bootstrap retires Helm checks from the overridden state directory" +# A project registered after the default board's Project field was last set +# up has no option there yet; the sync must provision it instead of silently +# never creating the card (bin/fm-helm-lib.sh's fm_helm_ensure_field_options). +case_dir="$TMP_ROOT/unmapped-project-provisioning" +mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" +fb=$(install_fakes "$case_dir") +printf '{"owner":"fixture-owner","number":999}\n' > "$case_dir/home/config/helm.json" +cat > "$case_dir/home/data/projects.md" <<'EOF' +# Projects + +- gamma [no-mistakes] - test project (added 2026-09-01) +EOF +cat > "$case_dir/home/data/backlog.md" <<'EOF' +# Backlog + +## Queued +- [ ] gamma-task - New project card (repo: gamma) (kind: ship) (since: 2026-09-09) +## Done +EOF +board_json '[]' > "$case_dir/board.json" +: > "$case_dir/gh.log"; : > "$case_dir/tasks-axi.log" +out=$(run_sync "$case_dir" "$fb") || fail "sync with an unmapped registered project exited nonzero: $out" +assert_contains "$out" "fm-helm-sync: synchronized" "sync did not finish after provisioning: $out" +grep -F 'updateProjectV2Field' "$case_dir/gh.log" >/dev/null \ + || fail "sync did not provision the missing Project option" +grep -F 'addProjectV2DraftIssue' "$case_dir/gh.log" >/dev/null \ + || fail "sync did not create the card once the Project option existed" +grep -F 'optionId=gamma-project' "$case_dir/gh.log" >/dev/null \ + || fail "the new card was not tagged with the newly provisioned Project option" +jq -e '.data.user.projectV2.fields.nodes[] | select(.name == "Project") | .options[] | select(.name == "gamma")' \ + "$case_dir/board-state.json" >/dev/null \ + || fail "the provisioned option was not persisted on the board" +jq -e '.data.user.projectV2.fields.nodes[] | select(.name == "Project") | .options[] | select(.name == "fixture-firstmate")' \ + "$case_dir/board-state.json" >/dev/null \ + || fail "provisioning a new option dropped an option already in use" +pass "an unmapped registered project's missing Project option is provisioned, not silently dropped" + case_dir="$TMP_ROOT/watcher-diagnostic" mkdir -p "$case_dir/home/config" "$case_dir/home/data" "$case_dir/home/state" fb=$(install_fakes "$case_dir") From 1118d1fd7ba1e19935a234b6386a68f9117783fb Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:02:55 +0000 Subject: [PATCH 03/23] no-mistakes(review): Fix Helm project routing edge cases --- bin/fm-helm-lib.sh | 21 ++++++++++++++++----- bin/fm-helm-project-map.sh | 24 ++++++++++++++++++++---- bin/fm-helm-reconcile.sh | 2 +- bin/fm-helm-sync.sh | 25 ++++++++++++++++++++----- 4 files changed, 57 insertions(+), 15 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index 2bb1be5f06e..c46deadbbc2 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -138,6 +138,15 @@ fm_helm_project_names() { done | awk '!seen[$0]++' } +fm_helm_project_added_date() { + local project=$1 home + shift + for home in "$@"; do + [ -f "$home/data/projects.md" ] && [ ! -L "$home/data/projects.md" ] || continue + awk -v p="$project" '$1 == "-" && $2 == p && match($0, /\(added [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]\)/) { print substr($0, RSTART + 7, 10); exit }' "$home/data/projects.md" + done | head -1 +} + # fm_helm_ensure_field_options <run-fn> <project-id> <field-name> <required-csv> # # Ensures GitHub Project <project-id>'s single-select field <field-name> has @@ -431,11 +440,11 @@ fm_helm_desired_program() { elif ($registered | index($base)) != null then $base elif $r == "other" then "other" else null end; - def board_of($routing; $default_owner; $default_number): + def board_of($routing; $registered; $default_owner; $default_number): (.repo // "") as $r | ($r | if contains("/") then split("/") | .[-1] else . end) as $base | ($routing[0].projects // {}) as $projects - | ([$projects | to_entries[] | select(.key == $r or .key == $base)][0].value // null) as $entry + | ([$projects | to_entries[] | select((.key == $r or .key == $base) and (.key as $key | ($registered | index($key) != null)))][0].value // null) as $entry | if $entry != null and (($entry.state // "active") == "active" or ($entry.state // "") == "migrating") and (($entry.owner // "") | type) == "string" @@ -472,7 +481,7 @@ fm_helm_desired_program() { kind_of as $kind | ((.priority // "3") | priority_name) as $priority | project_of($registered) as $project - | board_of($routing; $default_owner; $default_number) as $board + | board_of($routing; $registered; $default_owner; $default_number) as $board | .id as $id | (if (.report_path // "") != "" then .report_path elif any($report_ids[]; . == $id) then "data/" + $id + "/report.md" @@ -589,7 +598,9 @@ fm_helm_plan_program() { elif ($matches | length) > 1 then {phase: "error", action: ("duplicate Helm cards for " + $r.id)} elif ($matches | length) == 0 then - if $deleted_by_task[$r.id] != null then + if $retain_source == "1" then + {phase: "record", action: "skip", task: $r.id, note: $d.note} + elif $deleted_by_task[$r.id] != null then {phase: "record", action: "skip", task: $r.id, tombstone: $deleted_by_task[$r.id].line, note: $d.note} elif $old != null and $old.item != "" and (($old.owner != $board_owner) or ($old.number != $board_number)) then {phase: "record", action: "none", task: $r.id, cache: old_cache($old), note: $d.note} @@ -718,7 +729,7 @@ fm_helm_plan_program() { (if $title_write then $dt elif $title_conflict then $bt elif $rebuilt or $ct == $dt or ($title_board_changed | not) then $dt else $bt end) + "\t" + (if $body_write then $db elif $body_conflict then $bb elif $rebuilt or $cb == $db or ($body_board_changed | not) then $db else $bb end) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)) as $cache | {phase: "record", - action: (if $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), + action: (if $retain_source == "1" then "none" elif $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), task: $r.id, item: $card.id, cache: $cache, draft: $text.draft, title: (if $title_write then $d.title else $card_title end), body: (if $body_write then $d.body else $card_body end), fields: $writes, wakes: ($text.wakes + $disp.wakes + $st.wakes diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 2c8e8083892..3b51f0424af 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -216,7 +216,7 @@ ensure_schema() { } map_link() { - local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' now + local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' now prior prior_owner prior_number [ -n "$project" ] || fail "link requires a local project name" project_registered "$project" || fail "local project '$project' is not registered in data/projects.md" [ -n "$title" ] || title="$project" @@ -229,6 +229,9 @@ map_link() { esac shift done + prior=$(map_entry "$project") + prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') + prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') PROJECT_SCHEMA_OPTION=$project create_or_reuse_board "$owner" "$title" "$existing" ensure_schema @@ -242,7 +245,11 @@ map_link() { printf 'linked: %s -> %s/%s "%s"\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" "$BOARD_TITLE" printf 'link changes future cards only; use move %s --existing %s/%s --yes to relocate existing cards.\n' \ "$project" "$BOARD_OWNER" "$BOARD_NUMBER" - "$SCRIPT_DIR/fm-helm-sync.sh" || true + if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$BOARD_OWNER/$BOARD_NUMBER" ]; then + FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true + else + "$SCRIPT_DIR/fm-helm-sync.sh" || true + fi } backlog_task_ids() { # <project> <output-file> @@ -450,6 +457,8 @@ map_move() { if [ -n "$default_dest" ]; then BOARD_OWNER=$DEFAULT_OWNER; BOARD_NUMBER=$DEFAULT_NUMBER board_view "$BOARD_OWNER" "$BOARD_NUMBER" || fail "default Helm Project $BOARD_OWNER/$BOARD_NUMBER does not resolve" + PROJECT_SCHEMA_OPTION=$project + ensure_schema else [ -n "$title" ] || title="$project" PROJECT_SCHEMA_OPTION=$project @@ -511,15 +520,22 @@ map_move() { } map_unlink() { - local project=$1 + local project=$1 prior prior_owner prior_number map_load + prior=$(map_entry "$project") + prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') + prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') jq --arg p "$project" '.projects |= del(.[$p]) | .nudges |= del(.[$p])' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ || fail "could not prepare the unlink" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" printf 'unlinked: %s -> default Helm board %s/%s\n' "$project" "$DEFAULT_OWNER" "$DEFAULT_NUMBER" printf 'unlink changes future cards only; use move %s --default --yes to relocate existing cards.\n' "$project" - "$SCRIPT_DIR/fm-helm-sync.sh" || true + if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$DEFAULT_OWNER/$DEFAULT_NUMBER" ]; then + FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true + else + "$SCRIPT_DIR/fm-helm-sync.sh" || true + fi } map_list() { diff --git a/bin/fm-helm-reconcile.sh b/bin/fm-helm-reconcile.sh index 7c26c66798c..e6a74ae0924 100755 --- a/bin/fm-helm-reconcile.sh +++ b/bin/fm-helm-reconcile.sh @@ -152,7 +152,7 @@ else live_count=$(gh-axi project item-list "$DEFAULT_NUMBER" --owner "$DEFAULT_OWNER" --query "Project:$project" --limit 1000 2>/dev/null \ | awk '/^[[:space:]]+[A-Za-z0-9_]+,/{n++} END{print n+0}') old_enough=0 - added=$(grep -F -- "- $project " "$FM_HOME_PATH/data/projects.md" 2>/dev/null | sed -n 's/.*(added \([0-9-]*\)).*/\1/p' | head -1) + added=$(fm_helm_project_added_date "$project" "${HOME_PATHS[@]}") if [ -n "$added" ] && added_epoch=$(date -d "$added" +%s 2>/dev/null); then [ $((NOW - added_epoch)) -ge 2592000 ] && old_enough=1 fi diff --git a/bin/fm-helm-sync.sh b/bin/fm-helm-sync.sh index 86ab691e091..1905f6b0463 100755 --- a/bin/fm-helm-sync.sh +++ b/bin/fm-helm-sync.sh @@ -240,6 +240,16 @@ DISPATCH_STATUS=$(printf '%s\n' "$CONFIG_JSON" | jq -r '.dispatch_status // "In || helm_fail_open "config/helm.json is not valid JSON" [ -n "$DISPATCH_STATUS" ] || helm_fail_open "config/helm.json has an empty dispatch_status" +RETAIN_OWNER= +RETAIN_NUMBER= +RETAIN_PROJECT=${FM_HELM_RETAIN_PROJECT:-} +case "${FM_HELM_RETAIN_BOARD:-}" in + */*) RETAIN_OWNER=${FM_HELM_RETAIN_BOARD%/*}; RETAIN_NUMBER=${FM_HELM_RETAIN_BOARD##*/} ;; +esac +case "$RETAIN_OWNER/$RETAIN_NUMBER" in + /*|*//*|*/|*' '*|*/*[!0-9]*) RETAIN_OWNER=; RETAIN_NUMBER=; RETAIN_PROJECT= ;; +esac + if [ -f "$ROUTING_FILE" ]; then jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ "$ROUTING_FILE" >/dev/null 2>&1 \ @@ -952,11 +962,12 @@ process_board() { fi DISPATCH_OPTION_ID=$(option_id Status "$DISPATCH_STATUS") group_desired="$TMP_DIR/desired-$BOARD_INDEX.json" - jq --arg owner "$owner" --argjson number "$number" --rawfile cards "$OLD_CARDS" \ + jq --arg owner "$owner" --argjson number "$number" --arg retain_owner "$RETAIN_OWNER" --argjson retain_number "${RETAIN_NUMBER:-0}" --arg retain_project "$RETAIN_PROJECT" --rawfile cards "$CACHE_WORK" \ '($cards | split("\n") | map(select(. != "") | split("\t")) | map(select(length >= 11 and .[9] == $owner and (.[10] | tonumber) == $number) | .[0])) as $historical | map(. as $record | select(($record.desired.board.owner == $owner and $record.desired.board.number == $number) + or ($retain_owner == $owner and $retain_number == $number and $record.desired.project == $retain_project) or (($historical | index($record.id)) != null)))' \ "$DESIRED_JSON" >"$group_desired" \ || helm_fail_open "could not group desired Helm cards" @@ -1011,11 +1022,11 @@ process_board() { fi PLAN="$TMP_DIR/plan-$BOARD_INDEX.nul" jq -j --slurpfile desired "$group_desired" \ - --rawfile cards "$OLD_CARDS" --rawfile deleted "$OLD_DELETED" --rawfile markers "$OLD_MARKERS" \ + --rawfile cards "$CACHE_WORK" --rawfile deleted "$OLD_DELETED" --rawfile markers "$OLD_MARKERS" \ --rawfile divergences "$OLD_DIVERGENCES" \ --rawfile fps "$FPS" \ --arg force "$FORCE" --arg dispatch_status "$DISPATCH_STATUS" --arg now "$NOW_EPOCH" \ - --arg tsv_existed "$TSV_EXISTED" --arg board_owner "$owner" --argjson board_number "$number" \ + --arg tsv_existed "$TSV_EXISTED" --arg retain_source "$( [ "$owner" = "$RETAIN_OWNER" ] && [ "$number" = "$RETAIN_NUMBER" ] && printf 1 || printf 0 )" --arg board_owner "$owner" --argjson board_number "$number" \ --arg default_owner "$OWNER" --argjson default_number "$PROJECT_NUMBER" \ "$(fm_helm_plan_program)" "$BOARD_JSON" >"$PLAN" \ || { board_failure "$key" "could not plan the board reconciliation"; publish_progress || helm_fail_open "could not publish Helm board acknowledgement"; return 0; } @@ -1180,6 +1191,9 @@ fi BOARD_KEYS_RAW="$TMP_DIR/board-keys.raw" BOARD_KEYS_FILE="$TMP_DIR/board-keys.tsv" { + if [ -n "$RETAIN_OWNER" ]; then + printf '%s\t%s\n' "$RETAIN_OWNER" "$RETAIN_NUMBER" + fi printf '%s\t%s\n' "$OWNER" "$PROJECT_NUMBER" jq -r '.[] | [.desired.board.owner, (.desired.board.number | tostring)] | @tsv' "$DESIRED_JSON" awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" \ @@ -1187,8 +1201,9 @@ BOARD_KEYS_FILE="$TMP_DIR/board-keys.tsv" } | awk -F '\t' '!seen[$1 SUBSEP $2]++' >"$BOARD_KEYS_RAW" \ || helm_fail_open "could not build the Helm board groups" { - awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 == owner && $2 == number' "$BOARD_KEYS_RAW" - awk -F '\t' -v owner="$OWNER" -v number="$PROJECT_NUMBER" '$1 != owner || $2 != number' "$BOARD_KEYS_RAW" \ + awk -F '\t' -v owner="$RETAIN_OWNER" -v number="$RETAIN_NUMBER" '$1 == owner && $2 == number' "$BOARD_KEYS_RAW" + awk -F '\t' -v owner="$RETAIN_OWNER" -v number="$RETAIN_NUMBER" -v default_owner="$OWNER" -v default_number="$PROJECT_NUMBER" '$1 == default_owner && $2 == default_number && !($1 == owner && $2 == number)' "$BOARD_KEYS_RAW" + awk -F '\t' -v owner="$RETAIN_OWNER" -v number="$RETAIN_NUMBER" -v default_owner="$OWNER" -v default_number="$PROJECT_NUMBER" '!($1 == owner && $2 == number) && !($1 == default_owner && $2 == default_number) { print }' "$BOARD_KEYS_RAW" \ | sort -t $'\t' -k1,1 -k2,2n } >"$BOARD_KEYS_FILE" || helm_fail_open "could not order the Helm board groups" BOARD_COUNT=$(wc -l <"$BOARD_KEYS_FILE") From 0960b267772a236d5815a4db40ab3b7506c8b3b2 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:08:18 +0000 Subject: [PATCH 04/23] no-mistakes(review): Scope retention to remapped project and board --- bin/fm-helm-lib.sh | 6 ++++-- bin/fm-helm-sync.sh | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index c46deadbbc2..7286594c6c4 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -586,6 +586,7 @@ fm_helm_plan_program() { | def record_entries: [ $records[] as $r | $r.desired as $d + | ($retain_source == "1" and $d.project == $retain_project) as $retain | ($by_line1["`" + $r.id + "`"] // []) as $matches | ($fp_by_task[$r.id].fp // "") as $fp | $old_by_task[$r.id] as $old @@ -598,7 +599,7 @@ fm_helm_plan_program() { elif ($matches | length) > 1 then {phase: "error", action: ("duplicate Helm cards for " + $r.id)} elif ($matches | length) == 0 then - if $retain_source == "1" then + if $retain then {phase: "record", action: "skip", task: $r.id, note: $d.note} elif $deleted_by_task[$r.id] != null then {phase: "record", action: "skip", task: $r.id, tombstone: $deleted_by_task[$r.id].line, note: $d.note} @@ -729,7 +730,7 @@ fm_helm_plan_program() { (if $title_write then $dt elif $title_conflict then $bt elif $rebuilt or $ct == $dt or ($title_board_changed | not) then $dt else $bt end) + "\t" + (if $body_write then $db elif $body_conflict then $bb elif $rebuilt or $cb == $db or ($body_board_changed | not) then $db else $bb end) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)) as $cache | {phase: "record", - action: (if $retain_source == "1" then "none" elif $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), + action: (if $retain then "none" elif $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), task: $r.id, item: $card.id, cache: $cache, draft: $text.draft, title: (if $title_write then $d.title else $card_title end), body: (if $body_write then $d.body else $card_body end), fields: $writes, wakes: ($text.wakes + $disp.wakes + $st.wakes @@ -785,6 +786,7 @@ fm_helm_plan_program() { if $tsv_existed != "true" then [] else [ $old_rows[] as $o | if $o.task == "" or $o.item == "" then empty + elif $o.owner != $board_owner or $o.number != $board_number then empty elif $item_set[$o.item] then empty elif $line1_set["`" + $o.task + "`"] then empty else diff --git a/bin/fm-helm-sync.sh b/bin/fm-helm-sync.sh index 1905f6b0463..d7f57538201 100755 --- a/bin/fm-helm-sync.sh +++ b/bin/fm-helm-sync.sh @@ -1026,7 +1026,7 @@ process_board() { --rawfile divergences "$OLD_DIVERGENCES" \ --rawfile fps "$FPS" \ --arg force "$FORCE" --arg dispatch_status "$DISPATCH_STATUS" --arg now "$NOW_EPOCH" \ - --arg tsv_existed "$TSV_EXISTED" --arg retain_source "$( [ "$owner" = "$RETAIN_OWNER" ] && [ "$number" = "$RETAIN_NUMBER" ] && printf 1 || printf 0 )" --arg board_owner "$owner" --argjson board_number "$number" \ + --arg tsv_existed "$TSV_EXISTED" --arg retain_source "$( [ "$owner" = "$RETAIN_OWNER" ] && [ "$number" = "$RETAIN_NUMBER" ] && printf 1 || printf 0 )" --arg retain_project "$RETAIN_PROJECT" --arg board_owner "$owner" --argjson board_number "$number" \ --arg default_owner "$OWNER" --argjson default_number "$PROJECT_NUMBER" \ "$(fm_helm_plan_program)" "$BOARD_JSON" >"$PLAN" \ || { board_failure "$key" "could not plan the board reconciliation"; publish_progress || helm_fail_open "could not publish Helm board acknowledgement"; return 0; } From 025afe305d262d5da2b4d4cf1d305d8e5fa72de5 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:14:04 +0000 Subject: [PATCH 05/23] no-mistakes(review): Persist retention and disambiguate project routing --- bin/fm-helm-lib.sh | 5 ++++- bin/fm-helm-project-map.sh | 16 +++++++++------- bin/fm-helm-sync.sh | 33 +++++++++++++++++++++++++++------ 3 files changed, 40 insertions(+), 14 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index 7286594c6c4..fc0bc795065 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -444,7 +444,10 @@ fm_helm_desired_program() { (.repo // "") as $r | ($r | if contains("/") then split("/") | .[-1] else . end) as $base | ($routing[0].projects // {}) as $projects - | ([$projects | to_entries[] | select((.key == $r or .key == $base) and (.key as $key | ($registered | index($key) != null)))][0].value // null) as $entry + | (if ($registered | index($r)) != null then $r + elif ($registered | index($base)) != null then $base + else "" end) as $project + | ($projects[$project] // null) as $entry | if $entry != null and (($entry.state // "active") == "active" or ($entry.state // "") == "migrating") and (($entry.owner // "") | type) == "string" diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 3b51f0424af..90d8a8fd240 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -88,7 +88,7 @@ TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-project-map.XXXXXX") \ map_load() { if [ -f "$MAP_FILE" ]; then [ ! -L "$MAP_FILE" ] || fail "refusing symlinked Helm routing state" - jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ + jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object") and ((.retention // null) | type == "object" or . == null)' \ "$MAP_FILE" >/dev/null 2>&1 || fail "data/helm-project-map.json is invalid" cp -- "$MAP_FILE" "$TMP_DIR/map.json" || fail "could not stage Helm routing state" else @@ -229,16 +229,16 @@ map_link() { esac shift done - prior=$(map_entry "$project") + prior=$(jq -c --arg p "$project" 'if .retention.project == $p then .retention else .projects[$p] // null end' "$TMP_DIR/map.json") prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') PROJECT_SCHEMA_OPTION=$project create_or_reuse_board "$owner" "$title" "$existing" ensure_schema now=$(date -u +%Y-%m-%dT%H:%M:%SZ) - jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" \ + jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" --arg prior_owner "$prior_owner" --argjson prior_number "${prior_number:-0}" \ --arg title "$BOARD_TITLE" --arg url "$BOARD_URL" --arg now "$now" \ - '.projects[$p] = {owner:$owner,number:$number,title:$title,url:$url,state:"active",linked_at:$now,move:null,orphan_hold_task:null} | .nudges |= del(.[$p])' \ + '.projects[$p] = {owner:$owner,number:$number,title:$title,url:$url,state:"active",linked_at:$now,move:null,orphan_hold_task:null} | .nudges |= del(.[$p]) | if $prior_owner != "" and $prior_number > 0 then .retention = {project:$p,owner:$prior_owner,number:$prior_number} else . end' \ "$TMP_DIR/map.json" >"$TMP_DIR/map.next" || fail "could not prepare the Helm routing entry" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" @@ -259,7 +259,9 @@ backlog_task_ids() { # <project> <output-file> backlog_out="$TMP_DIR/backlog-${home_id}.json" fm_helm_parse_home_backlog "$home_id" "$home/data/backlog.md" "$backlog_out" 2>/dev/null \ || fail "could not parse $home/data/backlog.md" - jq -r --arg p "$project" '.[] | select((.repo // "") == $p or (((.repo // "") | split("/"))[-1] == $p)) | .id' "$backlog_out" >>"$output" \ + jq -r --arg p "$project" --argjson registered "$(fm_helm_project_names "${HOME_PATHS[@]}" | jq -Rsc 'split("\n") | map(select(length > 0))')" ' + .[] | (.repo // "") as $repo | ($repo | split("/") | .[-1]) as $base + | select(if ($registered | index($repo)) != null then $repo == $p else $base == $p end) | .id' "$backlog_out" >>"$output" \ || fail "could not find Helm tasks for $project" done <<EOF $HOMES_TSV @@ -522,10 +524,10 @@ map_move() { map_unlink() { local project=$1 prior prior_owner prior_number map_load - prior=$(map_entry "$project") + prior=$(jq -c --arg p "$project" 'if .retention.project == $p then .retention else .projects[$p] // null end' "$TMP_DIR/map.json") prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') - jq --arg p "$project" '.projects |= del(.[$p]) | .nudges |= del(.[$p])' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + jq --arg p "$project" --arg owner "$prior_owner" --argjson number "${prior_number:-0}" '.projects |= del(.[$p]) | .nudges |= del(.[$p]) | if $owner != "" and $number > 0 then .retention = {project:$p,owner:$owner,number:$number} else . end' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ || fail "could not prepare the unlink" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" diff --git a/bin/fm-helm-sync.sh b/bin/fm-helm-sync.sh index d7f57538201..7a2ac80e12c 100755 --- a/bin/fm-helm-sync.sh +++ b/bin/fm-helm-sync.sh @@ -243,6 +243,7 @@ DISPATCH_STATUS=$(printf '%s\n' "$CONFIG_JSON" | jq -r '.dispatch_status // "In RETAIN_OWNER= RETAIN_NUMBER= RETAIN_PROJECT=${FM_HELM_RETAIN_PROJECT:-} +RETAIN_INDEXED=0 case "${FM_HELM_RETAIN_BOARD:-}" in */*) RETAIN_OWNER=${FM_HELM_RETAIN_BOARD%/*}; RETAIN_NUMBER=${FM_HELM_RETAIN_BOARD##*/} ;; esac @@ -251,7 +252,7 @@ case "$RETAIN_OWNER/$RETAIN_NUMBER" in esac if [ -f "$ROUTING_FILE" ]; then - jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ + jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object") and ((.retention // null) | type == "object" or . == null)' \ "$ROUTING_FILE" >/dev/null 2>&1 \ || helm_fail_open "data/helm-project-map.json is not valid Helm routing JSON" fi @@ -832,6 +833,11 @@ fi REGISTERED_PROJECTS=$(fm_helm_project_names "${HOME_PATHS[@]}" \ | jq -Rsc 'split("\n") | map(select(length > 0))') \ || helm_fail_open "could not read the project registry" +if [ -z "$RETAIN_OWNER" ]; then + IFS=$'\t' read -r RETAIN_PROJECT RETAIN_OWNER RETAIN_NUMBER < <(jq -r ' + .retention // {} | select((.project | type) == "string" and (.owner | type) == "string" and (.owner | length) > 0 and (.number | type) == "number" and .number > 0) + | [.project,.owner,(.number | tostring)] | @tsv' "$ROUTING_JSON") +fi # A mapping key that no longer appears in any local registry is a broken # boundary, not an instruction to silently route future cards to the default. @@ -977,7 +983,7 @@ process_board() { # Project field may not have that option yet) so the per-board hot path # stays a single jq call, same as before this check grew a second part. schema_ok=false missing_projects= - IFS=$'\t' read -r schema_ok missing_projects < <(jq -r --argjson records "$(cat "$group_desired")" --arg dispatch "$DISPATCH_STATUS" ' + IFS=$'\t' read -r schema_ok missing_projects < <(jq -r --argjson records "$(cat "$group_desired")" --argjson registered "$REGISTERED_PROJECTS" --slurpfile routing "$ROUTING_JSON" --arg owner "$owner" --argjson number "$number" --arg default_owner "$OWNER" --argjson default_number "$PROJECT_NUMBER" --arg dispatch "$DISPATCH_STATUS" ' .data.user.projectV2.fields.nodes as $fields | (def has_option($field; $name): any($fields[]; .name == $field and .__typename == "ProjectV2SingleSelectField" and any(.options[]?; .name == $name)); any($fields[]; .name == "Status" and .__typename == "ProjectV2SingleSelectField") @@ -988,7 +994,10 @@ process_board() { and all(["P0", "P1", "P2", "P3", "P4"][]; has_option("Priority"; .)) and all(["ship", "investigation", "decision"][]; has_option("Kind"; .))) as $ok | ([$fields[] | select(.name == "Project" and .__typename == "ProjectV2SingleSelectField") | .options[]?.name]) as $have - | [$ok, (($records | map(.desired.project) | unique) - $have | join(","))] | @tsv + | (if $owner == $default_owner and $number == $default_number + then ($registered - [($routing[0].projects // {}) | keys[]]) + else [] end) as $unmapped + | [$ok, ((($records | map(.desired.project)) + $unmapped | unique) - $have | join(","))] | @tsv ' "$BOARD_JSON" 2>/dev/null) if [ "$schema_ok" != true ]; then board_failure "$key" "required Helm fields or options are unavailable" @@ -1180,9 +1189,11 @@ while read_entry; do done exec 3<&- publish_progress || helm_fail_open "could not publish Helm board acknowledgement" -if [ "$BOARD_WRITE_FAILURE" -eq 1 ]; then - board_failure "$key" "one or more board writes failed" -fi + if [ "$BOARD_WRITE_FAILURE" -eq 1 ]; then + board_failure "$key" "one or more board writes failed" + elif [ "$owner" = "$RETAIN_OWNER" ] && [ "$number" = "$RETAIN_NUMBER" ]; then + RETAIN_INDEXED=1 + fi } # Stable board order keeps the configured default responsive, then processes @@ -1216,6 +1227,16 @@ while IFS=$'\t' read -r board_owner board_number; do process_board "$board_owner" "$board_number" done <"$BOARD_KEYS_FILE" +if [ "$RETAIN_INDEXED" -eq 1 ]; then + jq --arg project "$RETAIN_PROJECT" --arg owner "$RETAIN_OWNER" --argjson number "$RETAIN_NUMBER" ' + if .retention.project == $project and .retention.owner == $owner and .retention.number == $number then del(.retention) else . end' \ + "$ROUTING_JSON" >"$TMP_DIR/routing.next" \ + || helm_fail_open "could not clear the indexed Helm retention" + mv -f -- "$TMP_DIR/routing.next" "$ROUTING_JSON" + publish_file "$ROUTING_JSON" "$ROUTING_FILE" \ + || helm_fail_open "could not publish Helm routing state" +fi + publish_board_failure_wakes if [ "$CONFLICT_WAKE" -eq 1 ]; then From defdac68189fe2527e35f2b98729b31c7bf53fa1 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:17:45 +0000 Subject: [PATCH 06/23] no-mistakes(review): Serialize Helm routing map updates --- bin/fm-helm-project-map.sh | 21 +++++++++++++++------ bin/fm-helm-reconcile.sh | 9 +++++++++ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 90d8a8fd240..389d2959a7c 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -104,6 +104,12 @@ map_publish() { chmod 0600 "$tmp" && mv -f -- "$tmp" "$MAP_FILE" } +map_release_lock() { + [ "$LOCK_HELD" -eq 1 ] || return 0 + fm_lock_release "$LOCK_FILE" || return 1 + LOCK_HELD=0 +} + project_registered() { local project=$1 fm_helm_project_names "${HOME_PATHS[@]}" | awk -v p="$project" '$0 == p { found=1 } END { exit(found ? 0 : 1) }' @@ -245,6 +251,7 @@ map_link() { printf 'linked: %s -> %s/%s "%s"\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" "$BOARD_TITLE" printf 'link changes future cards only; use move %s --existing %s/%s --yes to relocate existing cards.\n' \ "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + map_release_lock || fail "could not release the Helm routing lock" if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$BOARD_OWNER/$BOARD_NUMBER" ]; then FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true else @@ -493,10 +500,6 @@ map_move() { printf 're-run with --yes to execute.\n' exit 0 fi - if ! fm_lock_try_acquire "$LOCK_FILE"; then - fail "another Helm sync or move is already running" - fi - LOCK_HELD=1 now=$(date -u +%Y-%m-%dT%H:%M:%SZ) jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" \ --arg from_owner "$source_owner" --argjson from_number "$source_number" --arg now "$now" \ @@ -516,8 +519,7 @@ map_move() { else printf 'move partial: %s card(s) remain; rerun move or sync to resume.\n' "$remaining" fi - fm_lock_release "$LOCK_FILE" || fail "could not release the Helm move lock" - LOCK_HELD=0 + map_release_lock || fail "could not release the Helm routing lock" "$SCRIPT_DIR/fm-helm-sync.sh" || true } @@ -533,6 +535,7 @@ map_unlink() { map_publish || fail "could not publish data/helm-project-map.json" printf 'unlinked: %s -> default Helm board %s/%s\n' "$project" "$DEFAULT_OWNER" "$DEFAULT_NUMBER" printf 'unlink changes future cards only; use move %s --default --yes to relocate existing cards.\n' "$project" + map_release_lock || fail "could not release the Helm routing lock" if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$DEFAULT_OWNER/$DEFAULT_NUMBER" ]; then FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true else @@ -591,6 +594,8 @@ case "$command" in [ "$#" -gt 0 ] || fail "link requires a local project name" project=$1; shift; title= case "${1:-}" in --*) ;; '') ;; *) title=$1; shift ;; esac + fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" + LOCK_HELD=1 map_load map_link "$project" "$title" "$@" ;; @@ -598,10 +603,14 @@ case "$command" in [ "$#" -gt 0 ] || fail "move requires a local project name" project=$1; shift; title= case "${1:-}" in --*) ;; '') ;; *) title=$1; shift ;; esac + fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" + LOCK_HELD=1 map_move "$project" "$title" "$@" ;; unlink) [ "$#" -eq 1 ] || fail "unlink requires exactly one local project name" + fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" + LOCK_HELD=1 map_unlink "$1" ;; sync) [ "$#" -eq 0 ] || fail "sync takes no arguments"; map_sync ;; diff --git a/bin/fm-helm-reconcile.sh b/bin/fm-helm-reconcile.sh index e6a74ae0924..8bec8ecc306 100755 --- a/bin/fm-helm-reconcile.sh +++ b/bin/fm-helm-reconcile.sh @@ -18,10 +18,15 @@ STATE_PATH="${FM_STATE_OVERRIDE:-$FM_HOME_PATH/state}" CONFIG_FILE="$CONFIG_PATH/helm.json" MAP_FILE="$DATA_PATH/helm-project-map.json" LAST_FILE="$STATE_PATH/.helm-reconcile-last" +LOCK_FILE="$STATE_PATH/.helm-sync.lock" TMP_DIR= +LOCK_HELD=0 reconcile_cleanup() { local status=$? + if [ "$LOCK_HELD" -eq 1 ]; then + fm_lock_release "$LOCK_FILE" 2>/dev/null || true + fi [ -z "$TMP_DIR" ] || [ ! -d "$TMP_DIR" ] || rm -rf -- "$TMP_DIR" exit "$status" } @@ -73,6 +78,8 @@ if [ "$FORCE" -eq 0 ] && [ -f "$LAST_FILE" ] && [ ! -L "$LAST_FILE" ]; then fi TMP_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-helm-reconcile.XXXXXX") || fail "could not create temporary workspace" +fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" +LOCK_HELD=1 if [ -f "$MAP_FILE" ]; then [ ! -L "$MAP_FILE" ] || fail "refusing symlinked Helm routing state" jq -e '.version == 1 and ((.projects // {}) | type == "object") and ((.nudges // {}) | type == "object")' \ @@ -170,6 +177,8 @@ fi if [ "$MAP_DIRTY" -ne 0 ] && ! publish_map; then fail "could not publish Helm reconciliation state" fi +fm_lock_release "$LOCK_FILE" || fail "could not release the Helm routing lock" +LOCK_HELD=0 # A confirmed migration is an already-approved operation. Continue it without # asking for a second confirmation, then let the ordinary sync reconcile item From 8ca94c6b1daef84f1a27627bb9f48884b405a9c8 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:23:39 +0000 Subject: [PATCH 07/23] no-mistakes(review): Guard pending remaps and permit empty card bodies --- bin/fm-helm-project-map.sh | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 389d2959a7c..d86483f04b7 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -110,6 +110,12 @@ map_release_lock() { LOCK_HELD=0 } +require_no_pending_retention() { + local project + project=$(jq -r '.retention.project // empty' "$TMP_DIR/map.json") + [ -z "$project" ] || fail "a prior remap for '$project' is awaiting sync; wait for it to complete first" +} + project_registered() { local project=$1 fm_helm_project_names "${HOME_PATHS[@]}" | awk -v p="$project" '$0 == p { found=1 } END { exit(found ? 0 : 1) }' @@ -327,7 +333,7 @@ move_card_read_source() { MOVE_SOURCE_TITLE=$(jq -r '.data.node.content.title // empty' <<<"$source_json") MOVE_SOURCE_BODY=$(jq -r '.data.node.content.body // empty' <<<"$source_json") [ "$MOVE_SOURCE_TYPE" = DraftIssue ] || [ "$MOVE_SOURCE_TYPE" = Issue ] || return 1 - [ -n "$MOVE_SOURCE_TITLE" ] && [ -n "$MOVE_SOURCE_BODY" ] + [ -n "$MOVE_SOURCE_TITLE" ] } # move_card_copy_fields <source-item-id> <destination-item-id> copies the @@ -435,7 +441,6 @@ move_execute() { map_move() { local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' default_dest='' yes=0 entry source_owner source_number count now ids [ -n "$project" ] || fail "move requires a local project name" - map_load entry=$(map_entry "$project") if [ "$(printf '%s\n' "$entry" | jq -r '.state // empty')" = migrating ]; then destination_owner=$(printf '%s\n' "$entry" | jq -r '.owner') @@ -525,7 +530,6 @@ map_move() { map_unlink() { local project=$1 prior prior_owner prior_number - map_load prior=$(jq -c --arg p "$project" 'if .retention.project == $p then .retention else .projects[$p] // null end' "$TMP_DIR/map.json") prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') @@ -597,6 +601,7 @@ case "$command" in fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" LOCK_HELD=1 map_load + require_no_pending_retention map_link "$project" "$title" "$@" ;; move) @@ -605,12 +610,16 @@ case "$command" in case "${1:-}" in --*) ;; '') ;; *) title=$1; shift ;; esac fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" LOCK_HELD=1 + map_load + require_no_pending_retention map_move "$project" "$title" "$@" ;; unlink) [ "$#" -eq 1 ] || fail "unlink requires exactly one local project name" fm_lock_try_acquire "$LOCK_FILE" || fail "another Helm sync or map operation is already running" LOCK_HELD=1 + map_load + require_no_pending_retention map_unlink "$1" ;; sync) [ "$#" -eq 0 ] || fail "sync takes no arguments"; map_sync ;; From 31ea244447994165b6296b65fead49eaca22ebf7 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:30:17 +0000 Subject: [PATCH 08/23] no-mistakes(review): Discover cacheless Helm move source cards --- bin/fm-helm-project-map.sh | 64 +++++++++++++++++++++++++++++++ tests/fm-helm-project-map.test.sh | 22 +++++++++++ 2 files changed, 86 insertions(+) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index d86483f04b7..20c088fed16 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -320,6 +320,33 @@ move_card_delete() { # <owner> <number> <item-id> gh-axi project item-delete "$2" --owner "$1" --id "$3" >/dev/null 2>&1 } +move_discover_source_cards() { + local owner=$1 number=$2 ids=$3 output=$4 board_output board_id cursor='' page=0 response wanted + board_output=$(gh-axi project view "$number" --owner "$owner" 2>/dev/null) || return 1 + board_id=$(printf '%s\n' "$board_output" | sed -n 's/^id: //p' | head -1) + [ -n "$board_id" ] || return 1 + wanted=$(jq -Rsc 'split("\n") | map(select(length > 0))' "$ids") || return 1 + : >"$output" + while :; do + page=$((page + 1)) + [ "$page" -le 50 ] || return 1 + response=$(gh-axi api graphql --field 'query=query($projectId:ID!,$cursor:String){node(id:$projectId){... on ProjectV2 {items(first:100,after:$cursor){nodes{id content{__typename ... on DraftIssue {id body} ... on Issue {id body}} pageInfo{hasNextPage endCursor}}}}}' --field "projectId=$board_id" --field "cursor=$cursor" 2>/dev/null) || return 1 + printf '%s\n' "$response" | jq -r --argjson wanted "$wanted" ' + .data.node.items as $items + | $items.nodes[]? + | select(.content.__typename == "DraftIssue" or .content.__typename == "Issue") + | (.content.body // "" | split("\n")[0]) as $marker + | select($marker | test("^`[A-Za-z0-9._-]+`$")) + | ($marker[1:-1]) as $task + | select($wanted | index($task) != null) + | [$task, .id, .content.id, (if .content.__typename == "Issue" then "issue" else "draft" end)] | @tsv' >>"$output" \ + || return 1 + [ "$(printf '%s\n' "$response" | jq -r '.data.node.items.pageInfo.hasNextPage // false')" = false ] && return 0 + cursor=$(printf '%s\n' "$response" | jq -r '.data.node.items.pageInfo.endCursor // empty') + [ -n "$cursor" ] || return 1 + done +} + # move_card_read_source <source-item-id> reads the current source content # before a draft move. The identity cache is a routing index, not a board # snapshot, so captain edits must travel with the card when it is relocated. @@ -486,6 +513,43 @@ map_move() { MOVE_TASKS=$(cat -- "$ids") : >"$TMP_DIR/moves.tsv" [ -f "$MOVES_FILE" ] && [ ! -L "$MOVES_FILE" ] && cp -- "$MOVES_FILE" "$TMP_DIR/moves.tsv" + if [ -f "$CARDS_FILE" ]; then + awk -F '\t' -v ids_file="$ids" -v owner="$source_owner" -v number="$source_number" ' + FILENAME == ids_file { wanted[$1]=1; next } + NF >= 11 && $10 == owner && $11 == number { indexed[$1]=1 } + END { for (task in wanted) if (!indexed[task]) print task }' \ + "$ids" "$CARDS_FILE" >"$TMP_DIR/unindexed-task-ids" \ + || fail "could not inspect cached Helm card identities" + else + cp -- "$ids" "$TMP_DIR/unindexed-task-ids" + fi + if [ -s "$TMP_DIR/unindexed-task-ids" ]; then + move_discover_source_cards "$source_owner" "$source_number" "$TMP_DIR/unindexed-task-ids" "$TMP_DIR/source-cards.tsv" \ + || fail "could not inspect source Helm cards for the move" + else + : >"$TMP_DIR/source-cards.tsv" + fi + if [ -s "$TMP_DIR/source-cards.tsv" ]; then + if [ -f "$CARDS_FILE" ]; then + cp -- "$CARDS_FILE" "$TMP_DIR/cards.current" + else + : >"$TMP_DIR/cards.current" + fi + awk -F '\t' -v discovered="$TMP_DIR/source-cards.tsv" -v owner="$source_owner" -v number="$source_number" -v now="$(date +%s)" ' + FILENAME == discovered { found[$1]=$0; next } + { + if ($1 in found && $10 == owner && $11 == number) { + split(found[$1], row, "\t") + print $1 "\t" row[2] "\t" row[3] "\t" row[4] "\t\t\t\t\t" now "\t" owner "\t" number + written[$1]=1 + } else print + } + END { for (task in found) if (!written[task]) { split(found[task], row, "\t"); print task "\t" row[2] "\t" row[3] "\t" row[4] "\t\t\t\t\t" now "\t" owner "\t" number } }' \ + "$TMP_DIR/source-cards.tsv" "$TMP_DIR/cards.current" >"$TMP_DIR/cards.discovered" \ + || fail "could not stage discovered Helm card identities" + chmod 0600 "$TMP_DIR/cards.discovered" && mv -f -- "$TMP_DIR/cards.discovered" "$CARDS_FILE" \ + || fail "could not publish discovered Helm card identities" + fi awk -F '\t' -v ids_file="$ids" -v moves_file="$TMP_DIR/moves.tsv" \ -v owner="$source_owner" -v number="$source_number" \ -v dest_owner="$BOARD_OWNER" -v dest_number="$BOARD_NUMBER" -v epoch="$(date +%s)" \ diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index 9c5eb9abd3c..b8ed7514caf 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -177,6 +177,8 @@ if [ "\${1:-}" = project ] && [ "\${2:-}" = item-delete ]; then fi if [ "\${1:-}" = api ]; then case "\$*" in + *'items(first:100,after:\$cursor)'* ) + jq -n '{data:{node:{items:{nodes:[{id:"old-item",content:{__typename:"DraftIssue",id:"old-draft",body:"\`alpha-task\`\\n\\nCaptain body"}}],pageInfo:{hasNextPage:false,endCursor:null}}}}}' ;; *'itemId=old-item'* ) jq -n '{data:{node:{content:{__typename:"DraftIssue",title:"Captain-edited Alpha",body:"\`alpha-task\`\\n\\nCaptain body"},fieldValues:{nodes:[ {name:"Queued",optionId:"queued-status",field:{name:"Status"}}, @@ -285,6 +287,26 @@ grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item || fail "move ledger did not retain its complete add/delete history" pass "confirmed moves preserve card history in the resumable ledger" +# A rebuilt cache must not make a confirmed move overlook source-board cards. +case_dir="$TMP_ROOT/cacheless-move" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-owner",number:999,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_FAIL_DELETE=1 FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >/dev/null 2>&1 \ + || fail "cacheless confirmed move exited nonzero" +grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item\tdest-ready' \ + "$case_dir/home/state/helm-moves.tsv" >/dev/null \ + || fail "cacheless move did not discover and stage the source card" +grep -F $'alpha-task\told-item\told-draft\tdraft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "cacheless move did not rebuild the source card identity" +pass "cacheless moves discover live source cards" + # Linking a second project onto a board another project already linked must # add the missing Project option, not refuse the board as "incomplete" # (spec decision 6: several mapping entries sharing one board is normal). From 872745410f20eee04705c55b92c7abef16a00b6c Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:33:11 +0000 Subject: [PATCH 09/23] no-mistakes(review): Reject duplicate cacheless move source cards --- bin/fm-helm-project-map.sh | 7 ++++++- tests/fm-helm-project-map.test.sh | 31 ++++++++++++++++++++++++++++++- 2 files changed, 36 insertions(+), 2 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 20c088fed16..98de48918da 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -341,10 +341,15 @@ move_discover_source_cards() { | select($wanted | index($task) != null) | [$task, .id, .content.id, (if .content.__typename == "Issue" then "issue" else "draft" end)] | @tsv' >>"$output" \ || return 1 - [ "$(printf '%s\n' "$response" | jq -r '.data.node.items.pageInfo.hasNextPage // false')" = false ] && return 0 + [ "$(printf '%s\n' "$response" | jq -r '.data.node.items.pageInfo.hasNextPage // false')" = false ] && break cursor=$(printf '%s\n' "$response" | jq -r '.data.node.items.pageInfo.endCursor // empty') [ -n "$cursor" ] || return 1 done + if awk -F '\t' '++seen[$1] == 2 { exit 1 }' "$output"; then + return 0 + fi + printf 'fm-helm-project-map: duplicate source cards found; resolve them before moving\n' >&2 + return 1 } # move_card_read_source <source-item-id> reads the current source content diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index b8ed7514caf..4d2f9e22c5c 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -178,7 +178,14 @@ fi if [ "\${1:-}" = api ]; then case "\$*" in *'items(first:100,after:\$cursor)'* ) - jq -n '{data:{node:{items:{nodes:[{id:"old-item",content:{__typename:"DraftIssue",id:"old-draft",body:"\`alpha-task\`\\n\\nCaptain body"}}],pageInfo:{hasNextPage:false,endCursor:null}}}}}' ;; + if [ -n "\${FM_HELM_DUPLICATE_SOURCE:-}" ]; then + jq -n '{data:{node:{items:{nodes:[ + {id:"old-item",content:{__typename:"DraftIssue",id:"old-draft",body:"\`alpha-task\`\\n\\nCaptain body"}}, + {id:"duplicate-item",content:{__typename:"DraftIssue",id:"duplicate-draft",body:"\`alpha-task\`\\n\\nDuplicate body"}} + ],pageInfo:{hasNextPage:false,endCursor:null}}}}}' + else + jq -n '{data:{node:{items:{nodes:[{id:"old-item",content:{__typename:"DraftIssue",id:"old-draft",body:"\`alpha-task\`\\n\\nCaptain body"}}],pageInfo:{hasNextPage:false,endCursor:null}}}}}' + fi ;; *'itemId=old-item'* ) jq -n '{data:{node:{content:{__typename:"DraftIssue",title:"Captain-edited Alpha",body:"\`alpha-task\`\\n\\nCaptain body"},fieldValues:{nodes:[ {name:"Queued",optionId:"queued-status",field:{name:"Status"}}, @@ -307,6 +314,28 @@ grep -F $'alpha-task\told-item\told-draft\tdraft' "$case_dir/home/state/helm-car || fail "cacheless move did not rebuild the source card identity" pass "cacheless moves discover live source cards" +case_dir="$TMP_ROOT/cacheless-duplicate-move" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-owner",number:999,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +if FM_HELM_DUPLICATE_SOURCE=1 FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >/dev/null 2>&1; then + fail "cacheless duplicate source move unexpectedly succeeded" +fi +jq -e '.projects.alpha.state == "active" and .projects.alpha.number == 999' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "duplicate source move changed the routing mapping" +[ ! -e "$case_dir/home/state/helm-cards.tsv" ] \ + || fail "duplicate source move rebuilt a partial card cache" +[ ! -e "$case_dir/home/state/helm-moves.tsv" ] \ + || fail "duplicate source move published a partial move ledger" +pass "cacheless moves reject duplicate source cards" + # Linking a second project onto a board another project already linked must # add the missing Project option, not refuse the board as "incomplete" # (spec decision 6: several mapping entries sharing one board is normal). From 76643b7446b46183b8cc1d136e6e15ceddcdb5a5 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:39:40 +0000 Subject: [PATCH 10/23] no-mistakes(review): Preserve source sync and refresh move metadata --- bin/fm-helm-lib.sh | 6 +---- bin/fm-helm-project-map.sh | 6 ++--- tests/fm-helm-project-map.test.sh | 39 +++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 8 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index fc0bc795065..12f48ff4188 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -625,10 +625,7 @@ fm_helm_plan_program() { end else $matches[0] as $card - | if $old != null and $old.item == $card.id - and (($d.board.owner != $board_owner) or ($d.board.number != $board_number)) then - {phase: "record", action: "none", task: $r.id, item: $card.id, cache: old_cache($old), note: $d.note} - else + | ($card.content.__typename == "Issue") as $is_issue | ($card.content.id // "") as $node | ($d.title | @base64) as $dt @@ -759,7 +756,6 @@ fm_helm_plan_program() { fields: ($writes | map({name, value, option}))} | tojson)} end end - end end ]; def missing_entries: if ($records | length) == 0 then [] else diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 98de48918da..956e319d144 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -575,9 +575,9 @@ map_move() { exit 0 fi now=$(date -u +%Y-%m-%dT%H:%M:%SZ) - jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" \ + jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" --arg title "$BOARD_TITLE" --arg url "$BOARD_URL" \ --arg from_owner "$source_owner" --argjson from_number "$source_number" --arg now "$now" \ - '.projects[$p] = ((.projects[$p] // {}) + {owner:$owner,number:$number,state:"migrating",move:{from:{owner:$from_owner,number:$from_number},to:{owner:$owner,number:$number},requested_at:$now,confirmed:true},orphan_hold_task:null})' \ + '.projects[$p] = ((.projects[$p] // {}) + {owner:$owner,number:$number,title:$title,url:$url,state:"migrating",move:{from:{owner:$from_owner,number:$from_number},to:{owner:$owner,number:$number},requested_at:$now,confirmed:true},orphan_hold_task:null})' \ "$TMP_DIR/map.json" >"$TMP_DIR/map.next" || fail "could not prepare the migration mapping" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish the migration mapping" @@ -586,7 +586,7 @@ map_move() { fi remaining=$(move_execute) if [ "$remaining" -eq 0 ]; then - jq --arg p "$project" '.projects[$p].state = "active" | .projects[$p].move = null' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + jq --arg p "$project" --arg title "$BOARD_TITLE" --arg url "$BOARD_URL" '.projects[$p].state = "active" | .projects[$p].move = null | .projects[$p].title = $title | .projects[$p].url = $url' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ || fail "could not finish the migration mapping" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish the completed migration mapping" diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index 4d2f9e22c5c..e86d1c05935 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -287,6 +287,9 @@ grep -F 'item-delete 999 --owner fixture-owner --id old-item' "$case_dir/gh-axi. jq -e '.projects.alpha.state == "active" and .projects.alpha.move == null and .projects.alpha.number == 998' \ "$case_dir/home/data/helm-project-map.json" >/dev/null \ || fail "move did not finish the routing mapping" +jq -e '.projects.alpha.title == "Helm Default" and .projects.alpha.url == "https://github.com/users/fixture-owner/projects/999"' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "move did not refresh the destination routing metadata" grep -F $'alpha-task\tmoved-item\told-draft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ || fail "move did not update the cache to the destination item" grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item\tcomplete' \ @@ -294,6 +297,42 @@ grep -F $'alpha-task\tfixture-owner\t999\told-item\tfixture-org\t998\tmoved-item || fail "move ledger did not retain its complete add/delete history" pass "confirmed moves preserve card history in the resumable ledger" +case_dir="$TMP_ROOT/future-only-source-sync" +mkdir -p "$case_dir/plan" +title_b64=$(printf '%s' 'Alpha task' | base64 | tr -d '\n') +body_b64=$(printf '%s\n\n%s' '`alpha-task`' 'Body' | base64 | tr -d '\n') +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-task old-item old-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ + >"$case_dir/plan/cards.tsv" +jq -n '[{id:"alpha-task",state:"done",home_path:"/tmp/backlog",desired:{project:"alpha",kind:"ship",priority:"P3",status:"Done",title:"Alpha task",body:"`alpha-task`\n\nBody",board:{owner:"fixture-org",number:998},note:""}}]' \ + >"$case_dir/plan/desired.json" +jq -n '{data:{user:{projectV2:{fields:{nodes:[ + {__typename:"ProjectV2SingleSelectField",id:"status-field",name:"Status",options:[{id:"queued-status",name:"Queued"},{id:"done-status",name:"Done"},{id:"waiting-status",name:"Waiting on you"}]}, + {__typename:"ProjectV2SingleSelectField",id:"project-field",name:"Project",options:[{id:"alpha-project",name:"alpha"}]}, + {__typename:"ProjectV2SingleSelectField",id:"kind-field",name:"Kind",options:[{id:"ship-kind",name:"ship"}]}, + {__typename:"ProjectV2SingleSelectField",id:"priority-field",name:"Priority",options:[{id:"p3-priority",name:"P3"}]} +]},items:{nodes:[{id:"old-item",content:{__typename:"DraftIssue",id:"old-draft",title:"Alpha task",body:"`alpha-task`\n\nBody"},fieldValues:{nodes:[ + {name:"Queued",optionId:"queued-status",field:{name:"Status"}}, + {name:"alpha",optionId:"alpha-project",field:{name:"Project"}}, + {name:"ship",optionId:"ship-kind",field:{name:"Kind"}}, + {name:"P3",optionId:"p3-priority",field:{name:"Priority"}} +]}}]}}}}}' >"$case_dir/plan/board.json" +: >"$case_dir/plan/deleted.tsv" +: >"$case_dir/plan/markers.tsv" +: >"$case_dir/plan/divergences.tsv" +: >"$case_dir/plan/fps.tsv" +FM_ROOT_OVERRIDE="$ROOT" bash -c ' + . "$1" + jq -j --slurpfile desired "$2" --rawfile cards "$3" --rawfile deleted "$4" --rawfile markers "$5" --rawfile divergences "$6" --rawfile fps "$7" \ + --arg force 0 --arg dispatch_status "In flight" --arg now 2 --arg tsv_existed true --arg retain_source 0 --arg retain_project "" \ + --arg board_owner fixture-owner --argjson board_number 999 --arg default_owner fixture-owner --argjson default_number 999 \ + "$(fm_helm_plan_program)" "$8" +' _ "$LIB" "$case_dir/plan/desired.json" "$case_dir/plan/cards.tsv" "$case_dir/plan/deleted.tsv" "$case_dir/plan/markers.tsv" "$case_dir/plan/divergences.tsv" "$case_dir/plan/fps.tsv" "$case_dir/plan/board.json" >"$case_dir/plan/output.nul" \ + || fail "future-only source-board planning failed" +[ "$(tr '\0' '\n' <"$case_dir/plan/output.nul" | sed -n '2p')" = update ] \ + || fail "future-only routing suppressed an existing source card update" +pass "future-only routing keeps existing source cards synchronized" + # A rebuilt cache must not make a confirmed move overlook source-board cards. case_dir="$TMP_ROOT/cacheless-move" seed_home "$case_dir" From ee790c42a0212ff039068e86058925d4a625a563 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:45:22 +0000 Subject: [PATCH 11/23] no-mistakes(review): Avoid no-op retention and clarify removed moves --- bin/fm-helm-project-map.sh | 7 +++++-- tests/fm-helm-project-map.test.sh | 7 +++++++ 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 956e319d144..a96b6a6e16a 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -250,7 +250,7 @@ map_link() { now=$(date -u +%Y-%m-%dT%H:%M:%SZ) jq --arg p "$project" --arg owner "$BOARD_OWNER" --argjson number "$BOARD_NUMBER" --arg prior_owner "$prior_owner" --argjson prior_number "${prior_number:-0}" \ --arg title "$BOARD_TITLE" --arg url "$BOARD_URL" --arg now "$now" \ - '.projects[$p] = {owner:$owner,number:$number,title:$title,url:$url,state:"active",linked_at:$now,move:null,orphan_hold_task:null} | .nudges |= del(.[$p]) | if $prior_owner != "" and $prior_number > 0 then .retention = {project:$p,owner:$prior_owner,number:$prior_number} else . end' \ + '.projects[$p] = {owner:$owner,number:$number,title:$title,url:$url,state:"active",linked_at:$now,move:null,orphan_hold_task:null} | .nudges |= del(.[$p]) | if $prior_owner != "" and $prior_number > 0 and ($prior_owner != $owner or $prior_number != $number) then .retention = {project:$p,owner:$prior_owner,number:$prior_number} else . end' \ "$TMP_DIR/map.json" >"$TMP_DIR/map.next" || fail "could not prepare the Helm routing entry" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" @@ -425,7 +425,10 @@ move_execute() { [ -n "$task" ] || continue case $'\n'$MOVE_TASKS$'\n' in *$'\n'"$task"$'\n'*) ;; - *) continue ;; + *) + [ "$phase" = complete ] || printf 'fm-helm-project-map: %s is no longer in the backlog; re-add it or manually clean up state/helm-moves.tsv before resuming its move\n' "$task" >&2 + continue + ;; esac [ "$phase" = complete ] && continue if [ "$phase" = pending ]; then diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index e86d1c05935..8f0b175d8b3 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -462,6 +462,13 @@ grep -F 'name0=other' "$case_dir/gh-axi.log" >/dev/null \ jq -e '.projects.alpha.number == 998 and .projects.beta.number == 998 and .projects.beta.state == "active"' \ "$case_dir/home/data/helm-project-map.json" >/dev/null \ || fail "a second project did not join the first project's board in the mapping" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" \ + FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" link alpha --existing fixture-org/998 >/dev/null 2>&1 \ + || fail "relinking a project to its current board exited nonzero" +jq -e '.retention == null' "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "a no-op link left pending retention" pass "linking a second project onto an already-linked board provisions its option instead of refusing" # Reconciliation covers two sides of a broken mapping through the existing From 1c98adf20169beebd35def5a5f6bba5ccd35983b Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 05:49:12 +0000 Subject: [PATCH 12/23] no-mistakes(review): Preserve retained updates and pending moves --- bin/fm-helm-lib.sh | 2 +- bin/fm-helm-project-map.sh | 3 ++- tests/fm-helm-project-map.test.sh | 39 ++++++++++++++++++++++++++++++- 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index 12f48ff4188..f0499e27520 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -730,7 +730,7 @@ fm_helm_plan_program() { (if $title_write then $dt elif $title_conflict then $bt elif $rebuilt or $ct == $dt or ($title_board_changed | not) then $dt else $bt end) + "\t" + (if $body_write then $db elif $body_conflict then $bb elif $rebuilt or $cb == $db or ($body_board_changed | not) then $db else $bb end) + "\t" + $now + "\t" + $board_owner + "\t" + ($board_number | tostring)) as $cache | {phase: "record", - action: (if $retain then "none" elif $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), + action: (if $text.draft != "" or ($writes | length) > 0 then "update" else "none" end), task: $r.id, item: $card.id, cache: $cache, draft: $text.draft, title: (if $title_write then $d.title else $card_title end), body: (if $body_write then $d.body else $card_body end), fields: $writes, wakes: ($text.wakes + $disp.wakes + $st.wakes diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index a96b6a6e16a..ae7db97e3e2 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -427,6 +427,7 @@ move_execute() { *$'\n'"$task"$'\n'*) ;; *) [ "$phase" = complete ] || printf 'fm-helm-project-map: %s is no longer in the backlog; re-add it or manually clean up state/helm-moves.tsv before resuming its move\n' "$task" >&2 + [ "$phase" = complete ] || remaining=$((remaining + 1)) continue ;; esac @@ -605,7 +606,7 @@ map_unlink() { prior=$(jq -c --arg p "$project" 'if .retention.project == $p then .retention else .projects[$p] // null end' "$TMP_DIR/map.json") prior_owner=$(printf '%s\n' "$prior" | jq -r '.owner // empty') prior_number=$(printf '%s\n' "$prior" | jq -r '.number // empty') - jq --arg p "$project" --arg owner "$prior_owner" --argjson number "${prior_number:-0}" '.projects |= del(.[$p]) | .nudges |= del(.[$p]) | if $owner != "" and $number > 0 then .retention = {project:$p,owner:$owner,number:$number} else . end' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ + jq --arg p "$project" --arg owner "$prior_owner" --argjson number "${prior_number:-0}" --arg default_owner "$DEFAULT_OWNER" --argjson default_number "$DEFAULT_NUMBER" '.projects |= del(.[$p]) | .nudges |= del(.[$p]) | if $owner != "" and $number > 0 and ($owner != $default_owner or $number != $default_number) then .retention = {project:$p,owner:$owner,number:$number} else . end' "$TMP_DIR/map.json" >"$TMP_DIR/map.next" \ || fail "could not prepare the unlink" mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index 8f0b175d8b3..02cb1cdb681 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -324,7 +324,7 @@ jq -n '{data:{user:{projectV2:{fields:{nodes:[ FM_ROOT_OVERRIDE="$ROOT" bash -c ' . "$1" jq -j --slurpfile desired "$2" --rawfile cards "$3" --rawfile deleted "$4" --rawfile markers "$5" --rawfile divergences "$6" --rawfile fps "$7" \ - --arg force 0 --arg dispatch_status "In flight" --arg now 2 --arg tsv_existed true --arg retain_source 0 --arg retain_project "" \ + --arg force 0 --arg dispatch_status "In flight" --arg now 2 --arg tsv_existed true --arg retain_source 1 --arg retain_project alpha \ --arg board_owner fixture-owner --argjson board_number 999 --arg default_owner fixture-owner --argjson default_number 999 \ "$(fm_helm_plan_program)" "$8" ' _ "$LIB" "$case_dir/plan/desired.json" "$case_dir/plan/cards.tsv" "$case_dir/plan/deleted.tsv" "$case_dir/plan/markers.tsv" "$case_dir/plan/divergences.tsv" "$case_dir/plan/fps.tsv" "$case_dir/plan/board.json" >"$case_dir/plan/output.nul" \ @@ -333,6 +333,27 @@ FM_ROOT_OVERRIDE="$ROOT" bash -c ' || fail "future-only routing suppressed an existing source card update" pass "future-only routing keeps existing source cards synchronized" +case_dir="$TMP_ROOT/removed-move-task" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +printf '%s\n' '# Backlog' '' '## In flight' '## Queued' '## Done' >"$case_dir/home/data/backlog.md" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,title:"Alpha",state:"migrating",move:{from:{owner:"fixture-owner",number:999},to:{owner:"fixture-org",number:998},confirmed:true}}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +printf '%s\t%s\t%s\t%s\t%s\t%s\t\tpending\t1\n' \ + alpha-task fixture-owner 999 old-item fixture-org 998 >"$case_dir/home/state/helm-moves.tsv" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --yes >"$case_dir/output" 2>&1 \ + || fail "resuming a removed move task exited nonzero" +grep -F 're-add it or manually clean up state/helm-moves.tsv' "$case_dir/output" >/dev/null \ + || fail "removed move task did not explain the recovery options" +jq -e '.projects.alpha.state == "migrating"' "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "removed move task incorrectly completed the migration" +pass "removed move tasks remain resumable" + # A rebuilt cache must not make a confirmed move overlook source-board cards. case_dir="$TMP_ROOT/cacheless-move" seed_home "$case_dir" @@ -471,6 +492,22 @@ jq -e '.retention == null' "$case_dir/home/data/helm-project-map.json" >/dev/nul || fail "a no-op link left pending retention" pass "linking a second project onto an already-linked board provisions its option instead of refusing" +case_dir="$TMP_ROOT/noop-unlink" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-owner",number:999,title:"Default",state:"active",move:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" unlink alpha >/dev/null 2>&1 \ + || fail "unlinking a default-board mapping exited nonzero" +jq -e '.projects.alpha == null and .retention == null' "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "a no-op unlink left pending retention" +pass "unlinking a default-board mapping leaves no retention" + # Reconciliation covers two sides of a broken mapping through the existing # captain-hold path: a board that disappeared and a local project that left the # registry. It also refreshes title drift on another board in the same pass. From 88415eb300514d5d71780451cfdf9b3cd49476eb Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 07:37:53 +0000 Subject: [PATCH 13/23] no-mistakes(review): Fix Helm move source resolution and duplicate-card cache merge --- bin/fm-helm-project-map.sh | 47 +++++++++++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index ae7db97e3e2..845ceef670b 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -255,8 +255,13 @@ map_link() { mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" printf 'linked: %s -> %s/%s "%s"\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" "$BOARD_TITLE" - printf 'link changes future cards only; use move %s --existing %s/%s --yes to relocate existing cards.\n' \ - "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$BOARD_OWNER/$BOARD_NUMBER" ]; then + printf 'link changes future cards only; existing cards remain on %s/%s until you run move %s --existing %s/%s --yes to relocate them.\n' \ + "$prior_owner" "$prior_number" "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + else + printf 'link changes future cards only; use move %s --existing %s/%s --yes to relocate existing cards.\n' \ + "$project" "$BOARD_OWNER" "$BOARD_NUMBER" + fi map_release_lock || fail "could not release the Helm routing lock" if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$BOARD_OWNER/$BOARD_NUMBER" ]; then FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true @@ -487,6 +492,9 @@ map_move() { board_view "$BOARD_OWNER" "$BOARD_NUMBER" || fail "migration destination $BOARD_OWNER/$BOARD_NUMBER does not resolve" printf 'resuming confirmed move for %s -> %s/%s\n' "$project" "$BOARD_OWNER" "$BOARD_NUMBER" yes=1 + ids="$TMP_DIR/task-ids" + backlog_task_ids "$project" "$ids" + MOVE_TASKS=$(cat -- "$ids") else project_registered "$project" || fail "local project '$project' is not registered in data/projects.md" shift 2 @@ -500,10 +508,25 @@ map_move() { esac shift done - source_owner=$(printf '%s\n' "$entry" | jq -r '.owner // empty') - source_number=$(printf '%s\n' "$entry" | jq -r '.number // empty') - [ -n "$source_owner" ] || source_owner=$DEFAULT_OWNER - [ -n "$source_number" ] || source_number=$DEFAULT_NUMBER + ids="$TMP_DIR/task-ids" + backlog_task_ids "$project" "$ids" + MOVE_TASKS=$(cat -- "$ids") + # A just-completed link/unlink already points the routing entry at the + # destination; the project's cards are still sitting wherever the cache + # last indexed them, so that identity (not the entry) is the real source. + source_owner= + source_number= + if [ -f "$CARDS_FILE" ] && [ -s "$ids" ]; then + IFS=$'\t' read -r source_owner source_number < <(awk -F '\t' -v ids_file="$ids" ' + FILENAME == ids_file { wanted[$1] = 1; next } + NF >= 11 && wanted[$1] && $10 != "" && $11 != "" { print $10 "\t" $11; exit }' "$ids" "$CARDS_FILE") + fi + if [ -z "$source_owner" ] || [ -z "$source_number" ]; then + source_owner=$(printf '%s\n' "$entry" | jq -r '.owner // empty') + source_number=$(printf '%s\n' "$entry" | jq -r '.number // empty') + [ -n "$source_owner" ] || source_owner=$DEFAULT_OWNER + [ -n "$source_number" ] || source_number=$DEFAULT_NUMBER + fi if [ -n "$default_dest" ]; then BOARD_OWNER=$DEFAULT_OWNER; BOARD_NUMBER=$DEFAULT_NUMBER board_view "$BOARD_OWNER" "$BOARD_NUMBER" || fail "default Helm Project $BOARD_OWNER/$BOARD_NUMBER does not resolve" @@ -517,9 +540,6 @@ map_move() { fi fi [ "$source_owner/$source_number" != "$BOARD_OWNER/$BOARD_NUMBER" ] || fail "source and destination boards are the same" - ids="$TMP_DIR/task-ids" - backlog_task_ids "$project" "$ids" - MOVE_TASKS=$(cat -- "$ids") : >"$TMP_DIR/moves.tsv" [ -f "$MOVES_FILE" ] && [ ! -L "$MOVES_FILE" ] && cp -- "$MOVES_FILE" "$TMP_DIR/moves.tsv" if [ -f "$CARDS_FILE" ]; then @@ -547,7 +567,7 @@ map_move() { awk -F '\t' -v discovered="$TMP_DIR/source-cards.tsv" -v owner="$source_owner" -v number="$source_number" -v now="$(date +%s)" ' FILENAME == discovered { found[$1]=$0; next } { - if ($1 in found && $10 == owner && $11 == number) { + if ($1 in found) { split(found[$1], row, "\t") print $1 "\t" row[2] "\t" row[3] "\t" row[4] "\t\t\t\t\t" now "\t" owner "\t" number written[$1]=1 @@ -611,7 +631,12 @@ map_unlink() { mv -f -- "$TMP_DIR/map.next" "$TMP_DIR/map.json" map_publish || fail "could not publish data/helm-project-map.json" printf 'unlinked: %s -> default Helm board %s/%s\n' "$project" "$DEFAULT_OWNER" "$DEFAULT_NUMBER" - printf 'unlink changes future cards only; use move %s --default --yes to relocate existing cards.\n' "$project" + if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$DEFAULT_OWNER/$DEFAULT_NUMBER" ]; then + printf 'unlink changes future cards only; existing cards remain on %s/%s until you run move %s --default --yes to relocate them.\n' \ + "$prior_owner" "$prior_number" "$project" + else + printf 'unlink changes future cards only; use move %s --default --yes to relocate existing cards.\n' "$project" + fi map_release_lock || fail "could not release the Helm routing lock" if [ -n "$prior_owner" ] && [ -n "$prior_number" ] && [ "$prior_owner/$prior_number" != "$DEFAULT_OWNER/$DEFAULT_NUMBER" ]; then FM_HELM_RETAIN_BOARD="$prior_owner/$prior_number" FM_HELM_RETAIN_PROJECT="$project" "$SCRIPT_DIR/fm-helm-sync.sh" || true From 968992a9a3cf6cf206663c144e60f645013885cd Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 07:49:33 +0000 Subject: [PATCH 14/23] no-mistakes(review): Add regression tests for move source resolution and cache merge fixes --- tests/fm-helm-project-map.test.sh | 73 +++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index 02cb1cdb681..c3c920650d3 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -396,6 +396,79 @@ jq -e '.projects.alpha.state == "active" and .projects.alpha.number == 999' \ || fail "duplicate source move published a partial move ledger" pass "cacheless moves reject duplicate source cards" +# After link/unlink, the routing entry already points at the destination +# board while the existing card is still cached on the prior board. move must +# resolve its source from that cached identity, not the now-overwritten entry +# (otherwise source == destination and move always refuses to run). +case_dir="$TMP_ROOT/move-after-link" +seed_home "$case_dir" +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +title_b64=$(printf '%s' 'Alpha task' | base64 | tr -d '\n') +# shellcheck disable=SC2016 # backticks are card-body literals, not expansions. +body_b64=$(printf '%s' '`alpha-task`\n\nBody' | base64 | tr -d '\n') +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-task old-item old-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ + >"$case_dir/home/state/helm-cards.tsv" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >"$case_dir/output" 2>&1 \ + || fail "move after link exited nonzero: $(cat "$case_dir/output")" +grep -qF 'source and destination boards are the same' "$case_dir/output" \ + && fail "move after link still resolved its source from the overwritten routing entry" +grep -F 'item-delete 999 --owner fixture-owner --id old-item' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move after link did not delete the card from the prior cached board" +jq -e '.projects.alpha.state == "active" and .projects.alpha.owner == "fixture-org" and .projects.alpha.number == 998' \ + "$case_dir/home/data/helm-project-map.json" >/dev/null \ + || fail "move after link left the routing mapping incomplete" +grep -F $'alpha-task\tmoved-item\told-draft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "move after link did not update the cache to the destination item" +pass "move resolves its source from the cached card identity after link overwrites the entry" + +# A stale cache row for a task pointing at a board other than the move's +# source must be overwritten by live discovery, not duplicated into a second +# row for the same task id. A second, correctly-cached anchor task keeps +# source-board resolution pinned to fixture-owner/999 so the stale row is +# exercised through discovery rather than mistaken for the source itself. +case_dir="$TMP_ROOT/cacheless-move-stale-identity" +seed_home "$case_dir" +cat >"$case_dir/home/data/backlog.md" <<'EOF' +# Backlog + +## In flight +## Queued +- [ ] alpha-task - Alpha task (repo: alpha) (kind: ship) (since: 2026-09-10) +- [ ] alpha-anchor - Alpha anchor (repo: alpha) (kind: ship) (since: 2026-09-10) +## Done +EOF +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-owner",number:999,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-anchor anchor-item anchor-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ + >"$case_dir/home/state/helm-cards.tsv" +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-task stale-item stale-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 stale-org 5 \ + >>"$case_dir/home/state/helm-cards.tsv" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_FAIL_DELETE=1 FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >/dev/null 2>&1 \ + || fail "cacheless move with a stale identity exited nonzero" +[ "$(awk -F '\t' '$1 == "alpha-task"' "$case_dir/home/state/helm-cards.tsv" | wc -l)" -eq 1 ] \ + || fail "a stale cache row was duplicated instead of overwritten by live discovery" +grep -F $'alpha-task\told-item\told-draft\tdraft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "the discovered source card did not replace the stale cache row" +grep -F $'\tstale-org\t5' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + && fail "the stale board identity was left behind alongside the discovered row" +pass "cacheless move discovery overwrites a stale cache row instead of duplicating it" + # Linking a second project onto a board another project already linked must # add the missing Project option, not refuse the board as "incomplete" # (spec decision 6: several mapping entries sharing one board is normal). From a0b3a86feed1ee5a35aaa42ebebeb49cc829d0f5 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 07:58:04 +0000 Subject: [PATCH 15/23] no-mistakes(review): Exclude destination-board rows when detecting Helm move source --- bin/fm-helm-project-map.sh | 18 +++++++++++--- tests/fm-helm-project-map.test.sh | 40 +++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 3 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 845ceef670b..1e2217880a1 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -480,7 +480,7 @@ move_execute() { } map_move() { - local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' default_dest='' yes=0 entry source_owner source_number count now ids + local project=$1 title=${2:-} owner=$DEFAULT_OWNER existing='' default_dest='' yes=0 entry source_owner source_number count now ids exclude_owner exclude_number [ -n "$project" ] || fail "move requires a local project name" entry=$(map_entry "$project") if [ "$(printf '%s\n' "$entry" | jq -r '.state // empty')" = migrating ]; then @@ -514,12 +514,24 @@ map_move() { # A just-completed link/unlink already points the routing entry at the # destination; the project's cards are still sitting wherever the cache # last indexed them, so that identity (not the entry) is the real source. + # Exclude rows already on the known destination so a task that already + # landed there (e.g. a newer card routed by the just-updated mapping) + # cannot be mistaken for the still-unmigrated source board. + exclude_owner= + exclude_number= + if [ -n "$default_dest" ]; then + exclude_owner=$DEFAULT_OWNER + exclude_number=$DEFAULT_NUMBER + elif [ -n "$existing" ] && parse_board_ref "$existing"; then + exclude_owner=$BOARD_OWNER + exclude_number=$BOARD_NUMBER + fi source_owner= source_number= if [ -f "$CARDS_FILE" ] && [ -s "$ids" ]; then - IFS=$'\t' read -r source_owner source_number < <(awk -F '\t' -v ids_file="$ids" ' + IFS=$'\t' read -r source_owner source_number < <(awk -F '\t' -v ids_file="$ids" -v ex_owner="$exclude_owner" -v ex_number="$exclude_number" ' FILENAME == ids_file { wanted[$1] = 1; next } - NF >= 11 && wanted[$1] && $10 != "" && $11 != "" { print $10 "\t" $11; exit }' "$ids" "$CARDS_FILE") + NF >= 11 && wanted[$1] && $10 != "" && $11 != "" && !($10 == ex_owner && $11 == ex_number) { print $10 "\t" $11; exit }' "$ids" "$CARDS_FILE") fi if [ -z "$source_owner" ] || [ -z "$source_number" ]; then source_owner=$(printf '%s\n' "$entry" | jq -r '.owner // empty') diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index c3c920650d3..756b05ec4e7 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -469,6 +469,46 @@ grep -F $'\tstale-org\t5' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ && fail "the stale board identity was left behind alongside the discovered row" pass "cacheless move discovery overwrites a stale cache row instead of duplicating it" +# A task already cached on the destination board (e.g. a newer card the +# just-updated mapping routed straight there) must never be picked as the +# move's source just because its row happens to sort first in the cache. +case_dir="$TMP_ROOT/move-source-skips-destination-rows" +seed_home "$case_dir" +cat >"$case_dir/home/data/backlog.md" <<'EOF' +# Backlog + +## In flight +## Queued +- [ ] alpha-anchor - Alpha anchor (repo: alpha) (kind: ship) (since: 2026-09-10) +- [ ] alpha-task - Alpha task (repo: alpha) (kind: ship) (since: 2026-09-10) +## Done +EOF +write_empty_board "$case_dir/board.json" +jq -n '{version:1,projects:{alpha:{owner:"fixture-org",number:998,title:"Alpha",state:"active",linked_at:"2026-09-10T00:00:00Z",move:null,orphan_hold_task:null}},nudges:{}}' \ + >"$case_dir/home/data/helm-project-map.json" +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-anchor dest-item dest-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-org 998 \ + >"$case_dir/home/state/helm-cards.tsv" +printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ + alpha-task old-item old-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ + >>"$case_dir/home/state/helm-cards.tsv" +fb=$(install_gh_axi "$case_dir" no) +assert_fake_tools "$fb" +mkdir -p "$case_dir/gh-config" +FM_HELM_GH_AXI_LOG="$case_dir/gh-axi.log" FM_HELM_GH_LOG="$case_dir/gh.log" \ + GH_CONFIG_DIR="$case_dir/gh-config" GH_HOST=127.0.0.1:9 FM_HELM_BOARD_JSON="$case_dir/board.json" PATH="$fb:$PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$ROOT" \ + "$MAP" move alpha --existing fixture-org/998 --yes >"$case_dir/output" 2>&1 \ + || fail "move exited nonzero when an unmigrated card still needed to move: $(cat "$case_dir/output")" +grep -qF 'source and destination boards are the same' "$case_dir/output" \ + && fail "move mistook a card already on the destination board for the source" +grep -F 'item-delete 999 --owner fixture-owner --id old-item' "$case_dir/gh-axi.log" >/dev/null \ + || fail "move did not relocate the still-unmigrated card off its real source board" +grep -F $'alpha-anchor\tdest-item\tdest-draft' "$case_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "the already-migrated card was disturbed even though it was not part of this move" +[ "$(awk -F '\t' '$1 == "alpha-anchor"' "$case_dir/home/state/helm-cards.tsv" | wc -l)" -eq 1 ] \ + || fail "the already-migrated card's cache row was duplicated" +pass "move source detection skips cache rows already on the destination board" + # Linking a second project onto a board another project already linked must # add the missing Project option, not refuse the board as "incomplete" # (spec decision 6: several mapping entries sharing one board is normal). From c416c74d28aa139cd12ce2a0878efbc90f014278 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 15:05:29 +0000 Subject: [PATCH 16/23] no-mistakes(test): Classify Helm routing documentation audiences --- docs/documentation-audiences.json | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/documentation-audiences.json b/docs/documentation-audiences.json index b10e3fb3bf8..d8e46ae713c 100644 --- a/docs/documentation-audiences.json +++ b/docs/documentation-audiences.json @@ -440,6 +440,18 @@ "path": "docs/architecture.md", "audience": "maintainer-architecture" }, + { + "path": "docs/1.architecture/helm-project-routing.md", + "audience": "maintainer-architecture" + }, + { + "path": "docs/2.api/guides/helm-project-routing.md", + "audience": "operator-current" + }, + { + "path": "docs/2.api/helm-project-map.md", + "audience": "operator-current" + }, { "path": "docs/arm-pretool-check.md", "audience": "maintainer-architecture" @@ -524,6 +536,10 @@ "path": "docs/configuration.md", "audience": "operator-current" }, + { + "path": "docs/4.configuration.md", + "audience": "operator-current" + }, { "path": "docs/captain-hold-lifecycle.md", "audience": "maintainer-architecture" From 20ace54ffeffbc0bbe0fea8ac15e355dd1d6fc5a Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 15:35:54 +0000 Subject: [PATCH 17/23] no-mistakes(test): Fix Helm deletion board-number comparison --- bin/fm-helm-lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index f0499e27520..8d505b38b32 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -785,7 +785,7 @@ fm_helm_plan_program() { if $tsv_existed != "true" then [] else [ $old_rows[] as $o | if $o.task == "" or $o.item == "" then empty - elif $o.owner != $board_owner or $o.number != $board_number then empty + elif $o.owner != $board_owner or ($o.number | tostring) != ($board_number | tostring) then empty elif $item_set[$o.item] then empty elif $line1_set["`" + $o.task + "`"] then empty else From 00ce4faec7c863848ac3373605ffd29ebb7b5248 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 16:06:47 +0000 Subject: [PATCH 18/23] no-mistakes(document): Correct Helm routing documentation --- docs/2.api/guides/helm-project-routing.md | 10 +++++----- docs/4.configuration.md | 7 ++++--- docs/configuration.md | 12 ++++++------ 3 files changed, 15 insertions(+), 14 deletions(-) diff --git a/docs/2.api/guides/helm-project-routing.md b/docs/2.api/guides/helm-project-routing.md index e1b675a23fa..62337a9083b 100644 --- a/docs/2.api/guides/helm-project-routing.md +++ b/docs/2.api/guides/helm-project-routing.md @@ -8,8 +8,8 @@ tags: ['helm', 'setup'] ## Overview Helm routing is opt-in. Configure a default board, register local projects in -`data/projects.md`, and use the mapping command when a project needs a separate -board or should share another board intentionally. +their local home's `data/projects.md`, and use the mapping command when a +project needs a separate board or should share another board intentionally. ## Usage @@ -30,9 +30,9 @@ bin/fm-helm-project-map.sh move firetabs --existing geojitsu/5 --yes ### `data/helm-project-map.json` -Version 1 stores `owner`, `number`, `title`, `url`, `state`, `linked_at`, -`move`, and `orphan_hold_task` for each project, plus nudge cooldowns under -`nudges`. The file is main-home-only and mode `0600`. +Version 1 uses a `projects` object keyed by registered project name, plus nudge +cooldowns under `nudges`. Project entries are `active` or `migrating`. The file +is main-home-only and mode `0600`. ## Notes diff --git a/docs/4.configuration.md b/docs/4.configuration.md index 5a1f303838e..f77d0809370 100644 --- a/docs/4.configuration.md +++ b/docs/4.configuration.md @@ -26,9 +26,10 @@ to `In flight`; `reconcile_interval_hours` defaults to `6`. ### `data/helm-project-map.json` Version 1 contains `projects` and `nudges`. A project entry's `state` is -`active`, `provisioning`, or `migrating`; absent entries use the default board. +`active` or `migrating`; absent entries use the default board. ## Notes -The map is keyed by the exact project identity from `data/projects.md`, not by a -checkout path. Both the map and move ledger are mode `0600` local state. +The map is keyed by the exact project identity from a local home's +`data/projects.md`, not by a checkout path. Both the map and move ledger are +mode `0600` local state. diff --git a/docs/configuration.md b/docs/configuration.md index e4073d689d9..0a1405b0bc3 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -130,7 +130,7 @@ The accepted configuration fields are `owner`, `number`, optional `dispatch_stat The configured GitHub account needs the `project` scope, which provides the project read and write access used by the sync. An absent `config/helm.json` makes the script exit 0 without reading any backlog or contacting GitHub. -The sync discovers every local secondmate home from `data/secondmates.md` and reconciles the union of every home's `data/backlog.md` against grouped boards. A project named in `data/projects.md` can route its cards to its mapped `owner`/`number`; unmapped projects use the configured default board. Multiple projects may intentionally share one mapped board. +The sync discovers every local secondmate home from `data/secondmates.md` and reconciles the union of every home's `data/backlog.md` against grouped boards. A project registered in a local home's `data/projects.md` can route its cards to its mapped `owner`/`number`; unmapped projects use the configured default board. Multiple projects may intentionally share one mapped board. Remote secondmate homes are not handled yet; see the "Remote homes" note in `bin/fm-helm-lib.sh`. Bootstrap automatically registers `state/helm-sync.check.sh`, `state/helm-board.check.sh`, and `state/helm-reconcile.check.sh` with the main home's watcher while this configuration exists. The watcher runs the sync and authenticated board poll at its ordinary check cadence, and the combined backlog hash avoids a GitHub call when no local home changed. @@ -161,11 +161,11 @@ The sync records each request and unresolved field divergence with durable marke The sync rebuilds the cache silently from the board when absent, and the `bin/fm-helm-sync.sh` header owns its row format and the per-card publication rule. `data/helm-project-map.json` is the optional main-home routing map. Its -version-1 `projects` entries use the exact local project name as the key and -contain `owner`, `number`, `title`, `url`, `state`, `linked_at`, `move`, and -`orphan_hold_task`; an absent entry means the configured default board. The -`nudges` object remembers the once-per-30-days suggestion for an unlinked -project. The file is mode `0600` and is included in the sync debounce hash. +version-1 `projects` entries use the exact registered project name as the key. +An entry records its board identity and may be `active` or `migrating`; an +absent entry means the configured default board. The `nudges` object remembers +the once-per-30-days suggestion for an unlinked project. The file is mode +`0600` and is included in the sync debounce hash. Use `bin/fm-helm-project-map.sh list [--counts]` to inspect routing, `link <project> [<title>] [--owner <login>] [--existing <owner>/<number>]` to From 46f72b57fe2c63e9ef0d6cecfe0bcd19671f5649 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 16:10:27 +0000 Subject: [PATCH 19/23] no-mistakes(lint): Fix ShellCheck warnings in Helm routing --- bin/fm-helm-project-map.sh | 6 ++++-- tests/fm-helm-project-map.test.sh | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index 1e2217880a1..f4a8900fb81 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -335,7 +335,7 @@ move_discover_source_cards() { while :; do page=$((page + 1)) [ "$page" -le 50 ] || return 1 - response=$(gh-axi api graphql --field 'query=query($projectId:ID!,$cursor:String){node(id:$projectId){... on ProjectV2 {items(first:100,after:$cursor){nodes{id content{__typename ... on DraftIssue {id body} ... on Issue {id body}} pageInfo{hasNextPage endCursor}}}}}' --field "projectId=$board_id" --field "cursor=$cursor" 2>/dev/null) || return 1 + response=$(gh-axi api graphql --field "query=query(\$projectId:ID!,\$cursor:String){node(id:\$projectId){... on ProjectV2 {items(first:100,after:\$cursor){nodes{id content{__typename ... on DraftIssue {id body} ... on Issue {id body}} pageInfo{hasNextPage endCursor}}}}}" --field "projectId=$board_id" --field "cursor=$cursor" 2>/dev/null) || return 1 printf '%s\n' "$response" | jq -r --argjson wanted "$wanted" ' .data.node.items as $items | $items.nodes[]? @@ -588,7 +588,9 @@ map_move() { END { for (task in found) if (!written[task]) { split(found[task], row, "\t"); print task "\t" row[2] "\t" row[3] "\t" row[4] "\t\t\t\t\t" now "\t" owner "\t" number } }' \ "$TMP_DIR/source-cards.tsv" "$TMP_DIR/cards.current" >"$TMP_DIR/cards.discovered" \ || fail "could not stage discovered Helm card identities" - chmod 0600 "$TMP_DIR/cards.discovered" && mv -f -- "$TMP_DIR/cards.discovered" "$CARDS_FILE" \ + chmod 0600 "$TMP_DIR/cards.discovered" \ + || fail "could not publish discovered Helm card identities" + mv -f -- "$TMP_DIR/cards.discovered" "$CARDS_FILE" \ || fail "could not publish discovered Helm card identities" fi awk -F '\t' -v ids_file="$ids" -v moves_file="$TMP_DIR/moves.tsv" \ diff --git a/tests/fm-helm-project-map.test.sh b/tests/fm-helm-project-map.test.sh index 756b05ec4e7..c7b3b347cf9 100755 --- a/tests/fm-helm-project-map.test.sh +++ b/tests/fm-helm-project-map.test.sh @@ -300,7 +300,7 @@ pass "confirmed moves preserve card history in the resumable ledger" case_dir="$TMP_ROOT/future-only-source-sync" mkdir -p "$case_dir/plan" title_b64=$(printf '%s' 'Alpha task' | base64 | tr -d '\n') -body_b64=$(printf '%s\n\n%s' '`alpha-task`' 'Body' | base64 | tr -d '\n') +body_b64=$(printf '%s\n\n%s' "\`alpha-task\`" 'Body' | base64 | tr -d '\n') printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \ alpha-task old-item old-draft draft queued-status p3-priority "$title_b64" "$body_b64" 1 fixture-owner 999 \ >"$case_dir/plan/cards.tsv" From 31621ac67d9f836809ec955bb36cc791d36c3cd9 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 16:49:53 +0000 Subject: [PATCH 20/23] no-mistakes(review): fix: cast cached board number to string before jq comparison --- bin/fm-helm-lib.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/fm-helm-lib.sh b/bin/fm-helm-lib.sh index 8d505b38b32..5b1e08784b6 100755 --- a/bin/fm-helm-lib.sh +++ b/bin/fm-helm-lib.sh @@ -606,7 +606,7 @@ fm_helm_plan_program() { {phase: "record", action: "skip", task: $r.id, note: $d.note} elif $deleted_by_task[$r.id] != null then {phase: "record", action: "skip", task: $r.id, tombstone: $deleted_by_task[$r.id].line, note: $d.note} - elif $old != null and $old.item != "" and (($old.owner != $board_owner) or ($old.number != $board_number)) then + elif $old != null and $old.item != "" and (($old.owner != $board_owner) or (($old.number | tostring) != ($board_number | tostring))) then {phase: "record", action: "none", task: $r.id, cache: old_cache($old), note: $d.note} elif $old != null and $r.state != "done" and $old.item != "" and ($item_set[$old.item] | not) then {phase: "record", action: "skip", task: $r.id, note: $d.note} From b231cbf221faa8f98366770019d51da37058ef3d Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 17:04:18 +0000 Subject: [PATCH 21/23] no-mistakes(review): test: add regression test for board-number type-mismatch fix --- tests/fm-helm-sync.test.sh | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/fm-helm-sync.test.sh b/tests/fm-helm-sync.test.sh index 523c55284cd..e7525983175 100644 --- a/tests/fm-helm-sync.test.sh +++ b/tests/fm-helm-sync.test.sh @@ -543,6 +543,31 @@ fi || fail "a restored card did not clear its deletion tombstone" pass "a deleted card holds its live task for the captain and is not treated as new" +# A card that stays on the same board (its cached owner/number still match) +# but no longer carries its task's marker line must be recreated, not +# silently marked "retained on another board" by a stale identity match. +marker_dir="$TMP_ROOT/stale-marker" +mkdir -p "$marker_dir/home/config" "$marker_dir/home/data" "$marker_dir/home/state" +marker_fb=$(install_fakes "$marker_dir") +printf '{"owner":"fixture-owner","number":999}\n' > "$marker_dir/home/config/helm.json" +cat > "$marker_dir/home/data/backlog.md" <<'EOF' +# Backlog + +## Queued +- [ ] marker-task - Marker task (repo: fixture-firstmate) (kind: ship) (since: 2026-09-05) +## Done +EOF +board_json "$(jq -n --argjson item "$(draft_item marker-item marker-draft marker-task 'Marker task' 'x' Queued queued-status P3 p3-priority)" '[$item]')" > "$marker_dir/board.json" +run_sync "$marker_dir" "$marker_fb" >/dev/null 2>&1 || fail "stale-marker seed run failed" +grep -F $'marker-task\tmarker-item\t' "$marker_dir/home/state/helm-cards.tsv" >/dev/null \ + || fail "stale-marker seed did not record the identity cache row" +board_json "$(jq -n --argjson item "$(draft_item marker-item marker-draft other-marker 'Marker task' 'x' Queued queued-status P3 p3-priority)" '[$item]')" > "$marker_dir/board.json" +: > "$marker_dir/gh.log" +run_sync "$marker_dir" "$marker_fb" --force >/dev/null 2>&1 || fail "stale-marker follow-up run failed" +grep -qF 'addProjectV2DraftIssue' "$marker_dir/gh.log" && grep -qF 'title=Marker task' "$marker_dir/gh.log" \ + || fail "a same-board card that lost its task marker was silently retained instead of recreated" +pass "a same-board card that loses its task marker is recreated, not silently retained" + held_delete_dir="$TMP_ROOT/delete-already-held" mkdir -p "$held_delete_dir/home/config" "$held_delete_dir/home/data" "$held_delete_dir/home/state" held_delete_fb=$(install_fakes "$held_delete_dir") From 4014377998d69da71a92b60371b73f96b94559d6 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 17:53:09 +0000 Subject: [PATCH 22/23] no-mistakes(test): Skip stale post-move sync cache eviction --- bin/fm-helm-project-map.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/bin/fm-helm-project-map.sh b/bin/fm-helm-project-map.sh index f4a8900fb81..86ced16bcc4 100755 --- a/bin/fm-helm-project-map.sh +++ b/bin/fm-helm-project-map.sh @@ -632,7 +632,6 @@ map_move() { printf 'move partial: %s card(s) remain; rerun move or sync to resume.\n' "$remaining" fi map_release_lock || fail "could not release the Helm routing lock" - "$SCRIPT_DIR/fm-helm-sync.sh" || true } map_unlink() { From 5887896218419f9e18bc6d26dd919199be0d14c4 Mon Sep 17 00:00:00 2001 From: firstmate-crewmate <brokentao@gmail.com> Date: Wed, 16 Sep 2026 22:14:44 +0000 Subject: [PATCH 23/23] no-mistakes(lint): test: replace ambiguous && || chain with explicit if in fm-helm-sync test --- tests/fm-helm-sync.test.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/fm-helm-sync.test.sh b/tests/fm-helm-sync.test.sh index e7525983175..113c3bf2074 100644 --- a/tests/fm-helm-sync.test.sh +++ b/tests/fm-helm-sync.test.sh @@ -564,8 +564,9 @@ grep -F $'marker-task\tmarker-item\t' "$marker_dir/home/state/helm-cards.tsv" >/ board_json "$(jq -n --argjson item "$(draft_item marker-item marker-draft other-marker 'Marker task' 'x' Queued queued-status P3 p3-priority)" '[$item]')" > "$marker_dir/board.json" : > "$marker_dir/gh.log" run_sync "$marker_dir" "$marker_fb" --force >/dev/null 2>&1 || fail "stale-marker follow-up run failed" -grep -qF 'addProjectV2DraftIssue' "$marker_dir/gh.log" && grep -qF 'title=Marker task' "$marker_dir/gh.log" \ - || fail "a same-board card that lost its task marker was silently retained instead of recreated" +if ! grep -qF 'addProjectV2DraftIssue' "$marker_dir/gh.log" || ! grep -qF 'title=Marker task' "$marker_dir/gh.log"; then + fail "a same-board card that lost its task marker was silently retained instead of recreated" +fi pass "a same-board card that loses its task marker is recreated, not silently retained" held_delete_dir="$TMP_ROOT/delete-already-held"