Skip to content

feat(harness): add GitHub Copilot CLI support - #3588

Open
landoncrabtree wants to merge 69 commits into
kunchenguid:mainfrom
landoncrabtree:feat/copilot-cli-support
Open

feat(harness): add GitHub Copilot CLI support#3588
landoncrabtree wants to merge 69 commits into
kunchenguid:mainfrom
landoncrabtree:feat/copilot-cli-support

Conversation

@landoncrabtree

@landoncrabtree landoncrabtree commented Sep 3, 2026

Copy link
Copy Markdown

Intent

Add comprehensive GitHub Copilot CLI support to Firstmate, incorporating lessons from prior attempts, preserving repository-owned Claude configuration and existing harness behavior, keep PR 3588 conflict-free on current upstream main, and deliver it with the full no-mistakes Test phase and all required validation passing.

What Changed

  • Added GitHub Copilot CLI as a verified Firstmate harness, including the tracked primary hook file at .github/hooks/fm-primary.json, Copilot-specific hook adapters, worker busy-state hooks, spawn/profile support, and remote secondmate allowlisting.
  • Hardened Copilot/Claude coexistence and supervision by sharing harness process detection, translating Copilot-native pretool and agentStop decisions, publishing single-use watcher completion receipts, and keeping tracked Claude/Copilot hook files repository-owned.
  • Documented the Copilot runtime and supervision contract and added broad regression plus live coverage for hook execution, trust handling, watcher notifications, spawn cleanup, composer detection, and remote control paths.

Risk Assessment

⚠️ Medium: This is a large, cross-cutting adapter addition spanning spawn, hook routing, watcher continuity, liveness, and teardown paths, so even without a substantiated defect the source change carries moderate integration risk.

Testing

I ran the live Copilot primary E2E against GitHub Copilot CLI 1.0.83-3 plus focused Copilot, watcher, session-lock, tmux-liveness, and spawn-profile regressions. The live hook flow passed for session-start context, watcher-arm denial, bounded agentStop continuation, watcher wake resume, and inert unrelated notifications; the only live scenario I could not complete was fm-spawn's wrapped-path trust/config-preservation path because this host lacks a working treehouse get implementation for entering an isolated task worktree.

  • Live validation: ✅ go - 3 of 4 scenarios driven live against the product
Scenario Result Live Evidence
Start Copilot in a new repo and see Firstmate session-start context, watcher-arm denial, and one bounded agentStop continuation ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Run bin/fm-watch-arm.sh as its own attached async task and have the completion wake resume Copilot so it drains and acknowledges exactly once ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Finish an unrelated attached background shell task and have Copilot resume without triggering bin/fm-wake-drain.sh ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Spawn a Copilot worker into a fresh wrapped-path task worktree and keep tracked .github/hooks/fm-primary.json and .claude/settings.json untouched while trust clears ⏸️ untested no This host does not provide a working treehouse get tool, which bin/fm-spawn.sh requires to move the tmux pane from the spawning project into an isolated task worktree before Copilot launch. Instal…
Evidence: Copilot primary live E2E log

ok - Copilot live hooks: denial, stop continuation, watcher wake, and inert unrelated notifications (GitHub Copilot CLI 1.0.83-3.)

ok - Copilot live hooks: denial, stop continuation, watcher wake, and inert unrelated notifications (GitHub Copilot CLI 1.0.83-3.)
Evidence: Copilot harness regression log
ok - Copilot fallback yields only to ambiguous Claude/Cursor conflicts
ok - Copilot process detection accepts native, path, MainThread, and node-bundle shapes while rejecting later-argument decoys
ok - Copilot-marked processes use actual ancestry to resolve inherited marker conflicts
ok - session-lock ancestry recognizes Copilot MainThread and node-bundle process shapes
ok - real processes identify native, path, and verified node-bundled Copilot shapes while rejecting decoys
ok - tmux composer identity and liveness recognize node-bundled Copilot and reject MainThread decoys
ok - tmux Copilot identity ignores stale pane_current_command fallbacks
ok - tmux foreground identity trims indented Copilot and Cursor argv0 values without matching decoys
ok - Copilot sessionStart returns the full digest as additionalContext
ok - Copilot agentStop translates the shared guard refusal into a native block
ok - Copilot pretool-arm delegates only in a genuine primary
ok - Copilot pretool-arm stands down in linked task worktrees
ok - Copilot native hooks bypass only the compatibility stand-down
ok - Copilot agentStop leaves a healthy turn end unchanged
ok - Copilot notifications accept receiptless command success evidence but still require receipts for title-only payloads
ok - Copilot notification stands down outside genuine primary scope
ok - tracked Copilot hook commands execute through the shipped registration
ok - Copilot repository hooks stay inert outside local Copilot CLI
ok - Claude compatibility hooks stay inert under native Copilot hooks
ok - Claude compatibility hooks still run under actual Claude with inherited Copilot markers
ok - Claude compatibility hooks resolve their worktree root from repository subdirectories
# all fm-copilot-harness tests passed
Evidence: Watch-arm regression log
ok - watch-arm: an attached arm reports the wake its cycle delivered instead of a false failure
ok - watch-arm: a delivered wake consumed by the handling turn still closes the attached arm cleanly
ok - watch-arm: a cycle that delivered no wake of its own still fails loudly
~/.no-mistakes/worktrees/009da2e88dc4/01M23TWB9F4ST2AV1EQ16H9Y0B/bin/fm-watch-arm.sh: line 584: 3247395 Killed                     "$WATCH" > "$child_out"
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 2 --recovery-generation 3248018.1788986599.nOqmkD
ok - watch-arm: re-arm surfaces every queued wake and an open remote decision after downtime
watcher: recovery state could not be persisted; retaining stale lock evidence
ok - watch-arm: marker publication failure retains stale-lock recovery evidence
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 2 --recovery-generation 3256097.1788986620.UeEfmw
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 3 --recovery-generation 3257588.1788986625.5VYtab
ok - watch-arm: a wake queued after handling drain is recovered once at successor arm
ok - watch-arm: interrupted handling leaves its wake durable for successor re-drain
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 0 --recovery-generation 3265014.1788986645.5S2gzW
ok - watch-arm: malformed recovery state is quarantined without a successor loop
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 1 --recovery-generation 3267024.1788986650.996rEC
ok - watch-arm: publication after recovery handoff is surfaced
ok - watch-arm: restart publishes recovery before clearing a reused-pid watcher lock
WAKE_ACK_REQUIRED: after handling completes run bin/fm-wake-drain.sh --ack-through 7 --recovery-generation 3269088.1788986657.DjVomJ
ok - watch-arm: markerless legacy queues are adopted and recovered
ok - watch-arm: a watcher close during handling keeps the printed acknowledgement valid
ok - watch-arm: a moved recovery generation consumes handled rows and names its remedy
ok - watch-arm: downtime marker publication does not follow symlinks
ok - watch-arm: successful Copilot arms publish single-use completion receipts
ok - watch-arm: Copilot receipt publication blocked failure preserves the wake
ok - watch-arm: Copilot receipt publication symlink failure preserves the wake
Evidence: Spawn dispatch profile regression log
ok - allowlist=absent preserves the operational floor and filters only when opted in
ok - allowlist=missing-config preserves the operational floor and filters only when opted in
ok - allowlist=enabled preserves the operational floor and filters only when opted in
ok - allowlist=empty preserves the operational floor and filters only when opted in
ok - invalid allowlist names refuse before launch or task publication
ok - inaccessible config with present allowlist refuses before launch or task publication
ok - inaccessible config with absent allowlist refuses before launch or task publication
ok - inaccessible ancestor with present allowlist refuses before launch or task publication
ok - inaccessible ancestor with absent allowlist refuses before launch or task publication
ok - secondmate launch inherits the allowlist for subsequent worker launches
ok - local inheritance preserves the allowlist on source errors and mirrors proven absence
ok - remote inheritance preserves the allowlist on source errors and mirrors proven absence
ok - fm-spawn: actual ship/scout launch commands deliver the worker role contract
ok - no --model/--effort records defaults and types the claude launch instructions
ok - non-cursor launches clear inherited Cursor identity markers
ok - non-copilot launches clear inherited Copilot identity markers
ok - relative home overrides ignore CDPATH and become absolute before spawn launch construction
ok - FM_HOME defaults resolve relative paths and preserve absolute spellings
ok - absolute override spellings are preserved in spawn launch paths
ok - unresolvable relative spawn overrides fail with named diagnostics
ok - active crew-dispatch profile requires an explicit harness for ship spawns
ok - active crew-dispatch profile requires an explicit harness for scout spawns
ok - active crew-dispatch profile allows an explicit resolved harness
ok - active crew-dispatch profile allows the legacy positional harness form
ok - active crew-dispatch profile allows the raw launch-command escape hatch
ok - claude receives --model and --effort profile flags
ok - codex receives --model and model_reasoning_effort profile flags
ok - codex omits unsupported max effort instead of passing a bad config value
ok - grok receives --model and --reasoning-effort profile flags
ok - grok omits unsupported max reasoning effort
ok - grok omits unsupported xhigh reasoning effort
ok - copilot launch and generated worker hooks carry the complete adapter contract
ok - copilot launch clears inherited Claude project roots before entering another worktree
ok - copilot spawn accepts only the exact session-only trust default and cleans up refusals
ok - copilot launches scrub foreign markers before child detection
ok - copilot spawn preserves repository-owned Copilot and Claude config files
ok - copilot spawn refuses only its exact owned hook filename
ok - failed fresh copilot spawns clean their task worker hook so retry works
ok - copilot spawn rejects symlinked hook paths before writing
ok - copilot worker hooks ignore foreign child sessions
ok - copilot secondmate skips task-specific worker hooks while crew spawns keep them
ok - cursor receives its model-qualified reasoning class and exact task workspace
ok - cursor refuses model ids absent from its resolved binary's live catalog
ok - cursor preserves the requested model when its live catalog is unreachable
ok - opencode receives --model and omits the unsupported effort axis
ok - native effort validator checks harness and model as separate axes
ok - Ultra is explicit for native Pi and Pi-signed, including direct-PR, and refuses unsupported profiles before provisioning
ok - batch dispatch preserves native Ultra in metadata and launch flags
ok - pi receives --model and --thinking max profile flags
ok - Pi launch probing omits --tui-mode on older Pi and preserves it on supporting Pi
ok - pi-signed shares Pi launch semantics while preserving its configured and recorded identity
ok - pi-signed refuses safely and actionably when the selected executable is unavailable
ok - pi-signed is a distinct persistent secondmate runtime with shared Pi supervision semantics
ok - batch dispatch forwards shared --harness, --model, and --effort to every pair
ok - claude forwards firstmate's CLAUDE_CONFIG_DIR so the crewmate uses the same credential store
ok - claude omits the config-dir prefix when firstmate runs with the single-store default
ok - non-claude harnesses do not receive the claude CLAUDE_CONFIG_DIR prefix
ok - a claude crewmate launch carries the attribution-off policy in its own settings
ok - a claude secondmate launch carries the attribution-off policy too
ok - active crew-dispatch profile does not block secondmate launches
# all fm-spawn-dispatch-profile tests passed
Evidence: Live fm-spawn blocker note

Attempted live Copilot fm-spawn validation from an isolated temp project/home with real tmux and Copilot CLI. Blocked because this host does not provide a working treehouse get implementation for fm-spawn's isolated-worktree step.

Attempted live Copilot fm-spawn validation from an isolated temp project/home with real tmux and Copilot CLI.
Result: blocked before Copilot launch because this host does not provide a working `treehouse get` implementation for fm-spawn's isolated-worktree step.
Observed error:
error: treehouse get did not enter an isolated worktree within 60s (last seen '~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/live-copilot-spawn-trust/project': it is the spawning project itself; spawning project '~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/live-copilot-spawn-trust/project'); inspect window firstmate:fm-live-copilot-trust
- Outcome: 🔧 2 issues found → auto-fixed ✅ across 2 runs (48m16s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

🔧 **Rebase** - 2 issues found → auto-fixed ✅
  • ⚠️ bin/fm-bootstrap.sh - merge conflict rebasing onto refs/remotes/no-mistakes-push/feat/copilot-cli-support
  • ⚠️ docs/remote-secondmates.md - merge conflict rebasing onto origin/main

🔧 Fix applied.
✅ Re-checked - no issues remain.

🔧 **Review** - 2 issues found → auto-fixed ✅
  • 🚨 bin/fm-copilot-hook.sh:66 - Command-bearing watcher completions are now rejected unless fm_copilot_watch_receipt_claim succeeds, but bin/fm-watch-arm.sh explicitly treats receipt publication as non-fatal. A successful bin/fm-watch-arm.sh run whose receipt write fails will therefore never inject the Copilot wake follow-up, leaving supervision stalled even though the classifier positively recognized the watcher command.
  • 🚨 bin/fm-spawn.sh:2932 - copilot_trust_dialog_exact requires the full $WT path to appear as one literal captured line. In real tmux/herdr panes, long worktree paths commonly wrap, so the verified default trust dialog is misclassified as unsafe and first-launch Copilot spawns abort before the worker starts.

🔧 Fix applied.
✅ Re-checked - no issues remain.

🔧 **Test** - 2 issues found → auto-fixed ✅
  • 🚨 bin/fm-copilot-hook.sh:206 - Live Copilot primary validation failed on watcher wake resume. In a real copilot session, the tracked notification hook ran, received the expected shell_completed payload, and emitted the expected {"additionalContext":...} watcher follow-up, but Copilot never resumed to run bin/fm-wake-drain.sh or the required --ack-through acknowledgement. That leaves Firstmate supervision stalled after a successful watcher completion even though the portable hook tests pass.
  • 🚨 live validation verdict: no-go (4 of 6 scenarios were driven live against the product); failed: Start Firstmate under GitHub Copilot CLI, arm the watcher as an attached async task, and have the watcher completion resume the session so Copilot drains and acknowledges the wake
  • Live validation: ❌ no-go - 4 of 6 scenarios driven live against the product
Scenario Result Live Evidence
Start Firstmate under GitHub Copilot CLI, arm the watcher as an attached async task, and have the watcher completion resume the session so Copilot drains and acknowledges the wake ❌ fail live FM_COPILOT_LIVE_E2E=1 tests/fm-copilot-primary-live-e2e.test.sh; artifacts copilot-primary-live.log and copilot-debug2-live.txt
Feed Copilot watcher completion payloads directly to the tracked notification hook and accept only proven watcher completions while rejecting missing-evidence and explicit-failure cases ✅ pass live artifact copilot-hook-live.log
Run the repository-owned Claude compatibility hook from a Copilot-owned session without double-running Claude helpers, while still allowing the same tracked config to run under Claude ancestry ✅ pass live artifact claude-compat-copilot-live.log
Validate a crew-dispatch config that selects harness=copilot with effort=max through the real bootstrap entrypoint without a CREW_DISPATCH error ✅ pass live artifact bootstrap-copilot-max-live.log
Launch and relaunch a live remote secondmate on the herdr backend with harness=copilot, preserving the Copilot runtime across both operations ⏸️ untested no This environment does not provide a real herdr runtime or a separate SSH-reachable secondmate home to drive fm-remote-secondmate-control.sh launch/relaunch end-to-end. Provide a configured Herdr s…
Spawn a real Copilot crewmate into a fresh worktree and clear the trust dialog using the session-only default even when the worktree path wraps across lines ⏸️ untested no This host is missing treehouse, and safely driving a real worker spawn would also require a dedicated disposable spawn backend/session rather than the shared gate environment. Provide treehouse pl…
  • FM_COPILOT_LIVE_E2E=1 tests/fm-copilot-primary-live-e2e.test.sh > ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/copilot-primary-live.log 2>&1
  • tests/fm-copilot-harness.test.sh
  • tests/fm-watch-arm.test.sh
  • tests/fm-spawn-dispatch-profile.test.sh
  • tests/fm-remote-secondmate-control.test.sh
  • FM_HOME=<tmp> FM_PROJECTS_OVERRIDE=<tmp>/projects FM_CONFIG_OVERRIDE=<tmp>/config FM_STATE_OVERRIDE=<tmp>/state FM_DATA_OVERRIDE=<tmp>/data FM_BOOTSTRAP_DETECT_ONLY=1 FM_BOOTSTRAP_NETWORK=skip ./bin/fm-bootstrap.sh (artifact: bootstrap-copilot-max-live.log)
  • exec -a copilot /bin/bash ./bin/fm-copilot-hook.sh notification < <payload> against an isolated real git repo fixture with real receipt files (artifact: copilot-hook-live.log)
  • exec -a claude /bin/bash ./bin/fm-claude-compat-hook.sh fm-turnend-guard.sh and exec -a copilot /bin/bash ./bin/fm-claude-compat-hook.sh fm-turnend-guard.sh against an isolated real git repo fixture (artifact: claude-compat-copilot-live.log)
  • tmux -L <private-socket> ... copilot -i ... manual live repro capturing pane text plus .hook-notification-payload.json and .hook-notification-output.json (artifacts: copilot-debug-live.txt, copilot-debug2-live.txt)

🔧 Fix applied.
✅ Re-checked - no issues remain.

  • Live validation: ✅ go - 3 of 4 scenarios driven live against the product
Scenario Result Live Evidence
Start Copilot in a new repo and see Firstmate session-start context, watcher-arm denial, and one bounded agentStop continuation ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Run bin/fm-watch-arm.sh as its own attached async task and have the completion wake resume Copilot so it drains and acknowledges exactly once ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Finish an unrelated attached background shell task and have Copilot resume without triggering bin/fm-wake-drain.sh ✅ pass live FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh -> ~/.no-mistakes/evidence/01M23TWB9F4ST2AV1EQ16H9Y0B/fm-copilot-primary-live-e2e.log
Spawn a Copilot worker into a fresh wrapped-path task worktree and keep tracked .github/hooks/fm-primary.json and .claude/settings.json untouched while trust clears ⏸️ untested no This host does not provide a working treehouse get tool, which bin/fm-spawn.sh requires to move the tmux pane from the spawning project into an isolated task worktree before Copilot launch. Instal…
  • bash tests/fm-copilot-harness.test.sh
  • bash tests/fm-watch-arm.test.sh
  • bash tests/fm-session-lock-ancestry.test.sh
  • bash tests/fm-tmux-agent-liveness.test.sh
  • bash tests/fm-spawn-dispatch-profile.test.sh
  • FM_COPILOT_LIVE_E2E=1 bash tests/fm-copilot-primary-live-e2e.test.sh
  • env -u NO_MISTAKES_GATE ... bin/fm-spawn.sh live-copilot-trust <temp-project> --scout --harness copilot inside a private tmux server with real Copilot CLI and tmux (blocked before live spawn by missing treehouse get)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@greptile-apps

greptile-apps Bot commented Sep 3, 2026

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge.

@landoncrabtree
landoncrabtree force-pushed the feat/copilot-cli-support branch from 72453ef to 9434ecc Compare September 3, 2026 17:29
@landoncrabtree landoncrabtree changed the title feat: add GitHub Copilot CLI support feat(harness): add GitHub Copilot CLI support Sep 3, 2026
@landoncrabtree
landoncrabtree force-pushed the feat/copilot-cli-support branch from 9434ecc to 2bbe77e Compare September 5, 2026 04:54
@landoncrabtree landoncrabtree changed the title feat(harness): add GitHub Copilot CLI support feat: add GitHub Copilot CLI support Sep 5, 2026
@landoncrabtree
landoncrabtree force-pushed the feat/copilot-cli-support branch from 2bbe77e to ca6d986 Compare September 8, 2026 18:27
@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate: contract-class new-default (not pure opt-in). Copilot as a verified harness is opt-in when the captain runs Copilot, but this PR also (1) rewires tracked .claude/settings.json hooks through fm-claude-compat-hook.sh on the unconfigured Claude path and (2) reorders fm-harness.sh detection to prefer ancestry before markers for every harness. Those are always-on / reorder changes — no auto-merge.

VISION: fleet-outlives-vendor aligns for the Copilot adapter itself; Authority/consent does not for the Claude hook + detection reorder. Security FYI (not a captain flag while blocked): crew Copilot launch uses --allow-all --no-ask-user. Attestation head_sha MATCH ca6d9867ebfba9707f75b0175538df69f76cbc35, but Require no-mistakes FAIL: pipeline test step is skipped (must be completed). First-time fork CI approved; CI in progress. Live help for open issue #604; draft #1184 is historical/related — keep helping this PR, do not open a competing one. Not 14-day stale (author active).

Waiting on: author — complete no-mistakes test (no skip) and re-push so NM is green; captain flag only if/when otherwise ready on the new-default pieces.

@landoncrabtree landoncrabtree changed the title feat: add GitHub Copilot CLI support feat: add GitHub Copilot CLI harness support Sep 8, 2026
@landoncrabtree

landoncrabtree commented Sep 8, 2026

Copy link
Copy Markdown
Author

Addressed the maintainer feedback and CI regressions at head 1d734e8.

  • Restored the existing marker-first detection contract for non-Copilot runtimes; Copilot uses ancestry only to resolve its inherited-marker conflict, and Copilot launches now clear Gemini/Rovo markers.
  • Fixed failed fresh Copilot spawns leaving generated hook files behind and restored the narrower tmux liveness identity boundary.
  • Updated the live guard to drive the hook-owning interactive Copilot surface. The full no-mistakes Test phase completed, including live Copilot CLI 1.0.83-3 coverage for session start, pre-tool denial, bounded agent-stop continuation, title-only watcher completion, wake drain/acknowledgement, and inert unrelated notifications.

The PR attestation now records test status completed. GitHub has held the fresh fork workflows before job creation, so an upstream maintainer needs to approve these runs:

GTiberi added a commit to GTiberi/firstmate that referenced this pull request Sep 9, 2026
bin/fm-copilot-hook.sh's notification handler checked the snake_case
field notification_type, but Copilot CLI 1.0.83 GA's real
shell_completed payload uses camelCase notificationType (confirmed
live: captured payload keys are cwd, message, notificationType,
sessionId, timestamp, title -- no notification_type anywhere). That
mismatch meant copilot_notification_has_watcher_completion returned
false unconditionally, for every real notification, regardless of
title, command, or shellId shape. This is very likely the same
mechanism behind the notification/resume discrepancy the earlier
scout reported against 1.0.83 (they tested through the -p proxy;
this fix and its live verification used the real interactive -i
primary path).

Also relaxed the title-matcher's shellId pattern from digits-only
([0-9]+) to any non-delimiter run ([^)]+ / [^"]+): Copilot lets the
model name its own async shell task's shellId freely, and a live
capture showed a real session choosing a descriptive string, not a
small integer. The digits-only assumption would have kept failing
even with the field-name fix applied.

Verified live end to end in an isolated lab: launched a real
interactive Copilot primary (COPILOT_HOME isolated, GH_TOKEN from
`gh auth token`), armed the watch-arm script as its own attached
async task, injected an external wake via bin/fm-inbox.sh note, and
confirmed the async task's completion resumed the model. With only
the pre-fix code, the resumption happened through Copilot's own
native shell-completion surfacing (the model noticed on its own and
ran read_bash), not through this hook's enriched
"FIRSTMATE WATCHER WAKE ... run the wake drain first" follow-up,
because the match silently failed. Re-tested the fixed matcher
directly against the real captured payload (this commit's tests) and
confirmed a match.

docs/supervision-protocols/copilot.md now tells the primary to name
the async watch-arm task exactly "Arm Firstmate watcher", the exact
phrase the fallback title matcher requires -- nothing before this
told the model to produce that specific string, so the fallback path
depended entirely on coincidence.

tests/fm-copilot-harness.test.sh's notification fixtures all used the
same wrong notification_type key as the code they were validating, so
they exercised a payload shape Copilot never actually sends; updated
all of them to notificationType, and changed the two "live shape"
fixtures' shellId from small integers to realistic model-chosen
strings so the relaxed pattern is actually exercised.

Not specific to this fork's carry of PR kunchenguid#3588 beyond being part of
the Copilot adapter itself; suitable to offer upstream as-is.
Landon Crabtree and others added 18 commits September 9, 2026 19:42
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
* feat(harness): verify gemini as a crewmate runtime adapter

Adds Gemini CLI as a fourth dispatch target alongside claude, codex, and
grok, scoped to crewmate and scout work only. Every axis was proven against
gemini-cli 0.58.0 rather than inferred; docs/verification/runtime-backends.md
carries the dated evidence and names what stayed unverified.

Busy state is semantic, not rendered: BeforeAgent opens a turn and AfterAgent
and SessionEnd close it. AfterAgent also fires on a manual interrupt, so a
cancelled turn closes its own record.

Three findings shaped the wiring rather than a config line:

- --skip-trust and GEMINI_CLI_TRUST_WORKSPACE=true are presented by the CLI
  as equivalents and are not. A controlled A/B showed --skip-trust leaves
  project configuration unloaded, so workspace skills never load.
- The worktree's .gemini/settings.json is the PROJECT's committed settings
  file, unlike claude's settings.local.json. Firstmate's hooks therefore go
  to a firstmate-owned state/<id>.gemini-settings.json reached through
  GEMINI_CLI_SYSTEM_SETTINGS_PATH, which also works untrusted and merges
  with a project's own hooks instead of replacing them.
- The shipped CLI is a node bundle whose live process reports comm as
  MainThread, so ancestry cannot see it. GEMINI_CLI=1 is load-bearing and is
  tested before an inherited CLAUDECODE, and pane liveness identifies gemini
  from the script argument through the new bin/fm-gemini-lib.sh.

Gemini is refused for secondmates: it has no primary supervision protocol.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GYdQkKfSEQrFUxtcTXZ66L

* test: clear gemini's marker in launch and detection expectations

Every non-gemini launch now clears GEMINI_CLI the way it already clears
cursor's markers, so the two tests that pin the exact launch prefix are
updated to match. The harness-detection tests that scrub foreign markers
before probing ancestry scrub GEMINI_CLI too, so running the suite from
inside a gemini session cannot produce a false verdict.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GYdQkKfSEQrFUxtcTXZ66L

* docs: classify the gemini harness reference

The documentation inventory is the single classification owner for maintained
prose surfaces, and every surface must appear in it exactly once. The new
harness reference is agent-runtime, matching its siblings.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GYdQkKfSEQrFUxtcTXZ66L

* no-mistakes(review): Narrow Gemini ancestry detection

* no-mistakes(review): Restrict Gemini hooks to canonical launches

* no-mistakes(document): Document Gemini adapter support boundaries

* no-mistakes(ci): Fixed Gemini process identity when interpreter or script paths contain whitespace. Tmux liveness now uses NUL-delimited /proc argv on Linux, with the existing flattened ps fallback elsewhere. Added a real-process regression test. Verified with the Gemini harness test suite, full fm-lint, ShellCheck, and git diff --check. The CI and Require no-mistakes runs were action_required/attestation outcomes rather than code failures

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
Landon Crabtree and others added 26 commits September 9, 2026 19:42
Keep ancestry-first resolution scoped to Copilot marker conflicts, adopt the shared live-test gate, and refresh the remote doctor integrity binding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
Drive the hook-owning interactive surface in the live guard and accept the two observed watcher task titles while retaining exact completion-message validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
Keep ancestry-first resolution scoped to Copilot marker conflicts, adopt the shared live-test gate, and refresh the remote doctor integrity binding.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803

# Conflicts:
#	.agents/skills/harness-adapters/SKILL.md
#	.agents/skills/harness-adapters/references/harness/copilot.md
#	bin/fm-harness.sh
#	tests/fm-copilot-harness.test.sh
#	tests/fm-copilot-primary-live-e2e.test.sh
#	tests/fm-kimi-harness.test.sh
Settle parent sessions without undocumented IDs, require successful watcher receipts for notifications, and verify session-only trust before unattended workers continue.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 53b6302f-b135-4862-8f96-3e39bbd03803
@landoncrabtree
landoncrabtree force-pushed the feat/copilot-cli-support branch from 1d734e8 to 56a83de Compare September 9, 2026 20:58
@landoncrabtree landoncrabtree changed the title feat: add GitHub Copilot CLI harness support feat(harness): add GitHub Copilot CLI support Sep 9, 2026
@landoncrabtree

Copy link
Copy Markdown
Author

Rebased and re-ran the complete no-mistakes pipeline at head 56a83de.

  • PR feat(harness): add GitHub Copilot CLI support #3588 is conflict-free against current upstream main 78318e2 (merge state UNSTABLE, not DIRTY).
  • Review, Test, documentation, lint, push, and PR update all completed. The attestation records test status completed.
  • The reloop preserved current upstream AFK, mail-plane, remote Herdr, and captain-call lifecycle changes while reapplying the Copilot-specific integration.
  • The completed run hardened watcher notification provenance and continuation, marker/process identity, primary scoping, failed-spawn cleanup, worker-session rebinding, session-only trust handling, and tracked hook execution.
  • Live Copilot CLI 1.0.83-3 coverage passed session-start context, watcher-arm denial, bounded agentStop continuation, watcher wake resume/drain/acknowledgement, and inert unrelated notifications. The fresh wrapped-path worker-spawn trust case remains portable-only because this host lacks a working treehouse get implementation.

GitHub has again held the new fork workflows before job creation. An upstream maintainer needs to approve:

@kunchenguid

Copy link
Copy Markdown
Owner

Speaking as Kun's firstmate:

Re-triaged after author rebase to HEAD 56a83de756febc93040d48ed48df6788640c83b9 vs main 78318e2c (now past merges this pass). MERGEABLE/UNSTABLE. Attestation MATCH. Fork CI/NM approved this pass (runs 34404280659, 34404280641, 34404314664) — no checks had been reported yet because of action_required.

Contract-class: new-default (unchanged). Copilot as a verified harness stays opt-in when the captain runs Copilot, and marker-first is restored for non-Copilot runtimes (ancestry authoritative only for Copilot's inherited-marker conflict). But tracked .claude/settings.json still rewires the unconfigured Claude primary through fm-claude-compat-hook.sh — that is always-on on the default Claude path. No auto-merge.

VISION: Fleet-outlives-vendor aligns for the Copilot adapter; Authority/consent does not for the Claude hook rewrite. Scripts/agents, restart, spine, one-captain, scope — otherwise fine.

Waiting on CI after approval, not on the captain. No captain card until otherwise ready except the default-behavior decision. Security FYI tip-only if Copilot launch still uses broad allow flags — not waiting-captain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants