diff --git a/.agents/skills/harness-adapters/references/harness/pi.md b/.agents/skills/harness-adapters/references/harness/pi.md index 0455efe6681..97136e0d171 100644 --- a/.agents/skills/harness-adapters/references/harness/pi.md +++ b/.agents/skills/harness-adapters/references/harness/pi.md @@ -44,13 +44,19 @@ Pi sets `PI_CODING_AGENT=true` for its children as its harness-detection marker. The primary turn-end behavior was verified on 2026-07-09 with Pi 0.80.5. `.pi/extensions/fm-primary-turnend-guard.ts` listens for logical-run `agent_settled`, not per-tool-loop `turn_end`, and uses `pi.sendUserMessage(..., { deliverAs: "followUp" })` to force one guarded follow-up when `../../../bin/fm-turnend-guard.sh` returns 2. Without `deliverAs: "followUp"`, Pi rejects the send while the agent is still processing. +That option does not cover two prompts that both start while the primary is idle, because Pi chooses between queueing and a new turn before its preflight; `.pi/extensions/lib/fm-pi-prompt-delivery.ts` owns how the primary joins such an overlapping prompt to the running turn instead of dropping it. On native Windows, the extension runs its session-start, both PreToolUse, turn-end, and operational-input Bash helpers through `bash`; macOS and Linux invoke those helpers directly. The primary watcher protocol also requires `.pi/extensions/fm-primary-pi-watch.ts`. The Pi engine auto-discovers both tracked project-local extensions once the project is trusted. The model arms through the `fm_watch_arm_pi` tool, never through a foreground shell arm. The tool result and clean-exit fallback are owned by `../../../docs/supervision-protocols/pi.md`. -`../../../bin/fm-session-start.sh` reports when the live Pi-family session has not loaded both extensions and points at the selected executable after project trust as the fix, with `-e` as a trust-free fallback. +`../../../bin/fm-session-start.sh` reports when the live Pi-family session has not loaded both extensions and points at `/reload` or restarting the selected executable after project trust as the fix, with `-e` as a trust-free fallback. + +Changed extension code activates only through `/reload` in the running primary or a full process restart (verified 2026-09-14 with Pi 0.85.1). +Pi caches each extension factory for the life of the process, and `/new`, `/resume`, and `/fork` rebind that cached factory, so a session replaced that way keeps running the extension code it first loaded even after the files on disk change. +Run `/reload` between turns: a handler already running keeps its old code, and the reload keeps the durable wake queue and any pending replacement wakes. +The loaded-generation markers are the proof of which build the lock-holding session loaded; only that session writes them, so a `--list-models` probe run from its shell cannot make stale code read as current. When a secondmate is launched on Pi or Pi-signed, `../../../bin/fm-spawn.sh --secondmate` launches the selected executable with both `-e .pi/extensions/fm-primary-turnend-guard.ts` and `-e .pi/extensions/fm-primary-pi-watch.ts`. Both files already exist in the secondmate home's git worktree. diff --git a/.agents/skills/updatefirstmate/SKILL.md b/.agents/skills/updatefirstmate/SKILL.md index d781cae31dd..92dacd69ddb 100644 --- a/.agents/skills/updatefirstmate/SKILL.md +++ b/.agents/skills/updatefirstmate/SKILL.md @@ -55,6 +55,7 @@ This touches only the firstmate repo and its own worktrees, never anything under When the updater printed `reread-firstmate: yes`, the tracked instruction surface (`AGENTS.md`, `bin/`, or `.agents/skills/`) just advanced under you. **Read `AGENTS.md` now** (CLAUDE.md is a real `@AGENTS.md` pointer to it) to refresh your operating instructions before doing anything else, so you are acting on the new instructions rather than the stale ones you were started with. When it printed `reread-firstmate: no`, nothing changed for you - skip the re-read. + A Pi primary also runs the tracked `.pi/extensions/` code it loaded, which a re-read cannot replace and a `/new`, `/resume`, or `/fork` does not re-import; when the update changed those files, ask the captain to run `/reload` between turns or restart Pi, as the Pi harness reference's primary integration section owns. 3. **Restart every second mate the updater named.** Pass the whole `restart-secondmates:` list to one command (skip this step entirely when it says `none`): diff --git a/.pi/extensions/fm-primary-pi-watch.ts b/.pi/extensions/fm-primary-pi-watch.ts index cff3962f9d3..08c75b3755b 100644 --- a/.pi/extensions/fm-primary-pi-watch.ts +++ b/.pi/extensions/fm-primary-pi-watch.ts @@ -26,10 +26,14 @@ // queued while main is streaming joins the running run without ever raising // before_agent_start, so waiting on that event stalls every later close. // Consumption is tracked only so a replacement can replay a follow-up Pi had -// not consumed. An idle main consumes at before_agent_start; a streaming main -// consumes at the user message_start carrying the exact wake text; either -// event finishes the pending record, and a still-unconsumed record rides the -// replacement handoff. +// not consumed. A wake is consumed only when a model turn accepts it, which is +// the user message_start carrying the exact wake text on every path: an idle +// main raises it in the turn the wake opens, a streaming main raises it when +// the running turn drains the follow-up, and a wake that lost a preflight race +// raises it in the turn it joined (./lib/fm-pi-prompt-delivery.ts owns that +// join). before_agent_start is not consumption, because Pi's preflight can +// still reject the prompt after it. Consumption finishes the pending record, +// and a still-unconsumed record rides the replacement handoff. // // Restore versus delivery (stated once here): // delivering is true while the serialized pending-wake pump is in flight. @@ -64,6 +68,8 @@ import { FIRSTMATE_CALM_PRESENTATION_EVENT, } from "./lib/fm-calm-visibility.ts"; import { encodeFirstmateOperationalInput } from "./lib/fm-operational-input.ts"; +import { markerWriterMayRecord } from "./lib/fm-pi-loaded-marker.ts"; +import { installPiPromptDelivery } from "./lib/fm-pi-prompt-delivery.ts"; type ArmResult = { ok: boolean; @@ -264,8 +270,15 @@ function lockOwnership(): LockOwnership { return pidAlive(lockPid) ? "other" : "missing"; } +// The loaded-generation marker is evidence that THIS session process loaded +// this build (bin/fm-wake-lib.sh fm_pi_extension_loaded owns the proof). Only +// a live session writes it - session_start or an arm - never factory load, +// because a descendant `pi --list-models` probe loads the same factory without +// starting a session. The writer must be the lock pid itself, or no live +// process may hold the lock yet; a descendant of the lock holder can never +// satisfy the proof and must not overwrite the holder's evidence. function markLoaded(): void { - if (lockOwnership() === "other") return; + if (!markerWriterMayRecord(`${state}/.lock`)) return; mkdirSync(state, { recursive: true }); writeFileSync(marker, `${extensionVersion}\n${process.pid}\n`); } @@ -536,6 +549,7 @@ const cleanupOnProcessExit = () => { process.once("exit", cleanupOnProcessExit); export default function (pi: ExtensionAPI) { + const promptDelivery = installPiPromptDelivery(); let generation = createGeneration(); activateGeneration(generation); @@ -582,8 +596,8 @@ export default function (pi: ExtensionAPI) { return generationIsLive(owner); } - // Pi consumed a main follow-up: an idle main at before_agent_start, a - // streaming main at the user message_start that joins the running run. + // A model turn accepted a main follow-up: the user message_start carrying + // its exact text (see "Delivery versus consumption" above). function consumeWake(owner: SessionGeneration, text: string): void { for (const [token, wake] of owner.unconsumedWakes) { if (wake.content !== text) continue; @@ -1247,17 +1261,20 @@ export default function (pi: ExtensionAPI) { return result; } - pi.on?.("before_agent_start", (event) => { - consumeWake(generation, event.prompt); - }); pi.on?.("message_start", (event) => { if (event.message.role !== "user") return; consumeWake(generation, userMessageText(event.message.content)); }); - pi.on?.("session_start", async () => { + pi.on?.("session_start", async (_event, ctx) => { if (generation.stopping) generation = createGeneration(); activateGeneration(generation); + if (!promptDelivery.ok) { + ctx?.ui?.notify?.( + `watcher: prompt delivery unprotected - overlapping Firstmate and captain prompts can still be dropped (${promptDelivery.detail})`, + "warning", + ); + } markLoaded(); if (lockOwnership() !== "owned") return; activateOwnedWatch(generation); @@ -1319,6 +1336,4 @@ export default function (pi: ExtensionAPI) { }; }, }); - - markLoaded(); } diff --git a/.pi/extensions/fm-primary-turnend-guard.ts b/.pi/extensions/fm-primary-turnend-guard.ts index b8f2285561a..adf1e4b69d0 100644 --- a/.pi/extensions/fm-primary-turnend-guard.ts +++ b/.pi/extensions/fm-primary-turnend-guard.ts @@ -9,11 +9,10 @@ import { encodeFirstmateOperationalInput, firstmateShellInvocation, } from "./lib/fm-operational-input.ts"; +import { markerWriterMayRecord } from "./lib/fm-pi-loaded-marker.ts"; let guardFollowupActive = false; -type LockOwnership = "owned" | "missing" | "other"; - const extensionFile = fileURLToPath(import.meta.url); const extensionDir = dirname(extensionFile); const root = resolve(extensionDir, "../.."); @@ -22,40 +21,10 @@ const state = process.env.FM_STATE_OVERRIDE || `${fmHome}/state`; const marker = `${state}/.pi-turnend-extension-loaded`; const extensionVersion = `sha256:${createHash("sha256").update(readFileSync(extensionFile)).digest("hex")}`; -function parentPid(pid: string): string { - const result = spawnSync("ps", ["-o", "ppid=", "-p", pid], { encoding: "utf8" }); - if (result.status !== 0) return ""; - return result.stdout.trim(); -} - -function pidAlive(pid: string): boolean { - try { - process.kill(Number(pid), 0); - return true; - } catch { - return false; - } -} - -function lockOwnership(): LockOwnership { - let lockPid = ""; - try { - lockPid = readFileSync(`${state}/.lock`, "utf8").trim(); - } catch { - return "missing"; - } - if (!/^[0-9]+$/.test(lockPid) || lockPid === "1") return "other"; - let pid = String(process.pid); - for (let i = 0; i < 8; i += 1) { - if (pid === lockPid) return "owned"; - pid = parentPid(pid); - if (!pid || pid === "1") break; - } - return pidAlive(lockPid) ? "other" : "missing"; -} - +// Written only from session_start, under the writer rule the watch extension +// shares (./lib/fm-pi-loaded-marker.ts owns it). function markLoaded(): void { - if (!existsSync(state) || lockOwnership() === "other") return; + if (!existsSync(state) || !markerWriterMayRecord(`${state}/.lock`)) return; writeFileSync(marker, `${extensionVersion}\n${process.pid}\n`); } @@ -625,6 +594,4 @@ export default function (pi: ExtensionAPI) { guardFollowupActive = false; } }); - - markLoaded(); } diff --git a/.pi/extensions/lib/fm-pi-loaded-marker.ts b/.pi/extensions/lib/fm-pi-loaded-marker.ts new file mode 100644 index 00000000000..c28e2b9a0f7 --- /dev/null +++ b/.pi/extensions/lib/fm-pi-loaded-marker.ts @@ -0,0 +1,31 @@ +import { readFileSync } from "node:fs"; + +// Writer rule for the Pi primary extensions' loaded-generation markers +// (state/.pi-watch-extension-loaded and state/.pi-turnend-extension-loaded). +// bin/fm-wake-lib.sh fm_pi_extension_loaded owns what a marker proves: the +// build it names was loaded by exactly the process recorded in state/.lock. +// +// So a marker may be recorded only by that process itself, or while no live +// process holds the lock yet (a fresh or dead lock, which the session that +// takes it next rewrites from its own session_start or arm). A live descendant +// of the lock holder - such as a `pi --list-models` probe the primary runs from +// its bash tool, which loads the same extension factories from disk - can never +// satisfy the proof, so it must never overwrite the holder's evidence with a +// newer build hash and its own pid. Callers also write only from a started +// session, never from factory load, because such a probe never starts one. +export function markerWriterMayRecord(lockFile: string): boolean { + let lockPid = ""; + try { + lockPid = readFileSync(lockFile, "utf8").trim(); + } catch { + return true; + } + if (lockPid === String(process.pid)) return true; + if (!/^[0-9]+$/.test(lockPid) || lockPid === "1") return false; + try { + process.kill(Number(lockPid), 0); + return false; + } catch (error) { + return (error as { code?: unknown }).code === "ESRCH"; + } +} diff --git a/.pi/extensions/lib/fm-pi-prompt-delivery.ts b/.pi/extensions/lib/fm-pi-prompt-delivery.ts new file mode 100644 index 00000000000..a47fe621d4e --- /dev/null +++ b/.pi/extensions/lib/fm-pi-prompt-delivery.ts @@ -0,0 +1,210 @@ +import * as PiCodingAgent from "@earendil-works/pi-coding-agent"; +import { classifyFirstmateCurrentOperationalText } from "./fm-operational-input.ts"; + +// Primary Pi prompt delivery owner (stated once here). +// +// Pi's AgentSession.prompt() decides between "queue into the running turn" and +// "start a new turn" before its preflight (auth check, pre-send compaction +// check, every extension before_agent_start handler), then enters +// _runAgentPrompt only after that preflight settles. Two prompts that both +// start while main is idle therefore both choose "start a new turn", and the +// one whose preflight settles second reaches Agent.prompt() while the first +// turn is running. Pi rejects it with "Agent is already processing a prompt", +// an extension send surfaces that rejection as the `Extension "" +// error` banner, a captain prompt surfaces it as `Error:`, and the losing +// message is dropped. The rejected run also emits a spurious agent_settled +// that marks the still-running turn idle. Pi's extension API offers no +// awaitable send and no preflight hook, so no extension-side retry can observe +// or prevent that rejection; delaying sends only narrows the window. +// +// This module owns the one seam where the collision happens. It wraps +// AgentSession._runAgentPrompt so a prompt that reaches it while another turn +// is running joins that turn through Pi's own queues instead of being +// rejected: a captain or other non-operational user prompt is queued as a +// steer, exactly as Pi queues input typed while main is streaming, and a +// Firstmate operational prompt or custom message is queued as a follow-up, +// exactly as its sender asked for a streaming main. Pi's running turn drains +// both queues before it settles, so the joined message reaches the model in +// that turn and raises its user message_start there. A join that lands after +// the running turn has already made its final queue check is started as its +// own turn once that turn settles, so nothing is left queued behind an idle +// main. Every Firstmate primary prompt source (watcher wakes, turn-end guard +// nudges, supervision-branch processing requests) and every captain prompt +// passes this seam, so no per-extension retry loop exists to race it. +// +// The wrap is process-global and idempotent: the first extension factory to +// load installs it, and later factories, reloads, and session replacements +// reuse a stable trampoline with the latest registered implementation. Joined +// user text enters the session queue bookkeeping for display and editor restore; +// stranded restart failures are reported through the extension runner. +// A Pi build that lacks the seam is reported, never +// silently patched around; the watcher extension surfaces that report. + +type QueuedMessage = { role?: unknown; content?: unknown }; + +type PendingQueue = { hasItems?: () => boolean; drain?: () => QueuedMessage[] }; + +type JoinableAgent = { + state?: { isStreaming?: unknown }; + steer?: (message: QueuedMessage) => void; + followUp?: (message: QueuedMessage) => void; + hasQueuedMessages?: () => boolean; + steeringQueue?: PendingQueue; + followUpQueue?: PendingQueue; +}; + +type JoinableSession = { + agent?: JoinableAgent; + _isAgentRunActive?: unknown; + isCompacting?: unknown; + _steeringMessages?: string[]; + _followUpMessages?: string[]; + _emitQueueUpdate?: () => void; + _extensionRunner?: { + emitError?: (error: { extensionPath: string; event: string; error: string }) => void; + }; +}; + +type RunAgentPrompt = (this: JoinableSession, messages: QueuedMessage | QueuedMessage[]) => Promise; + +type JoinableSessionClass = { prototype: { _runAgentPrompt?: RunAgentPrompt } }; + +export type PromptDeliveryInstall = { ok: true } | { ok: false; detail: string }; + +export type OperationalClassifier = (text: string) => boolean; + +type PromptDeliveryEntry = { original: RunAgentPrompt; implementation: RunAgentPrompt }; + +type PromptDeliveryRegistry = typeof globalThis & { + [key: symbol]: WeakMap | undefined; +}; + +// Keep the introduction-version symbol stable so a reload or a compatible +// upgrade of this module cannot wrap the same live prototype twice. +const PROMPT_DELIVERY_WRAPS = Symbol.for("firstmate:pi-prompt-delivery:pi-0.85.1"); + +function wrappedPrototypes(): WeakMap { + const registry = globalThis as PromptDeliveryRegistry; + return (registry[PROMPT_DELIVERY_WRAPS] ??= new WeakMap()); +} + +function messageText(content: unknown): string { + if (typeof content === "string") return content; + if (!Array.isArray(content)) return ""; + return content + .filter((part): part is { type: "text"; text: string } => + typeof part === "object" && part !== null && + (part as { type?: unknown }).type === "text" && + typeof (part as { text?: unknown }).text === "string") + .map((part) => part.text) + .join("\n"); +} + +function isOperationalText(text: string): boolean { + return classifyFirstmateCurrentOperationalText(text) !== undefined; +} + +function joinable(session: JoinableSession): boolean { + const agent = session.agent; + return typeof session._isAgentRunActive === "boolean" && + typeof agent?.steer === "function" && + typeof agent.followUp === "function"; +} + +function turnRunning(session: JoinableSession): boolean { + return session._isAgentRunActive === true || session.agent?.state?.isStreaming === true; +} + +function drainQueue(queue: PendingQueue | undefined): QueuedMessage[] { + const drained: QueuedMessage[] = []; + if (typeof queue?.drain !== "function" || typeof queue.hasItems !== "function") return drained; + while (queue.hasItems()) drained.push(...queue.drain()); + return drained; +} + +// Messages a session queued after its last turn's final queue check, found at +// that turn's settlement with no other turn running or preflighting into one. +function strandedMessages(session: JoinableSession): QueuedMessage[] { + const agent = session.agent; + if (turnRunning(session) || session.isCompacting === true) return []; + if (typeof agent?.hasQueuedMessages !== "function" || !agent.hasQueuedMessages()) return []; + return [...drainQueue(agent.steeringQueue), ...drainQueue(agent.followUpQueue)]; +} + +// Install the join on one AgentSession-shaped class. Exported so the portable +// suite can drive the same wrap against a session double; production callers +// use installPiPromptDelivery(). +export function installPromptJoin( + sessionClass: JoinableSessionClass | undefined, + isOperational: OperationalClassifier = isOperationalText, +): PromptDeliveryInstall { + const prototype = sessionClass?.prototype; + if (!prototype) return { ok: false, detail: "AgentSession is not exported" }; + const registry = wrappedPrototypes(); + const installed = registry.get(prototype); + const original = installed?.original ?? prototype._runAgentPrompt; + if (typeof original !== "function") return { ok: false, detail: "AgentSession._runAgentPrompt is missing" }; + + const joinRunningTurn = function (this: JoinableSession, messages: QueuedMessage | QueuedMessage[]): boolean { + if (!joinable(this) || !turnRunning(this)) return false; + const queued = Array.isArray(messages) ? messages : [messages]; + const lead = queued[0]; + const steer = lead?.role === "user" && !isOperational(messageText(lead.content)); + if (lead?.role === "user") { + const pending = steer ? this._steeringMessages : this._followUpMessages; + if (!Array.isArray(pending)) return false; + pending.push(messageText(lead.content)); + if (typeof this._emitQueueUpdate === "function") this._emitQueueUpdate(); + } + const agent = this.agent as Required>; + for (const message of queued) { + if (steer) agent.steer(message); + else agent.followUp(message); + } + return true; + }; + + const wrapped: RunAgentPrompt = async function (this: JoinableSession, messages) { + if (joinRunningTurn.call(this, messages)) return; + try { + await original.call(this, messages); + } finally { + const stranded = strandedMessages(this); + if (stranded.length > 0) { + void entry.implementation.call(this, stranded).catch((error: unknown) => { + try { + if (typeof this._extensionRunner?.emitError === "function") { + this._extensionRunner.emitError({ + extensionPath: ".pi/extensions/lib/fm-pi-prompt-delivery.ts", + event: "agent_settled", + error: error instanceof Error ? error.message : String(error), + }); + } + } catch {} + }); + } + } + }; + const entry = installed ?? { original, implementation: wrapped }; + entry.implementation = wrapped; + if (!installed) { + registry.set(prototype, entry); + prototype._runAgentPrompt = function (messages) { + return entry.implementation.call(this, messages); + }; + } + return { ok: true }; +} + +export function installPiPromptDelivery(): PromptDeliveryInstall { + // _runAgentPrompt is private in Pi's typings, so the class is read through the + // structural seam this module checks at runtime. + const sessionClass = (PiCodingAgent as unknown as { AgentSession?: JoinableSessionClass }).AgentSession; + const result = installPromptJoin(sessionClass); + if (result.ok) return result; + const version = (PiCodingAgent as { VERSION?: unknown }).VERSION; + return { + ok: false, + detail: `Pi ${typeof version === "string" ? version : "(unknown version)"}: ${result.detail}`, + }; +} diff --git a/bin/fm-session-start.sh b/bin/fm-session-start.sh index ed59aa40f35..79f8cd115f3 100755 --- a/bin/fm-session-start.sh +++ b/bin/fm-session-start.sh @@ -764,7 +764,7 @@ if [ "$PRIMARY_HARNESS" = pi ] || [ "$PRIMARY_HARNESS" = pi-signed ]; then PI_TURNEND_VERSION=$(fm_pi_extension_version "$PI_TURNEND_EXT" || printf '') if ! fm_pi_extension_loaded "$PI_WATCH_MARKER" "$PI_WATCH_VERSION" "$PI_LOCK" \ || ! fm_pi_extension_loaded "$PI_TURNEND_MARKER" "$PI_TURNEND_VERSION" "$PI_LOCK"; then - printf 'PI_WATCH_EXTENSION: not loaded - approve Pi project trust once per clone, then restart %s so %s and %s auto-load for turn-end guard and background wake coverage; use -e %s -e %s only if project hooks are not trusted\n' "$PI_RESTART_COMMAND" "$PI_TURNEND_EXT" "$PI_EXT" "$PI_TURNEND_EXT" "$PI_EXT" + printf 'PI_WATCH_EXTENSION: not loaded - approve Pi project trust once per clone, then run /reload in this Pi session or restart %s so %s and %s load their current code for turn-end guard and background wake coverage (/new, /resume, and /fork keep the extension code loaded earlier); use -e %s -e %s only if project hooks are not trusted\n' "$PI_RESTART_COMMAND" "$PI_TURNEND_EXT" "$PI_EXT" "$PI_TURNEND_EXT" "$PI_EXT" fi fi # omp (Oh My Pi) has no project-trust gate: it auto-discovers /.omp/extensions diff --git a/bin/fm-test-run.sh b/bin/fm-test-run.sh index f746a8225ae..b037e7c315f 100755 --- a/bin/fm-test-run.sh +++ b/bin/fm-test-run.sh @@ -320,6 +320,7 @@ family_for_basename() { printf '%s\n' pure-contract-unit ;; fm-daemon.test.sh|fm-guard-stale-banner.test.sh|fm-pi-watch-extension.test.sh|\ + fm-pi-loaded-marker.test.sh|fm-pi-prompt-delivery.test.sh|\ fm-session-lock-ancestry.test.sh|fm-cursor-primary.test.sh|\ fm-supervision-events.test.sh|fm-turnend-guard.test.sh|fm-wake-daemon-lifecycle-e2e.test.sh|\ fm-wake-drain-unread-status.test.sh|\ @@ -378,7 +379,7 @@ family_for_basename() { fm-rovo-signals-live-e2e.test.sh|\ fm-opencode-primary-live-e2e.test.sh|fm-pi-branch-live-e2e.test.sh|\ fm-pi-branch-responsiveness-live-e2e.test.sh|\ - fm-pi-hung-delivery-herdr-e2e.test.sh|\ + fm-pi-hung-delivery-herdr-e2e.test.sh|fm-pi-prompt-collision-live-e2e.test.sh|\ fm-pi-primary-live-e2e.test.sh|fm-omp-primary-live-e2e.test.sh|fm-stow-horizon-live-e2e.test.sh|\ fm-sessionstart-hook-live-e2e.test.sh|fm-sessionstart-instruction-refresh-live-e2e.test.sh|\ fm-quota-array-dispatch-live-e2e.test.sh|fm-send-secondmate-marker-herdr-e2e.test.sh|\ @@ -1482,6 +1483,25 @@ families_for_changed_path() { # a real Pi TUI can answer, so the live guards are selected too. printf '%s\n' live-harness-optin ;; + .pi/extensions/lib/fm-pi-prompt-delivery.ts) + # The primary prompt delivery owner: its own suites, the watch extension + # that installs it, and the live TUI guard for the captain-visible overlap. + printf '%s\n' __script__:fm-pi-prompt-delivery.test.sh + printf '%s\n' __script__:fm-pi-watch-extension.test.sh + printf '%s\n' __script__:fm-pi-loaded-marker.test.sh + printf '%s\n' __script__:fm-pi-primary-types.test.sh + printf '%s\n' live-harness-optin + ;; + .pi/extensions/lib/fm-pi-loaded-marker.ts) + # The loaded-marker writer rule both primary extensions apply, and the + # session-start proof that reads those markers. + printf '%s\n' __script__:fm-pi-loaded-marker.test.sh + printf '%s\n' __script__:fm-pi-watch-extension.test.sh + printf '%s\n' __script__:fm-turnend-guard.test.sh + printf '%s\n' __script__:fm-session-start.test.sh + printf '%s\n' __script__:fm-pi-primary-types.test.sh + printf '%s\n' live-harness-optin + ;; .pi/extensions/lib/fm-operational-input.ts) # The same rule for the operational-input library, whose reach is wider: # every Pi or OMP extension that classifies or encodes operational text. diff --git a/docs/verification/runtime-backends.md b/docs/verification/runtime-backends.md index c2fce574931..eaffdc064f9 100644 --- a/docs/verification/runtime-backends.md +++ b/docs/verification/runtime-backends.md @@ -1696,6 +1696,64 @@ ok - tracked Pi extensions pass strict no-emit typecheck against Pi 0.85.1 These are samples from this host; the guard compares each run with its own unloaded floor and does not assert those exact durations on another run. No provider request or credentialed live-model behavior is covered by these probes. +### 2026-09-14 overlapping primary prompts and loaded-marker writers + +Observed on Linux x86_64 with Node v22.23.2, Pi 0.85.1, and TypeScript 5.9.3 supplied through `npm exec` for the strict check. +Every probe uses an isolated `FM_HOME` and Pi agent directory, an in-process deterministic provider, and no credential; no request left the machine. +The TUI guard runs on a private tmux socket. + +```sh +bin/fm-test-run.sh tests/fm-pi-prompt-delivery.test.sh tests/fm-pi-loaded-marker.test.sh +FM_PI_PROMPT_COLLISION_LIVE_E2E=1 bin/fm-test-run.sh tests/fm-pi-prompt-collision-live-e2e.test.sh +npm exec --yes --package=typescript@5.9.3 -- bash tests/fm-pi-primary-types.test.sh +``` + +```text +ok - prompt delivery joins a running turn by shape (captain steers with context, Firstmate prompts follow up), starts a stranded join after settlement, installs idempotently, and reports a missing seam +ok - real Pi 0.85.1: an overlapping watcher wake, turn-end nudge, branch processing request, or captain prompt is rejected without the delivery owner and reaches one model turn exactly once with it, while non-overlapping prompts still run as separate turns +ok - watch and turn-end guard extensions record loaded evidence only from a started session run by the lock holder (or with no live holder), never at factory load, from a descendant, or over another live session +ok - a real pi 0.85.1 --list-models probe run under the lock holder leaves both loaded markers exactly as the holder recorded them +ok - real Pi 0.85.1 TUI: captain-first overlap (before-reload-1) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI: wake-first overlap (before-reload-2) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI: captain-first overlap (after-reload-1) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI: wake-first overlap (after-reload-2) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI: overlapping prompts kept exactly one linked monitoring cycle across /reload +ok - tracked Pi extensions pass strict no-emit typecheck against Pi 0.85.1 +``` + +The real-session suite proved each overlap is still rejected by an unwrapped Pi 0.85.1 `AgentSession` before it asserted the joined delivery, so the fixed verdict is not vacuous on this version. +The same TUI guard run against the extensions from before the delivery owner timed out waiting for the captain-first watcher wake to reach a model turn. +Rerun the live guard after every Pi upgrade, because the delivery owner wraps the private `AgentSession._runAgentPrompt` seam and the watcher extension reports rather than patches around a build that lacks it. + +### 2026-09-14 settlement controls and prompt overlap in a named Herdr lab + +Observed on Linux x86_64 with Node v22.23.2, Pi 0.85.1, and herdr 0.8.2. +One named non-default lab session was provisioned and torn down only through `bin/fm-herdr-lab.sh`, which both guards reused through `HERDR_LAB_SESSION`; teardown exited 0 with its default-session tripwire intact. +Each settlement control and the overlap guard used its own isolated `FM_HOME`, a synthetic worker endpoint that issues no Herdr call, and no credential. + +```sh +HERDR_LAB_HELPER=bin/fm-herdr-lab.sh +HERDR_LAB_SESSION=$("$HERDR_LAB_HELPER" name fm-pi-prompt-collision-fix) +trap '"$HERDR_LAB_HELPER" teardown "$HERDR_LAB_SESSION"' EXIT +"$HERDR_LAB_HELPER" provision "$HERDR_LAB_SESSION" +export HERDR_LAB_HELPER HERDR_LAB_SESSION +FM_PI_HUNG_DELIVERY_HERDR_E2E=1 FM_PI_PROMPT_COLLISION_LIVE_E2E=1 \ + bin/fm-test-run.sh tests/fm-pi-hung-delivery-herdr-e2e.test.sh tests/fm-pi-prompt-collision-live-e2e.test.sh +``` + +```text +ok - isolated Pi hung-settlement Herdr lab linked every close, kept a fresh beacon, and ran exactly one monitoring cycle +ok - isolated Pi slow-settlement Herdr lab linked every close, kept a fresh beacon, and ran exactly one monitoring cycle +ok - isolated Pi healthy-settlement Herdr lab linked every close, kept a fresh beacon, and ran exactly one monitoring cycle +ok - real Pi 0.85.1 TUI (herdr): captain-first overlap (before-reload-1) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI (herdr): wake-first overlap (before-reload-2) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI (herdr): captain-first overlap (after-reload-1) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI (herdr): wake-first overlap (after-reload-2) delivers the captain message and the watcher wake in one turn with no banner +ok - real Pi 0.85.1 TUI (herdr): overlapping prompts kept exactly one linked monitoring cycle across /reload +``` + +The slow control's branch settles each accepted wake after 20 s, so four closes restore successors while settlements are pending and a fifth runs after they resolve; the healthy control delivers every wake to a real main turn against an in-process provider. + ## Oh My Pi (omp) omp runs crewmate, scout, secondmate, and primary work; [`supervision.md`](supervision.md#omp-oh-my-pi-native-delivery-2026-09-05) owns the primary evidence. diff --git a/tests/fm-calm-pi-extension.test.sh b/tests/fm-calm-pi-extension.test.sh index a05178da3e3..433c8ae1b13 100755 --- a/tests/fm-calm-pi-extension.test.sh +++ b/tests/fm-calm-pi-extension.test.sh @@ -756,6 +756,8 @@ test_rendering_and_session_lifecycle() { cp "$VISIBILITY" "$fixture/lib/fm-calm-visibility.ts" cp "$WORKING_SHIP" "$fixture/lib/fm-calm-working-ship.ts" cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$fixture/lib/fm-operational-input.ts" + cp "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$fixture/lib/fm-pi-loaded-marker.ts" + cp "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$fixture/lib/fm-pi-prompt-delivery.ts" cp "$ROOT/.pi/extensions/lib/fm-branch-dispatch.ts" "$fixture/lib/fm-branch-dispatch.ts" cp "$ROOT/.pi/extensions/lib/fm-async-exec.ts" "$fixture/lib/fm-async-exec.ts" cp "$WATCH_EXT" "$fixture/fm-primary-pi-watch.ts" @@ -3460,6 +3462,8 @@ test_interactive_terminal_e2e() { cp "$VISIBILITY" "$project/.pi/extensions/lib/fm-calm-visibility.ts" cp "$WORKING_SHIP" "$project/.pi/extensions/lib/fm-calm-working-ship.ts" cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$project/.pi/extensions/lib/fm-operational-input.ts" + cp "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$project/.pi/extensions/lib/fm-pi-loaded-marker.ts" + cp "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$project/.pi/extensions/lib/fm-pi-prompt-delivery.ts" cp "$ROOT/.pi/extensions/lib/fm-branch-dispatch.ts" "$project/.pi/extensions/lib/fm-branch-dispatch.ts" cp "$ROOT/.pi/extensions/lib/fm-async-exec.ts" "$project/.pi/extensions/lib/fm-async-exec.ts" cp "$WATCH_EXT" "$project/.pi/extensions/fm-primary-pi-watch.ts" diff --git a/tests/fm-pi-branch-live-e2e.test.sh b/tests/fm-pi-branch-live-e2e.test.sh index 745f9346b4e..4941eab24ba 100644 --- a/tests/fm-pi-branch-live-e2e.test.sh +++ b/tests/fm-pi-branch-live-e2e.test.sh @@ -59,6 +59,8 @@ cp "$ROOT/.pi/extensions/lib/fm-async-exec.ts" "$repo/.pi/extensions/lib/fm-asyn cp "$ROOT/.pi/extensions/lib/fm-branch-model-picker.ts" "$repo/.pi/extensions/lib/fm-branch-model-picker.ts" cp "$ROOT/.pi/extensions/lib/fm-calm-visibility.ts" "$repo/.pi/extensions/lib/fm-calm-visibility.ts" cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$repo/.pi/extensions/lib/fm-operational-input.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$repo/.pi/extensions/lib/fm-pi-loaded-marker.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$repo/.pi/extensions/lib/fm-pi-prompt-delivery.ts" mkdir -p "$repo/bin" cp "$ROOT/bin/fm-operational-input.sh" "$repo/bin/fm-operational-input.sh" cat > "$repo/bin/fm-watch-arm.sh" <<'SH' diff --git a/tests/fm-pi-hung-delivery-herdr-e2e.test.sh b/tests/fm-pi-hung-delivery-herdr-e2e.test.sh index 1adceb44241..cc4f698cf78 100755 --- a/tests/fm-pi-hung-delivery-herdr-e2e.test.sh +++ b/tests/fm-pi-hung-delivery-herdr-e2e.test.sh @@ -1,8 +1,22 @@ #!/usr/bin/env bash -# Opt-in real Pi/Herdr regression for later-cycle restore while branch -# settlement is hung. Every Herdr call is routed through fm-herdr-lab.sh. -# The named lab is never default. Isolated FM_HOME. Abort before any provider -# call. The portable suite pins the same restore/delivery split without Pi. +# Opt-in real Pi/Herdr regression for later-cycle watcher restore under three +# branch-settlement controls. Every Herdr call is routed through fm-herdr-lab.sh. +# The named lab is never default. Each control gets an isolated FM_HOME. +# +# hung the branch accepts every wake and never settles; main turns abort +# before any provider call. +# slow the branch accepts every wake and settles after a bounded delay. +# healthy no branch accepts, so every wake reaches main and runs a real model +# turn against an in-process deterministic provider. +# +# Each control requires every actionable close to start a successor, a fresh +# beacon through an unattended interval, and exactly one monitoring cycle. +# The portable suite pins the same restore/delivery split without Pi. +# +# Lab ownership: by default this script provisions and tears down its own named +# lab. A caller that already provisioned one through fm-herdr-lab.sh passes it +# as HERDR_LAB_SESSION; the script then only runs commands inside it and leaves +# teardown to that caller. set -u # shellcheck source=tests/lib.sh @@ -11,7 +25,7 @@ set -u . "$(dirname "${BASH_SOURCE[0]}")/herdr-test-safety.sh" if [ "${FM_PI_HUNG_DELIVERY_HERDR_E2E:-0}" != 1 ]; then - echo "skip: set FM_PI_HUNG_DELIVERY_HERDR_E2E=1 to run the isolated Pi hung-settlement Herdr regression" + echo "skip: set FM_PI_HUNG_DELIVERY_HERDR_E2E=1 to run the isolated Pi settlement-control Herdr regression" exit 0 fi @@ -22,24 +36,33 @@ command -v jq >/dev/null 2>&1 || fail "jq not found" herdr_forget_inherited_pane LAB_HELPER=${HERDR_LAB_HELPER:-$ROOT/bin/fm-herdr-lab.sh} -SESSION=$("$LAB_HELPER" name fm-pi-hung-deliv) +OWN_LAB=0 +if [ -n "${HERDR_LAB_SESSION:-}" ]; then + SESSION=$HERDR_LAB_SESSION +else + SESSION=$("$LAB_HELPER" name fm-pi-hung-deliv) + OWN_LAB=1 +fi TMP_ROOT=$(fm_test_tmproot fm-pi-hung-delivery-herdr-e2e) -HOME_DIR="$TMP_ROOT/home" -PROJECT="$TMP_ROOT/project" -PI_DIR="$TMP_ROOT/pi-agent" -HANG_EXT="$TMP_ROOT/hang-dispatch.ts" -LAUNCH="$TMP_ROOT/launch-pi.sh" -PANE= +CONTROLS=${FM_PI_SETTLEMENT_CONTROLS:-"hung slow healthy"} +SLOW_SETTLE_MS=${FM_PI_SLOW_SETTLE_MS:-20000} UNATTENDED_SECS=8 +PANE= +HOME_DIR= + +quit_pane() { + [ -n "$PANE" ] || return 0 + "$LAB_HELPER" run "$SESSION" pane send-text "$PANE" '/quit' >/dev/null 2>&1 || true + "$LAB_HELPER" run "$SESSION" pane send-keys "$PANE" enter >/dev/null 2>&1 || true + sleep 2 + PANE= +} cleanup() { local rc=$? trap - EXIT - if [ -n "$PANE" ]; then - "$LAB_HELPER" run "$SESSION" pane send-text "$PANE" '/quit' >/dev/null 2>&1 || true - sleep 1 - fi - if ! "$LAB_HELPER" teardown "$SESSION"; then + quit_pane + if [ "$OWN_LAB" -eq 1 ] && ! "$LAB_HELPER" teardown "$SESSION"; then rc=1 fi rm -rf "$TMP_ROOT" @@ -47,35 +70,7 @@ cleanup() { } trap cleanup EXIT -"$LAB_HELPER" provision "$SESSION" - -mkdir -p "$HOME_DIR"/{state,config,data} "$PROJECT" "$PI_DIR" -printf '# Synthetic isolated hung-settlement lab\n' > "$PROJECT/AGENTS.md" - -cat > "$HANG_EXT" <<'EOF' -import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; -export default function (pi: ExtensionAPI) { - pi.on("project_trust", () => ({ trusted: "yes", remember: false })); - pi.events?.on?.("fm-branch-supervision:dispatch", (data: { accept?: (p: Promise) => void }) => { - data.accept?.(new Promise(() => {})); - }); - pi.on("before_agent_start", (_event, ctx) => { - ctx.abort(); - }); -} -EOF - -wait_markers() { - local i=0 - while [ "$i" -lt 120 ]; do - if [ -f "$HOME_DIR/state/.pi-watch-extension-loaded" ] && [ -f "$HOME_DIR/state/.pi-turnend-extension-loaded" ]; then - return 0 - fi - sleep 0.5 - i=$((i + 1)) - done - return 1 -} +[ "$OWN_LAB" -eq 0 ] || "$LAB_HELPER" provision "$SESSION" cycle_count() { # local needle=$1 file=$HOME_DIR/state/.watch-cycle-exits.log count=0 @@ -87,7 +82,7 @@ cycle_count() { # wait_started() { # local need=$1 i=0 - while [ "$i" -lt 80 ]; do + while [ "$i" -lt 120 ]; do [ "$(cycle_count 'successor=started:')" -ge "$need" ] && return 0 sleep 0.5 i=$((i + 1)) @@ -103,62 +98,191 @@ beacon_age_s() { printf '%s\n' "$((now - mtime))" } -OUT=$("$LAB_HELPER" run "$SESSION" workspace create --cwd "$PROJECT" --label fm-pi-hung-deliv --no-focus) \ - || fail "Herdr lab workspace create failed" -PANE=$(printf '%s' "$OUT" | jq -er '.result.root_pane.pane_id') \ - || fail "Herdr lab workspace create omitted pane id" +# Live watchers bound to this control's home; anything but one is a lost or +# duplicate monitoring cycle. +live_watchers() { + local pid count=0 + for pid in $(pgrep -f "$ROOT/bin/fm-watch.sh" 2>/dev/null || true); do + if tr '\0' '\n' < "/proc/$pid/environ" 2>/dev/null | grep -Fxq "FM_HOME=$HOME_DIR"; then + count=$((count + 1)) + fi + done + printf '%s\n' "$count" +} -cat > "$LAUNCH" < (the control is read from the environment at load) + local file=$2 + cat > "$file" <<'TS' +import { type AssistantMessage, createAssistantMessageEventStream } from "@earendil-works/pi-ai"; +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import { appendFileSync } from "node:fs"; + +const control = process.env.FM_SETTLEMENT_CONTROL ?? ""; +const settleMs = Number(process.env.FM_SLOW_SETTLE_MS ?? "0"); +const events = process.env.FM_SETTLEMENT_EVENTS ?? "/dev/null"; +const record = (line: string) => appendFileSync(events, `${line}\n`); + +export default function (pi: ExtensionAPI) { + pi.on("project_trust", () => ({ trusted: "yes", remember: false })); + pi.events?.on?.("fm-branch-supervision:dispatch", (data: { accept?: (p: Promise) => void }) => { + if (control === "hung") data.accept?.(new Promise(() => {})); + if (control === "slow") { + data.accept?.(new Promise((resolve) => setTimeout(resolve, settleMs))); + record("slow-accepted"); + } + }); + if (control !== "healthy") { + pi.on("before_agent_start", (_event, ctx) => { + ctx.abort(); + }); + return; + } + pi.registerProvider("fm-settlement", { + baseUrl: "http://127.0.0.1/unused", + apiKey: "test-only", + api: "fm-settlement-api", + models: [{ id: "deterministic", name: "deterministic", reasoning: false, input: ["text"], cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, contextWindow: 64000, maxTokens: 64 }], + streamSimple(model) { + record("provider-call"); + const stream = createAssistantMessageEventStream(); + const output: AssistantMessage = { + role: "assistant", content: [], api: model.api, provider: model.provider, model: model.id, + usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, totalTokens: 0, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } }, + stopReason: "stop", timestamp: Date.now(), + }; + setTimeout(() => { + stream.push({ type: "start", partial: output }); + output.content.push({ type: "text", text: "noted" }); + stream.push({ type: "done", reason: "stop", message: output }); + stream.end(); + }, 200); + return stream; + }, + }); + pi.on("session_start", async (_event, ctx) => { + const model = ctx.modelRegistry.find("fm-settlement", "deterministic"); + if (model) await pi.setModel(model); + }); + pi.on("message_start", (event) => { + if (event.message.role !== "user") return; + const content = event.message.content; + const text = typeof content === "string" ? content : content.map((part) => part.type === "text" ? part.text : "").join(""); + if (text.includes("FIRSTMATE WATCHER WAKE")) record("main-wake-turn"); + }); +} +TS +} + +run_control() { # + local control=$1 dir project pi_dir companion launch events out closes i age none + dir="$TMP_ROOT/$control" + HOME_DIR="$dir/home" + project="$dir/project" + pi_dir="$dir/pi-agent" + companion="$dir/companion.ts" + launch="$dir/launch-pi.sh" + events="$dir/events.log" + mkdir -p "$HOME_DIR"/{state,config,data} "$project" "$pi_dir" + printf '# Synthetic isolated %s settlement lab\n' "$control" > "$project/AGENTS.md" + write_companion "$control" "$companion" + + local model_flags="--model openai-codex/gpt-5.6-sol --thinking low" + [ "$control" != healthy ] || model_flags= + cat > "$launch" < $(printf %q "$HOME_DIR/state/.lock") exec env \\ FM_HOME=$(printf %q "$HOME_DIR") \\ FM_ROOT_OVERRIDE=$(printf %q "$ROOT") \\ - PI_CODING_AGENT_DIR=$(printf %q "$PI_DIR") \\ + PI_CODING_AGENT_DIR=$(printf %q "$pi_dir") \\ + PI_OFFLINE=1 \\ + FM_SETTLEMENT_CONTROL=$control \\ + FM_SLOW_SETTLE_MS=$SLOW_SETTLE_MS \\ + FM_SETTLEMENT_EVENTS=$(printf %q "$events") \\ FM_POLL=1 \\ FM_SIGNAL_GRACE=0 \\ FM_HEARTBEAT=600 \\ FM_CHECK_INTERVAL=999999 \\ pi --approve --no-session --no-context-files --no-extensions \\ - -e $(printf %q "$HANG_EXT") \\ + -e $(printf %q "$companion") \\ -e $(printf %q "$ROOT/.pi/extensions/fm-primary-turnend-guard.ts") \\ -e $(printf %q "$ROOT/.pi/extensions/fm-primary-pi-watch.ts") \\ - --model openai-codex/gpt-5.6-sol --thinking low + $model_flags EOF -chmod +x "$LAUNCH" + chmod +x "$launch" -"$LAB_HELPER" run "$SESSION" pane run "$PANE" "$LAUNCH" >/dev/null \ - || fail "could not launch isolated Pi in the named Herdr lab" + out=$("$LAB_HELPER" run "$SESSION" workspace create --cwd "$project" --label "fm-pi-$control" --no-focus) \ + || fail "$control: Herdr lab workspace create failed" + PANE=$(printf '%s' "$out" | jq -er '.result.root_pane.pane_id') \ + || fail "$control: Herdr lab workspace create omitted pane id" + "$LAB_HELPER" run "$SESSION" pane run "$PANE" "$launch" >/dev/null \ + || fail "$control: could not launch isolated Pi in the named Herdr lab" -wait_markers || fail "Pi watch and turn-end extensions did not load in the isolated lab" + i=0 + while [ "$i" -lt 120 ]; do + [ -f "$HOME_DIR/state/.pi-watch-extension-loaded" ] && [ -f "$HOME_DIR/state/.pi-turnend-extension-loaded" ] && break + sleep 0.5 + i=$((i + 1)) + done + [ -f "$HOME_DIR/state/.pi-watch-extension-loaded" ] && [ -f "$HOME_DIR/state/.pi-turnend-extension-loaded" ] \ + || fail "$control: Pi watch and turn-end extensions did not load in the isolated lab" -cat > "$HOME_DIR/state/hung-cycle.meta" < "$HOME_DIR/state/settlement.meta" <> "$HOME_DIR/state/hung-cycle.status" -wait_started 1 || fail "first actionable close did not start a successor while settlement hung" + closes=4 + for i in $(seq 1 "$closes"); do + printf 'done: simulated %s completion %s\n' "$control" "$i" >> "$HOME_DIR/state/settlement.status" + wait_started "$i" || fail "$control: actionable close $i did not start a successor" + sleep 1 + done + none=$(cycle_count 'successor=none') + [ "$none" -eq 0 ] || fail "$control: an actionable close left successor=none (none=$none)" -printf 'done: simulated hung completion 2\n' >> "$HOME_DIR/state/hung-cycle.status" -wait_started 2 || fail "second actionable close did not restore a successor while settlement hung" + if [ "$control" = slow ]; then + # Outlive every accepted settlement so later closes run after they resolve. + sleep $(( SLOW_SETTLE_MS / 1000 + 2 )) + fi + sleep "$UNATTENDED_SECS" + age=$(beacon_age_s) || fail "$control: watcher beacon missing after the unattended interval" + [ "$age" -le $((UNATTENDED_SECS + 5)) ] || fail "$control: watcher beacon went stale (age=${age}s)" -printf 'done: simulated hung completion 3\n' >> "$HOME_DIR/state/hung-cycle.status" -wait_started 3 || fail "third actionable close did not restore a successor while settlement hung" + printf 'done: simulated %s completion %s\n' "$control" "$((closes + 1))" >> "$HOME_DIR/state/settlement.status" + wait_started "$((closes + 1))" || fail "$control: the close after the unattended interval did not start a successor" + [ "$(cycle_count 'successor=none')" -eq 0 ] || fail "$control: the unattended interval recorded successor=none" + sleep 2 + [ "$(live_watchers)" -eq 1 ] || fail "$control: expected exactly one monitoring cycle, found $(live_watchers)" -none=$(cycle_count 'successor=none') -[ "$none" -eq 0 ] || fail "hung settlement left successor=none after later-cycle restores (none=$none)" + case "$control" in + slow) + grep -q '^slow-accepted$' "$events" 2>/dev/null || fail "slow: the branch never accepted a wake, so the control was vacuous" + ;; + healthy) + [ "$(grep -c '^main-wake-turn$' "$events" 2>/dev/null || true)" -ge "$closes" ] \ + || fail "healthy: wakes did not reach real main turns: $(cat "$events" 2>/dev/null)" + [ "$(grep -c '^provider-call$' "$events" 2>/dev/null || true)" -ge "$closes" ] \ + || fail "healthy: main wake turns made no provider calls" + if "$LAB_HELPER" run "$SESSION" pane read "$PANE" --source recent --lines 400 2>/dev/null | grep -Fq 'already processing'; then + fail "healthy: the prompt-collision banner appeared" + fi + ;; + esac + quit_pane + printf 'ok - isolated Pi %s-settlement Herdr lab linked every close, kept a fresh beacon, and ran exactly one monitoring cycle\n' "$control" +} -sleep "$UNATTENDED_SECS" -age=$(beacon_age_s) || fail "watcher beacon missing after the unattended interval" -[ "$age" -le $((UNATTENDED_SECS + 5)) ] || fail "watcher beacon went stale during the unattended interval (age=${age}s)" -[ "$(cycle_count 'successor=started:')" -ge 3 ] || fail "unattended interval lost restored successors" -[ "$(cycle_count 'successor=none')" -eq 0 ] || fail "unattended interval recorded successor=none" +for control in $CONTROLS; do + run_control "$control" +done -printf 'ok - isolated Pi hung-settlement Herdr lab restored later-cycle successors and kept a fresh beacon\n' printf '\nall fm-pi-hung-delivery-herdr-e2e tests passed\n' diff --git a/tests/fm-pi-loaded-marker.test.sh b/tests/fm-pi-loaded-marker.test.sh new file mode 100755 index 00000000000..1c70e5664e2 --- /dev/null +++ b/tests/fm-pi-loaded-marker.test.sh @@ -0,0 +1,207 @@ +#!/usr/bin/env bash +# Tests for the Pi primary extensions' loaded-generation marker writer rule +# (.pi/extensions/lib/fm-pi-loaded-marker.ts) as the watch and turn-end guard +# extensions apply it, and for the proof bin/fm-wake-lib.sh draws from those +# markers. A `pi --list-models` probe the primary runs from its own bash tool +# loads the same extension factories from disk as a descendant of the lock +# holder; it must never replace the holder's evidence, so stale primary code can +# never read as current and current code can never read as unloaded. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" +# shellcheck source=bin/fm-wake-lib.sh +. "$ROOT/bin/fm-wake-lib.sh" + +TMP_ROOT=$(fm_test_tmproot fm-pi-loaded-marker) +export NODE_NO_WARNINGS=1 + +install_marker_fixture() { # + local repo=$1 + mkdir -p "$repo/.pi/extensions/lib" "$repo/bin" \ + "$repo/node_modules/@earendil-works/pi-coding-agent" \ + "$repo/node_modules/@earendil-works/pi-tui" \ + "$repo/node_modules/typebox" + cp "$ROOT/.pi/extensions/fm-primary-pi-watch.ts" "$ROOT/.pi/extensions/fm-primary-turnend-guard.ts" "$repo/.pi/extensions/" + cp "$ROOT/.pi/extensions/lib/fm-branch-dispatch.ts" "$ROOT/.pi/extensions/lib/fm-async-exec.ts" \ + "$ROOT/.pi/extensions/lib/fm-calm-visibility.ts" "$ROOT/.pi/extensions/lib/fm-operational-input.ts" \ + "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" \ + "$repo/.pi/extensions/lib/" + cp "$ROOT/bin/fm-operational-input.sh" "$repo/bin/fm-operational-input.sh" + cat > "$repo/bin/fm-watch-arm.sh" <<'SH' +#!/usr/bin/env bash +printf 'watcher: started pid=%s (beacon fresh)\n' "$$" +trap 'exit 0' TERM INT +while :; do sleep 1; done +SH + chmod +x "$repo/bin/fm-operational-input.sh" "$repo/bin/fm-watch-arm.sh" + printf '%s\n' '{"name":"@earendil-works/pi-coding-agent","type":"module","exports":"./index.js"}' \ + > "$repo/node_modules/@earendil-works/pi-coding-agent/package.json" + printf '%s\n' 'export function getMarkdownTheme() { return {}; }' 'export class UserMessageComponent {}' \ + > "$repo/node_modules/@earendil-works/pi-coding-agent/index.js" + printf '%s\n' '{"name":"@earendil-works/pi-tui","type":"module","exports":"./index.js"}' \ + > "$repo/node_modules/@earendil-works/pi-tui/package.json" + printf '%s\n' 'export class Box { addChild() {} clear() {} setBgFn() {} }' 'export class Container {}' 'export class Text {}' \ + > "$repo/node_modules/@earendil-works/pi-tui/index.js" + printf '%s\n' '{"name":"typebox","type":"module","exports":"./index.js"}' > "$repo/node_modules/typebox/package.json" + printf '%s\n' 'export const Type = { Object(properties) { return { type: "object", properties }; } };' \ + > "$repo/node_modules/typebox/index.js" +} + +test_writer_rule_through_both_extensions() { + local repo home script out status + repo="$TMP_ROOT/writer-root" + home="$TMP_ROOT/writer-home" + script="$TMP_ROOT/writer-rule.mjs" + mkdir -p "$home/state" "$home/config" + install_marker_fixture "$repo" + # Written outside any command substitution: stock macOS Bash 3.2 cannot parse + # a quoted heredoc containing apostrophes inside $(...). + cat > "$script" <<'EOF' +import { spawn } from "node:child_process"; +import { createHash } from "node:crypto"; +import { existsSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { pathToFileURL } from "node:url"; + +const state = `${process.env.FM_HOME}/state`; +const lock = `${state}/.lock`; +const fail = (message) => { + throw new Error(message); +}; +const extensions = [ + { file: "fm-primary-pi-watch.ts", marker: `${state}/.pi-watch-extension-loaded` }, + { file: "fm-primary-turnend-guard.ts", marker: `${state}/.pi-turnend-extension-loaded` }, +]; +let load = 0; +async function startSession(extension) { + const handlers = new Map(); + const pi = { + on(event, handler) { handlers.set(event, handler); }, + registerCommand() {}, + registerTool() {}, + sendUserMessage() {}, + sendMessage() {}, + events: { on() {}, emit() {} }, + }; + const path = `${process.env.REPO}/.pi/extensions/${extension.file}`; + const mod = await import(`${pathToFileURL(path).href}?load=${++load}`); + mod.default(pi); + return { + version: `sha256:${createHash("sha256").update(readFileSync(path)).digest("hex")}`, + start: () => handlers.get("session_start")?.({ type: "session_start", reason: "reload" }, {}), + stop: () => handlers.get("session_shutdown")?.({ type: "session_shutdown", reason: "quit" }, {}), + }; +} +const read = (marker) => existsSync(marker) ? readFileSync(marker, "utf8") : "(absent)"; +const other = spawn(process.execPath, ["-e", "setInterval(() => {}, 1000)"], { stdio: "ignore" }); + +try { + for (const extension of extensions) { + const { marker } = extension; + rmSync(marker, { force: true }); + + // Factory load alone (what a model-list probe does) records nothing. + writeFileSync(lock, `${process.pid}\n`); + const probe = await startSession(extension); + if (existsSync(marker)) fail(`${extension.file} recorded a marker at factory load: ${read(marker)}`); + + // A live descendant of the lock holder never replaces the holder's evidence. + writeFileSync(lock, `${process.ppid}\n`); + writeFileSync(marker, `sha256:holder-evidence\n${process.ppid}\n`); + await probe.start(); + if (read(marker) !== `sha256:holder-evidence\n${process.ppid}\n`) { + fail(`${extension.file} descendant overwrote holder evidence: ${read(marker)}`); + } + await probe.stop(); + + // Nor does a live unrelated holder's session. + writeFileSync(lock, `${other.pid}\n`); + const unrelated = await startSession(extension); + await unrelated.start(); + if (read(marker) !== `sha256:holder-evidence\n${process.ppid}\n`) { + fail(`${extension.file} overwrote another live session's evidence: ${read(marker)}`); + } + await unrelated.stop(); + + // The lock holder's own session records exactly its build and pid. + writeFileSync(lock, `${process.pid}\n`); + const holder = await startSession(extension); + await holder.start(); + if (read(marker) !== `${holder.version}\n${process.pid}\n`) fail(`${extension.file} holder did not record its evidence: ${read(marker)}`); + await holder.stop(); + + // No live holder yet (absent or dead lock): the starting session records. + for (const [label, prepare] of [["absent", () => rmSync(lock, { force: true })], ["dead", () => writeFileSync(lock, "999999\n")]]) { + rmSync(marker, { force: true }); + prepare(); + const fresh = await startSession(extension); + await fresh.start(); + if (read(marker) !== `${fresh.version}\n${process.pid}\n`) fail(`${extension.file} ${label} lock did not record: ${read(marker)}`); + await fresh.stop(); + } + } +} finally { + other.kill("SIGTERM"); +} +process.exit(0); +EOF + out=$(cd "$repo" && FM_HOME="$home" FM_ROOT_OVERRIDE="$repo" REPO="$repo" \ + node --experimental-strip-types "$script" 2>&1) + status=$? + expect_code 0 "$status" "loaded-marker writer rule: $out" + [ -z "$out" ] || fail "loaded-marker writer rule printed output: $out" + pass "watch and turn-end guard extensions record loaded evidence only from a started session run by the lock holder (or with no live holder), never at factory load, from a descendant, or over another live session" +} + +test_real_model_list_probe_cannot_rewrite_evidence() { + local project home agent watch turnend watch_version turnend_version holder out + if ! command -v pi >/dev/null 2>&1; then + echo "skip: pi not found for the real model-list probe" + return 0 + fi + project="$TMP_ROOT/probe-project" + home="$TMP_ROOT/probe-home" + agent="$TMP_ROOT/probe-agent" + mkdir -p "$project" "$home/state" "$home/config" "$agent" + watch="$ROOT/.pi/extensions/fm-primary-pi-watch.ts" + turnend="$ROOT/.pi/extensions/fm-primary-turnend-guard.ts" + watch_version=$(fm_pi_extension_version "$watch") + turnend_version=$(fm_pi_extension_version "$turnend") + # This shell stands in for the primary: it holds the lock, and the probe is + # its descendant, exactly as when the primary's bash tool runs the probe. + holder=$$ + printf '%s\n' "$holder" > "$home/state/.lock" + + probe() { + (cd "$project" && FM_HOME="$home" FM_ROOT_OVERRIDE="$ROOT" PI_CODING_AGENT_DIR="$agent" PI_OFFLINE=1 \ + pi --approve --no-extensions -e "$turnend" -e "$watch" --list-models >/dev/null 2>&1) + } + + # Stale primary code: the holder loaded an older build. A probe loading the + # current files must not make that holder read as running current code. + printf 'sha256:stale-watch\n%s\n' "$holder" > "$home/state/.pi-watch-extension-loaded" + printf 'sha256:stale-turnend\n%s\n' "$holder" > "$home/state/.pi-turnend-extension-loaded" + probe || fail "pi --list-models probe failed to run" + out=$(cat "$home/state/.pi-watch-extension-loaded" "$home/state/.pi-turnend-extension-loaded") + [ "$out" = "$(printf 'sha256:stale-watch\n%s\nsha256:stale-turnend\n%s' "$holder" "$holder")" ] \ + || fail "model-list probe rewrote stale holder evidence: $out" + if fm_pi_extension_loaded "$home/state/.pi-watch-extension-loaded" "$watch_version" "$home/state/.lock"; then + fail "stale primary watch code reads as current after a model-list probe" + fi + + # Current primary code: the probe must not replace the holder's pid with its + # own and make a loaded primary read as unloaded. + printf '%s\n%s\n' "$watch_version" "$holder" > "$home/state/.pi-watch-extension-loaded" + printf '%s\n%s\n' "$turnend_version" "$holder" > "$home/state/.pi-turnend-extension-loaded" + probe || fail "pi --list-models probe failed to run" + fm_pi_extension_loaded "$home/state/.pi-watch-extension-loaded" "$watch_version" "$home/state/.lock" \ + || fail "model-list probe made current watch evidence unprovable: $(cat "$home/state/.pi-watch-extension-loaded")" + fm_pi_extension_loaded "$home/state/.pi-turnend-extension-loaded" "$turnend_version" "$home/state/.lock" \ + || fail "model-list probe made current turn-end evidence unprovable: $(cat "$home/state/.pi-turnend-extension-loaded")" + pass "a real pi $(pi --version 2>/dev/null) --list-models probe run under the lock holder leaves both loaded markers exactly as the holder recorded them" +} + +test_writer_rule_through_both_extensions +test_real_model_list_probe_cannot_rewrite_evidence + +printf '\nall fm-pi-loaded-marker tests passed\n' diff --git a/tests/fm-pi-primary-live-e2e.test.sh b/tests/fm-pi-primary-live-e2e.test.sh index 0538692eb8f..b54c15c0823 100755 --- a/tests/fm-pi-primary-live-e2e.test.sh +++ b/tests/fm-pi-primary-live-e2e.test.sh @@ -173,7 +173,8 @@ run_native_ahoy_regressions() { "$later_home/state" "$later_home/config" git init -q "$AHOY_PROJECT" cp "$ROOT/.pi/extensions/fm-primary-turnend-guard.ts" "$AHOY_PROJECT/.pi/extensions/" - cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$AHOY_PROJECT/.pi/extensions/lib/" + cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" \ + "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$AHOY_PROJECT/.pi/extensions/lib/" cp \ "$ROOT/bin/fm-sessionstart-nudge.sh" \ "$ROOT/bin/fm-primary-scope-lib.sh" \ @@ -255,6 +256,8 @@ cp "$ROOT/.pi/extensions/lib/fm-calm-working-ship.ts" "$PROJECT/.pi/extensions/l cp "$ROOT/.pi/extensions/lib/fm-branch-dispatch.ts" "$PROJECT/.pi/extensions/lib/fm-branch-dispatch.ts" cp "$ROOT/.pi/extensions/lib/fm-async-exec.ts" "$PROJECT/.pi/extensions/lib/fm-async-exec.ts" cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$PROJECT/.pi/extensions/lib/fm-operational-input.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$PROJECT/.pi/extensions/lib/fm-pi-loaded-marker.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$PROJECT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" cp "$ROOT/.pi/extensions/fm-primary-turnend-guard.ts" "$PROJECT/.pi/extensions/fm-primary-turnend-guard.ts" cp "$ROOT/bin/fm-watch-arm.sh" "$PROJECT/bin/fm-watch-arm.sh" cp "$ROOT/bin/fm-operational-input.sh" "$PROJECT/bin/fm-operational-input.sh" @@ -262,8 +265,19 @@ cp "$ROOT/bin/fm-supervision-instructions.sh" "$PROJECT/bin/fm-supervision-instr chmod +x "$PROJECT/bin/fm-operational-input.sh" mkdir -p "$HOME_DIR/state" "$HOME_DIR/config" +# The session lock names the Pi process itself, exactly as bin/fm-session-start.sh +# records the engine pid: the loaded-generation markers are written only by the +# lock holder (.pi/extensions/lib/fm-pi-loaded-marker.ts), never by a child. +cat > "$LAB/launch-pi.sh" <<'SH' +#!/usr/bin/env bash +bash -c 'printf "%s\n" "$$" > "$FM_HOME/state/.lock"; exec pi --approve --no-session --no-context-files --no-extensions -e .pi/extensions/fm-calm.ts -e .pi/extensions/fm-primary-turnend-guard.ts -e .pi/extensions/fm-primary-pi-watch.ts --model openai-codex/gpt-5.6-sol --thinking low' +rc=$? +printf 'PI_EXIT=%s\n' "$rc" +sleep 300 +SH +chmod +x "$LAB/launch-pi.sh" "$TMUX" -L "$SOCKET" new-session -d -s "$SESSION" -c "$PROJECT" \ - "env FM_HOME='$HOME_DIR' FM_ROOT_OVERRIDE='$PROJECT' FM_POLL=1 FM_SIGNAL_GRACE=0 FM_HEARTBEAT=600 bash -lc 'printf \"%s\\n\" \"\$\$\" > \"\$FM_HOME/state/.lock\"; pi --approve --no-session --no-context-files --no-extensions -e .pi/extensions/fm-calm.ts -e .pi/extensions/fm-primary-turnend-guard.ts -e .pi/extensions/fm-primary-pi-watch.ts --model openai-codex/gpt-5.6-sol --thinking low; rc=\$?; printf \"PI_EXIT=%s\\n\" \"\$rc\"; sleep 300'" + "env FM_HOME='$HOME_DIR' FM_ROOT_OVERRIDE='$PROJECT' FM_POLL=1 FM_SIGNAL_GRACE=0 FM_HEARTBEAT=600 bash -lc '$LAB/launch-pi.sh'" i=0 while [ "$i" -lt 120 ]; do @@ -294,7 +308,15 @@ printf '%s\n' "$pane" | grep -Fq "Working..." \ && fail "Calm left Pi's stock working row visible on the credentialed provider path" wait_for_exact_line "CALM_LIVE_WORKING_VISIBLE" 120 \ || fail "Pi did not settle the Calm working-ship provider probe" -pane=$(capture) +# The reply renders before the run settles, so give the settlement a bounded +# moment to retire the ship rather than sampling the frame that painted the reply. +i=0 +while [ "$i" -lt 100 ]; do + pane=$(capture) + printf '%s\n' "$pane" | grep -Fq '\__/' || break + sleep 0.1 + i=$((i + 1)) +done printf '%s\n' "$pane" | grep -Fq '\__/' \ && fail "Calm left the working ship on screen after the run settled" printf '%s\n' "$pane" | grep -Fq "calm transcript" \ @@ -304,7 +326,18 @@ sleep 0.2 : > "$HOME_DIR/state/pi-e2e.meta" send_prompt "Start supervision with fm_watch_arm_pi and never use bash to arm supervision. After the watcher wake arrives, run bin/fm-wake-drain.sh and reply exactly HANDLED." -wait_for_text "watcher: started Pi extension arm child 1" || fail "Pi did not render the initial watcher tool result" +# The lock is already held when Pi starts, so the extension arms the first cycle +# at session start and the model's required call reports that ownership as a +# no-op (docs/supervision-protocols/pi.md); a start without that owned lock +# would report the first arm child instead. +i=0 +while [ "$i" -lt 240 ]; do + capture | grep -Eq 'watcher: (started Pi extension arm child 1|unchanged - Pi extension already owns an arm child)' && break + sleep 0.5 + i=$((i + 1)) +done +capture | grep -Eq 'watcher: (started Pi extension arm child 1|unchanged - Pi extension already owns an arm child)' \ + || fail "Pi did not render the initial watcher tool result" printf 'done: pi live e2e watcher fire\n' > "$HOME_DIR/state/pi-e2e.status" i=0 diff --git a/tests/fm-pi-primary-types.test.sh b/tests/fm-pi-primary-types.test.sh index bf3ed2ad925..efe28ce30ff 100755 --- a/tests/fm-pi-primary-types.test.sh +++ b/tests/fm-pi-primary-types.test.sh @@ -39,6 +39,8 @@ cp "$ROOT/.pi/extensions/lib/fm-calm-operational-user-layout.ts" "$TMP_ROOT/lib/ cp "$ROOT/.pi/extensions/lib/fm-calm-visibility.ts" "$TMP_ROOT/lib/fm-calm-visibility.ts" cp "$ROOT/.pi/extensions/lib/fm-calm-working-ship.ts" "$TMP_ROOT/lib/fm-calm-working-ship.ts" cp "$ROOT/.pi/extensions/lib/fm-operational-input.ts" "$TMP_ROOT/lib/fm-operational-input.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-loaded-marker.ts" "$TMP_ROOT/lib/fm-pi-loaded-marker.ts" +cp "$ROOT/.pi/extensions/lib/fm-pi-prompt-delivery.ts" "$TMP_ROOT/lib/fm-pi-prompt-delivery.ts" ln -s "$PI_PACKAGE_DIR" "$TMP_ROOT/node_modules/@earendil-works/pi-coding-agent" ln -s "$PI_PACKAGE_DIR/node_modules/@earendil-works/pi-tui" "$TMP_ROOT/node_modules/@earendil-works/pi-tui" ln -s "$PI_PACKAGE_DIR/node_modules/@earendil-works/pi-ai" "$TMP_ROOT/node_modules/@earendil-works/pi-ai" diff --git a/tests/fm-pi-prompt-collision-live-e2e.test.sh b/tests/fm-pi-prompt-collision-live-e2e.test.sh new file mode 100755 index 00000000000..75d9b808414 --- /dev/null +++ b/tests/fm-pi-prompt-collision-live-e2e.test.sh @@ -0,0 +1,331 @@ +#!/usr/bin/env bash +# Opt-in real Pi TUI regression for overlapping primary prompts. +# +# A captain prompt typed into the real interactive Pi and a real watcher wake +# from the tracked watch extension are made to overlap inside one preflight +# window, in both orders, before and after /reload. Without the delivery owner +# (.pi/extensions/lib/fm-pi-prompt-delivery.ts) Pi rejects whichever prompt's +# preflight settles second: the captain sees `Extension "" error: +# Agent is already processing a prompt` or `Error: Agent is already processing a +# prompt`, and that message is dropped. This guard requires both messages to +# reach one model turn with no banner, and one monitoring cycle to keep running. +# +# Isolation: an isolated FM_HOME and Pi agent directory, an in-process +# deterministic provider (no credentials, no network), and a companion +# extension that adds preflight latency and records Pi's lifecycle. The TUI runs +# on a private tmux socket by default. A caller that provisioned a named Herdr +# lab through fm-herdr-lab.sh passes HERDR_LAB_HELPER and HERDR_LAB_SESSION, and +# the TUI then runs in a workspace of that lab, with every Herdr call made +# through the helper and teardown left to that caller. +# tests/fm-pi-prompt-delivery.test.sh owns the portable, always-run layer that +# also proves the unwrapped session still rejects each overlap. +set -u + +# shellcheck source=tests/lib.sh +. "$(dirname "${BASH_SOURCE[0]}")/lib.sh" + +fm_live_gate opt-in FM_PI_PROMPT_COLLISION_LIVE_E2E pi + +TERMINAL=tmux +if [ -n "${HERDR_LAB_SESSION:-}" ]; then + TERMINAL=herdr + [ -x "${HERDR_LAB_HELPER:-}" ] || fail "HERDR_LAB_SESSION requires HERDR_LAB_HELPER to name fm-herdr-lab.sh" + command -v jq >/dev/null 2>&1 || fail "jq not found" +else + command -v tmux >/dev/null 2>&1 || fail "tmux not found" +fi +PI_VERSION=$(pi --version 2>/dev/null) || fail "pi --version failed" +[ -n "$PI_VERSION" ] || fail "pi --version printed nothing" + +TMP_ROOT=$(fm_test_tmproot fm-pi-prompt-collision-live-e2e) +HOME_DIR="$TMP_ROOT/home" +PROJECT="$TMP_ROOT/project" +PI_DIR="$TMP_ROOT/pi-agent" +EVENTS="$TMP_ROOT/events.log" +COMPANION="$TMP_ROOT/companion.ts" +LAUNCH="$TMP_ROOT/launch-pi.sh" +SOCKET="fm-pi-collision-$$" +SESSION=pi-collision +PANE= +PREFLIGHT_MS=3000 +REPLY_MS=4000 + +cleanup() { + local rc=$? pid + trap - EXIT + if [ "$TERMINAL" = herdr ]; then + if [ -n "$PANE" ]; then + type_line "/quit" >/dev/null 2>&1 || true + sleep 2 + fi + else + if tmux -L "$SOCKET" has-session -t "$SESSION" 2>/dev/null; then + type_line "/quit" >/dev/null 2>&1 || true + sleep 2 + fi + tmux -L "$SOCKET" kill-server >/dev/null 2>&1 || true + fi + # The lab watcher runs under the terminal server, outside this shell's + # process tree, so it is retired by its recorded pid when it is still this lab's. + pid=$(cat "$HOME_DIR/state/.watch.lock/pid" 2>/dev/null || true) + if [ -n "$pid" ] && tr '\0' '\n' < "/proc/$pid/environ" 2>/dev/null | grep -Fxq "FM_HOME=$HOME_DIR"; then + kill "$pid" 2>/dev/null || true + fi + if [ -n "${FM_COLLISION_KEEP:-}" ]; then + printf 'kept lab: %s\n' "$TMP_ROOT" >&2 + else + fm_test_cleanup + fi + exit "$rc" +} +trap cleanup EXIT + +mkdir -p "$HOME_DIR"/{state,config,data} "$PROJECT" "$PI_DIR" +printf '# Synthetic isolated prompt-overlap lab\n' > "$PROJECT/AGENTS.md" + +cat > "$COMPANION" <<'TS' +import { type AssistantMessage, createAssistantMessageEventStream } from "@earendil-works/pi-ai"; +import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; +import { appendFileSync } from "node:fs"; + +const log = process.env.FM_COLLISION_EVENTS ?? "/dev/null"; +const preflightMs = Number(process.env.FM_COLLISION_PREFLIGHT_MS ?? "0"); +const replyMs = Number(process.env.FM_COLLISION_REPLY_MS ?? "0"); +const record = (line: string) => appendFileSync(log, `${line}\n`); +const label = (text: string) => text.includes("FIRSTMATE WATCHER WAKE: signal:") + ? "wake-signal" + : text.includes("FIRSTMATE WATCHER WAKE") ? "wake-other" : text.startsWith("CAPTAIN_") ? text.split(/\s/)[0] : "other"; +const text = (content: unknown): string => typeof content === "string" + ? content + : Array.isArray(content) ? content.filter((part) => part?.type === "text").map((part) => part.text).join("\n") : ""; + +export default function (pi: ExtensionAPI) { + pi.on("project_trust", () => ({ trusted: "yes", remember: false })); + pi.registerProvider("fm-collision", { + baseUrl: "http://127.0.0.1/unused", + apiKey: "test-only", + api: "fm-collision-api", + models: [{ id: "deterministic", name: "deterministic", reasoning: false, input: ["text"], cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }, contextWindow: 64000, maxTokens: 128 }], + streamSimple(model, context) { + const stream = createAssistantMessageEventStream(); + const users = context.messages.filter((message) => message.role === "user").map((message) => label(text(message.content))); + const reply = `REPLY_SEES ${users.slice(-2).join("+")}`; + const output: AssistantMessage = { + role: "assistant", content: [], api: model.api, provider: model.provider, model: model.id, + usage: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, totalTokens: 0, cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 } }, + stopReason: "stop", timestamp: Date.now(), + }; + setTimeout(() => { + stream.push({ type: "start", partial: output }); + output.content.push({ type: "text", text: reply }); + stream.push({ type: "done", reason: "stop", message: output }); + stream.end(); + }, replyMs); + return stream; + }, + }); + pi.on("session_start", async (event, ctx) => { + const model = ctx.modelRegistry.find("fm-collision", "deterministic"); + if (model) await pi.setModel(model); + record(`session_start ${(event as { reason?: string }).reason ?? ""}`); + }); + pi.on("input", (event) => { + record(`input ${event.source} ${label(event.text)}`); + record(`detail ${JSON.stringify(event.text.slice(0, 160))}`); + }); + pi.on("before_agent_start", async (event) => { + record(`before_agent_start ${label(event.prompt)}`); + await new Promise((resolve) => setTimeout(resolve, preflightMs)); + }); + pi.on("agent_start", () => record("agent_start")); + pi.on("message_start", (event) => { + if (event.message.role === "user") record(`user ${label(text(event.message.content))}`); + }); + pi.on("agent_settled", () => record("agent_settled")); +} +TS + +cat > "$LAUNCH" < $(printf %q "$HOME_DIR/state/.lock") +cd $(printf %q "$PROJECT") +exec env \\ + FM_HOME=$(printf %q "$HOME_DIR") \\ + FM_ROOT_OVERRIDE=$(printf %q "$ROOT") \\ + PI_CODING_AGENT_DIR=$(printf %q "$PI_DIR") \\ + PI_OFFLINE=1 \\ + FM_COLLISION_EVENTS=$(printf %q "$EVENTS") \\ + FM_COLLISION_PREFLIGHT_MS=$PREFLIGHT_MS \\ + FM_COLLISION_REPLY_MS=$REPLY_MS \\ + FM_POLL=1 \\ + FM_SIGNAL_GRACE=0 \\ + FM_HEARTBEAT=600 \\ + FM_CHECK_INTERVAL=999999 \\ + pi --approve --no-context-files --no-skills --no-prompt-templates --no-extensions --no-session \\ + -e $(printf %q "$COMPANION") \\ + -e $(printf %q "$ROOT/.pi/extensions/fm-primary-turnend-guard.ts") \\ + -e $(printf %q "$ROOT/.pi/extensions/fm-primary-pi-watch.ts") +EOF +chmod +x "$LAUNCH" + +capture() { + if [ "$TERMINAL" = herdr ]; then + "$HERDR_LAB_HELPER" run "$HERDR_LAB_SESSION" pane read "$PANE" --source recent --lines 2000 2>/dev/null || true + else + tmux -L "$SOCKET" capture-pane -p -t "$SESSION" -S -2000 2>/dev/null || true + fi +} + +type_line() { # + if [ "$TERMINAL" = herdr ]; then + "$HERDR_LAB_HELPER" run "$HERDR_LAB_SESSION" pane send-text "$PANE" "$1" >/dev/null + "$HERDR_LAB_HELPER" run "$HERDR_LAB_SESSION" pane send-keys "$PANE" enter >/dev/null + else + tmux -L "$SOCKET" send-keys -t "$SESSION" -l "$1" + tmux -L "$SOCKET" send-keys -t "$SESSION" Enter + fi +} + +event_count() { # [first line] + local count + count=$(tail -n +"${2:-1}" "$EVENTS" 2>/dev/null | grep -Fxc "$1") || true + printf '%s\n' "${count:-0}" +} + +event_lines() { + local lines + lines=$(wc -l < "$EVENTS" 2>/dev/null) || true + printf '%s\n' "${lines:-0}" +} + +wait_event() { #