Skip to content

fix(bin): stop capturing supervision engine notes as captain messages - #53

Merged
talktejas merged 4 commits into
mainfrom
fm/fm-capture-skip-helper-session
Oct 7, 2026
Merged

talktejas merged 4 commits into
mainfrom
fm/fm-capture-skip-helper-session

Conversation

@talktejas

Copy link
Copy Markdown
Owner

Intent

Reply to the captured message "The stale wake was for jt-metal-process-explained, which is on a declared pause. It is holding for..." : "what the fuck why the fuck u use a language and way of communicateing which is hard to understand."
Reply to the captured message "I reported this stale wake as a silent routine outcome (seq 61), so it won't open a turn for MAIN...": "what the fuck"

Context: since this home took the upstream merge (PR 52), a background supervision session runs beside the main firstmate session in the same Claude project directory. Its internal working notes (stale wake, seq numbers, MAIN, pane ids) are being recorded in data/captain-messages.jsonl with source "transcript" and shown in the command center as messages to the reader, sometimes with a wrong project label (project "oc" on a JewelTrek task). Only the main firstmate session's messages are meant for the reader.

What Changed

  • bin/fm-captain-message-sweep.py now drops any transcript whose file name matches a supervision engine session id before sweeping. The ids come from state/.supervision-host-engine-sessions plus the session= line in state/.supervision-host-engine, so the engine's working notes are no longer recorded in data/captain-messages.jsonl or shown in the command center.
  • bin/fm-supervision-host.sh appends each engine session id to state/.supervision-host-engine-sessions before that conversation's turn starts and never removes it, so a conversation stays excluded after a failed turn or after the host rotates to a new one.
  • Tests cover the sweep skip (current, listed-only, rotated and removed engine records) and the host listing the id before a turn, including one that fails. docs/command-center.md, docs/supervision-host.md and the operational-home-layout skill describe the new state file and the skip.

Risk Assessment

✅ Low: The change is a small additive filter in the sweep plus one idempotent append in the host before the engine's only launch site, and tracing new, resumed, rotated and failed-turn conversations shows each engine transcript is excluded from its first byte while main-session transcripts are still read.

Testing

Stood up a disposable lab home with a real Claude primary on its private tmux socket, parked the real supervision host under it, and let it run two real engine turns (sonnet, then haiku after a model change forced a new conversation). Swept the lab with the target code and the real command center, swept the same real transcripts with the base commit for comparison, captured API output and before/after screenshots, and ran the four existing tests that cover this change. All five scenarios driven live passed and the base commit reproduced the reported leak. Two scenarios are untested live: a really failing engine turn (stub-engine test only) and the real Stop-hook path.

  • Live validation: ✅ go - 5 of 7 scenarios driven live against the product
Scenario Result Live Evidence
Reader opens the command center while a supervision session has handled a wake beside the main session: only the main session's replies are listed ✅ pass live command-center-target.png, command-center-api.txt: target lists 2 messages, both from main session 5e77d2cd; capture record shows transcripts=1
Same real transcripts on the base commit: the supervision session's working notes appear as messages (reproduces the reported problem) ✅ pass live command-center-base.png, base-sweep-repro.txt: base lists 4 messages, 2 from engine session eabfa2ac ("...recorded as seq 1...", "...nothing more to do for this wake...")
A brand-new supervision conversation is swept during its very first turn, before any engine record exists: its session id is already listed and its transcript is skipped ✅ pass live host-live-run.txt (id listed, engine record absent, real engine process running), inflight-sweep.txt (transcripts=1, only main rows)
The supervision session rotates to a new conversation (engine model changed): the earlier conversation's notes are still never recorded ✅ pass live rotation-sweep.txt, command-center-target-after-rotation.png: record names 160c7546, list holds both ids, target still shows 2 main messages; base shows 5 including both engine sessions (command-cente…
The engine record disappears (as after a failed turn): previously launched conversations are still not recorded ✅ pass live rotation-sweep.txt, ADVERSARIAL section: record moved away by hand, target sweep leaves 2 rows, 0 from engine sessions. The removal was done by hand, not by a real failed turn.
An engine turn that really fails still leaves its session id listed ⏸️ untested no The prior payload did not establish a live result: a real failed engine turn was never provoked against the running product. The only coverage recorded was the existing test test_attended_latch_keeps_…
Main session's turn end records its reply through the real Claude Stop hook while the supervision host is parked by the Stop auto-arm ⏸️ untested no The gate worktree is a linked git worktree, so the tracked Stop and SessionStart hooks stand down there (bin/fm-primary-scope-lib.sh treats a linked worktree as not a primary; the lab primary's two tu…

Command center on this change: only the main session's two replies

Evidence: Command center /api/messages, target vs base, before and after rotation

Source: Command center /api/messages, target vs base, before and after rotation

== command center target-793c1eb /api/messages ==
{"capture":{"ok":true,"transcripts":null,"new":null,"named":0},"messages":[{"session":"5e77d2cd","source":"transcript","text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E"},{"session":"5e77d2cd","source":"transcript","text":"The fix landed on main."}]}
== command center base-5e730b3 /api/messages ==
{"capture":{"ok":true,"transcripts":null,"new":null,"named":0},"messages":[{"session":"eabfa2ac","source":"transcript","text":"That was the earlier file-search command finishing in the background. I already have what I needed from it, so"},{"session":"eabfa2ac","source":"transcript","text":"The demo worker's status line said \"working: step one\", so I took no action. I reported it as a routine, silen"},{"session":"5e77d2cd","source":"transcript","text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E"},{"session":"5e77d2cd","source":"transcript","text":"The fix landed on main."}]}
== after rotation: command center target-793c1eb /api/messages ==
[{"session":"5e77d2cd","text":"Captain, the supervision host is parked and running in the background, with its output goi"},{"session":"5e77d2cd","text":"The fix landed on main."}]
== after rotation: command center base-5e730b3 /api/messages ==
[{"session":"160c7546","text":"**Wake handled complete.**\n\nDrained the signal wake for task `demo`. The status shows heal"},{"session":"eabfa2ac","text":"That was the earlier file-search command finishing in the background. I already have what "},{"session":"eabfa2ac","text":"The demo worker's status line said \"working: step one\", so I took no action. I reported it"},{"session":"5e77d2cd","text":"Captain, the supervision host is parked and running in the background, with its output goi"},{"session":"5e77d2cd","text":"The fix landed on main."}]
Evidence: Sweep during the first engine turn (no engine record yet)

Source: Sweep during the first engine turn (no engine record yet)

engine process alive: yes; engine record present: no sweep rc=0 capture: transcripts=1 new=2 rows: 2, both session 5e77d2cd (main): "The fix landed on main." and "Captain, the supervision host is parked..."

engine process alive: yes; engine record present: no
sweep rc=0
{"active": true, "dir": "/tmp/fm-lab.E5NDXu/cc/projects/-tmp-fm-lab-E5NDXu", "transcripts": 1, "new": 2, "named": 0, "at": "2026-10-07T06:41:45Z", "ok": true, "error": null}
{"source":"transcript","session":"5e77d2cd-6e79-4a7e-b6ab-58e897c92dba","project":null,"text":"The fix landed on main."}
{"source":"transcript","session":"5e77d2cd-6e79-4a7e-b6ab-58e897c92dba","project":null,"text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E5NDXu/host.out`; I have not wa"}
Evidence: Rotation and removed-record sweeps, target vs base

Source: Rotation and removed-record sweeps, target vs base

# after a live rotation (engine model changed sonnet -> haiku): lab state
state/.supervision-host-engine-sessions:
eabfa2ac-b054-4821-909d-6edbb6169fce
160c7546-eade-4801-92ea-6293f328193e
state/.supervision-host-engine:
engine=claude
model=haiku
session=160c7546-eade-4801-92ea-6293f328193e
key=96966:219154337:4294967295
turns=1
conversation_cost=0.056084

# transcripts in the swept directory:
160c7546-eade-4801-92ea-6293f328193e.jsonl
5e77d2cd-6e79-4a7e-b6ab-58e897c92dba.jsonl
eabfa2ac-b054-4821-909d-6edbb6169fce.jsonl

# TARGET 793c1eb sweep of the lab home
rc=0
{"active": true, "dir": "/tmp/fm-lab.E5NDXu/cc/projects/-tmp-fm-lab-E5NDXu", "transcripts": 1, "new": 0, "named": 0, "at": "2026-10-07T06:46:39Z", "ok": true, "error": null}
{"session":"5e77d2cd","text":"The fix landed on main."}
{"session":"5e77d2cd","text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E5NDXu/host"}

# ADVERSARIAL: engine record removed (what the host does after a failed turn), then TARGET sweep again
rc=0
{"active": true, "dir": "/tmp/fm-lab.E5NDXu/cc/projects/-tmp-fm-lab-E5NDXu", "transcripts": 1, "new": 0, "named": 0, "at": "2026-10-07T06:46:39Z", "ok": true, "error": null}
rows: 2, from engine sessions: 0

# BASE 5e730b3 sweep, same transcripts, same rotated record
rc=0
{"active": true, "dir": "/tmp/fm-basehome.yNwHxz/cc/projects/-tmp-fm-basehome-yNwHxz", "transcripts": 3, "new": 1, "named": 0, "at": "2026-10-07T06:46:39Z", "ok": true, "error": null}
{"session":"5e77d2cd","text":"The fix landed on main."}
{"session":"5e77d2cd","text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E5NDXu/host"}
{"session":"eabfa2ac","text":"The demo worker's status line said \"working: step one\", so I took no action. I reported it as a routine, silent outcome,"}
{"session":"eabfa2ac","text":"That was the earlier file-search command finishing in the background. I already have what I needed from it, so there's n"}
{"session":"160c7546","text":"**Wake handled complete.**\n\nDrained the signal wake for task `demo`. The status shows healthy progress: step one complet"}
Evidence: Base-commit sweep reproducing the leak

Source: Base-commit sweep reproducing the leak

# base commit 5e730b3 sweep, same two real transcripts (main 5e77d2cd, engine eabfa2ac)
sweep rc=0
{"active": true, "dir": "/tmp/fm-basehome.yNwHxz/cc/projects/-tmp-fm-basehome-yNwHxz", "transcripts": 2, "new": 4, "named": 0, "at": "2026-10-07T06:45:36Z", "ok": true, "error": null}
{"source":"transcript","session":"5e77d2cd","project":null,"text":"The fix landed on main."}
{"source":"transcript","session":"5e77d2cd","project":null,"text":"Captain, the supervision host is parked and running in the background, with its output going to `/tmp/fm-lab.E5NDXu/host.out`; I have not waited on it or checked its result."}
{"source":"transcript","session":"eabfa2ac","project":null,"text":"The demo worker's status line said \"working: step one\", so I took no action. I reported it as a routine, silent outcome, which was recorded as seq 1, and then ran the drain's acknowledgement command.\n\nI claimed no leases because I"}
{"source":"transcript","session":"eabfa2ac","project":null,"text":"That was the earlier file-search command finishing in the background. I already have what I needed from it, so there's nothing more to do for this wake: the demo event is reported and acknowledged."}
Evidence: Real host log: two handled engine turns, and the id listed before the first turn's record

Source: Real host log: two handled engine turns, and the id listed before the first turn's record

# real host log in the lab home (two real engine turns)
1791355006	start	gen=host-74879-1791355005	primary=claude
1791355518	handled	turn=host-74879-1791355005.1	posture=away	rc=0	reports=1	error=0 cost=0.062597 conversation_cost=0.062597 input=4 cache_read=16898 cache_write=588 output=57 turns=1	signal: /tmp/fm-lab.E5NDXu/state/demo.status
1791355572	handled	turn=host-74879-1791355005.2	posture=away	rc=0	reports=1	error=0 cost=0.056084 conversation_cost=0.056084 input=74 cache_read=115510 cache_write=16382 output=2339 turns=12	signal: /tmp/fm-lab.E5NDXu/state/demo.status

# engine launch seen while turn 1 was still running, before any engine record existed
== at first sight of the list (t=3x3s) ==
eabfa2ac-b054-4821-909d-6edbb6169fce
engine record present: no
ls: cannot access '~/.claude/projects/-home-tds--no-mistakes-worktrees-3605d2c32b02-01M4AGC3NYSJSVFS84
23951 perl -MPOSIX=WNOHANG,setpgid -MTime::HiRes=time -e        my ($bound, $gra
23995 ~/.local/bin/claude -p FIRSTMATE SUPERVISION WAKE: signal: /tmp/fm
Evidence: Engine record after turn 1

Source: Engine record after turn 1

engine=claude
model=sonnet
session=eabfa2ac-b054-4821-909d-6edbb6169fce
key=96966:219154337:4294967295
turns=1
conversation_cost=0.062597
Evidence: The four targeted existing tests (non-live; stub engine)

Source: The four targeted existing tests (non-live; stub engine)

ok - the supervision engine's own transcript is identified by session identity and never captured
ok - a transcript the payload names is captured wherever it lives
rc=0
ok - host: an away wake is handled on the engine through the branch contract and never reaches main
ok - host: attended, two engine errors latch the session, main keeps every close unchanged in the cooldown, a failed probe doubles it up to its cap, and a routine probe's recovery stays in the ledger, off main
rc=0

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • 🚨 bin/fm-captain-message-sweep.py:761 - The fix only excludes the ONE session id currently in state/.supervision-host-engine, and it deletes that transcript's cursor entry (cursor["files"].pop). As soon as the record names a different session or is removed, the previous engine transcript is an ordinary unknown file again and is read from offset 0, so every working note in it since the floor is recorded in one burst. bin/fm-supervision-host.sh does this routinely: rotation after ROTATE_TURNS=20 turns, a main-session key change, an engine/model change, and rm -f "$ENGINE_RECORD" on any failed turn (lines 988, 997). Reproduced in a temp home: record session=old -> old.jsonl skipped; record rewritten to session=new -> next directory sweep appended "stale wake seq 61 old engine note" with source "transcript". This is exactly the behavior the intent says must stop ("Only the main firstmate session's messages are meant for the reader"), delayed by up to 20 turns. Remedy inside the sweep: make the exclusion sticky instead of popping - keep the cursor entry marked as an engine transcript (e.g. {"engine": true}, cleared of named) and drop every such path from targets on every run, so a transcript once identified as the engine's is never read after the record moves on. Extend the new test with a record that rotates to a second session id.
  • 🚨 bin/fm-captain-message-sweep.py:182 - The session id is written to state/.supervision-host-engine only AFTER a turn that succeeded (write_engine_record at bin/fm-supervision-host.sh:987). A new engine conversation (--session-id <fresh uuid>, chosen at line 735) is therefore unidentified for its whole first turn, and permanently if that turn fails (no receipt, unacked rows, timeout, error -> rm -f "$ENGINE_RECORD"), so its transcript is swept by the command-center backstop like main's. Reproduced: record absent or still naming the previous session, new.jsonl in the transcript directory -> "stale wake seq 62 NEW engine note" recorded. Failed turns are the ones most likely to carry confusing notes. The sweep cannot close this alone; the host has to make the session id known before the engine starts (for example an append-only list of engine session ids written in handle_wake before fm_supervision_engine_turn, which the sweep reads instead of the single session= line). That adds new durable state and touches the upstream-owned host script, so the remedy, not the defect, is what needs authorization.
  • ⚠️ bin/fm-captain-message-sweep.py:764 - The self-heal only forgets the cursor entry; engine rows already written to data/captain-messages.jsonl (the "stale wake ... seq 61 ... MAIN" messages the intent quotes, including the ones labelled project "oc") are left in the log and the command center keeps showing them. The intent describes them as "being recorded ... and shown"; decide whether already-recorded engine rows should be removed or hidden (rows carry session, so they can be matched to the engine's session id), or whether stopping new ones is enough.
  • ⚠️ bin/fm-captain-message-sweep.py:882 - Simplification: the FM_SUPERVISION_ACTOR == "branch" early return in main() is a second, parallel definition of "this is the engine" that no intent requirement needs. The engine runs with --safe-mode, which loads none of the home's hooks (docs/supervision-host.md:355), so the Stop hook never fires this script from the engine; the leak came from the command-center's directory sweep, which runs without that variable. No test exercises the guard. Recommend removing it and keeping session identity as the single rule.

🔧 Fix applied.
1 warning still open:

  • ⚠️ bin/fm-captain-message-sweep.py:774 - Simplification: the cursor purge in sweep() (lines 774-779: collect remembered, delete those entries, write_cursor, recompute named) is not needed by any intent requirement. Line 773 already drops every engine transcript from targets, including ones the cursor remembers as named, so nothing is ever read from them whether or not their cursor entry survives. The purge only changes the reported named count, and it covers a state that intended usage does not produce: an engine transcript can only become named through a Stop-hook payload, and the engine runs with --safe-mode, which loads no hooks. Recommend removing the purge and the two test sections that exist only for it (the payload naming the engine's transcript, and the hand-injected named cursor entry) in tests/command-center.test.sh.

🔧 Fix applied.
1 info still open:

  • ℹ️ bin/fm-supervision-host.sh:145 - The header's "STATE (all under state/, owned here)" inventory lists every host-owned state file but not the new state/.supervision-host-engine-sessions (append-only, one line per engine conversation, never removed, read by bin/fm-captain-message-sweep.py). Add it to that list so the file's owner and its one outside reader are recorded where the others are.
✅ **Test** - passed

✅ No issues found.

  • Live validation: ✅ go - 5 of 7 scenarios driven live against the product
Scenario Result Live Evidence
Reader opens the command center while a supervision session has handled a wake beside the main session: only the main session's replies are listed ✅ pass live command-center-target.png, command-center-api.txt: target lists 2 messages, both from main session 5e77d2cd; capture record shows transcripts=1
Same real transcripts on the base commit: the supervision session's working notes appear as messages (reproduces the reported problem) ✅ pass live command-center-base.png, base-sweep-repro.txt: base lists 4 messages, 2 from engine session eabfa2ac ("...recorded as seq 1...", "...nothing more to do for this wake...")
A brand-new supervision conversation is swept during its very first turn, before any engine record exists: its session id is already listed and its transcript is skipped ✅ pass live host-live-run.txt (id listed, engine record absent, real engine process running), inflight-sweep.txt (transcripts=1, only main rows)
The supervision session rotates to a new conversation (engine model changed): the earlier conversation's notes are still never recorded ✅ pass live rotation-sweep.txt, command-center-target-after-rotation.png: record names 160c7546, list holds both ids, target still shows 2 main messages; base shows 5 including both engine sessions (command-cente…
The engine record disappears (as after a failed turn): previously launched conversations are still not recorded ✅ pass live rotation-sweep.txt, ADVERSARIAL section: record moved away by hand, target sweep leaves 2 rows, 0 from engine sessions. The removal was done by hand, not by a real failed turn.
An engine turn that really fails still leaves its session id listed ⏸️ untested no The prior payload did not establish a live result: a real failed engine turn was never provoked against the running product. The only coverage recorded was the existing test test_attended_latch_keeps_…
Main session's turn end records its reply through the real Claude Stop hook while the supervision host is parked by the Stop auto-arm ⏸️ untested no The gate worktree is a linked git worktree, so the tracked Stop and SessionStart hooks stand down there (bin/fm-primary-scope-lib.sh treats a linked worktree as not a primary; the lab primary's two tu…
  • bin/fm-lab-home.sh create "$LAB" + tmux -L fm-lab new-session ... -e FM_HOME="$LAB" claude (real primary, two real turns)
  • FM_HOME=$LAB bin/fm-afk-contract.sh enter --words ... then primary ran FM_SUPERVISION_HOST_PRIMARY=claude bin/fm-supervision-host.sh park in the background
  • Appended working [at=...]: step one to $LAB/state/demo.status -> real engine turn 1 (claude -p ... --session-id eabfa2ac...), host log handled
  • Read state/.supervision-host-engine-sessions while turn 1 was still running and state/.supervision-host-engine did not exist
  • CLAUDE_CONFIG_DIR=$LAB/cc python3 bin/fm-captain-message-sweep.py --home $LAB during turn 1, after turn 1, after rotation, and with the engine record moved away
  • echo 'claude haiku' > $LAB/config/supervision-host + second status append -> real engine turn 2 on a new conversation (160c7546...)
  • python3 bin/command-center.py --port 18765 --home $LAB -> curl localhost:18765/api/messages and page screenshots
  • Base commit 5e730b3 (git archive 5e730b3 bin) sweep and command center on port 18766 over the same real transcripts
  • Only test_the_supervision_engines_own_transcript_is_never_captured and test_a_transcript_the_payload_names_is_captured_wherever_it_lives from tests/command-center.test.sh
  • Only test_away_wake_is_handled_on_the_engine_and_never_reaches_main and test_attended_latch_keeps_closes_on_main_and_records_recovery_off_main from tests/fm-supervision-host.test.sh
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

…weep

bin/fm-supervision-host.sh runs a headless branch engine beside main in the
same per-home transcript directory. Its working notes (stale wake, seq
numbers, MAIN, pane ids) were being swept and shown to the reader as chat.
The sweep now identifies that engine's transcript by session identity - the
session id state/.supervision-host-engine records for it - excludes it from
directory sweeps and payload naming, and forgets it from the remembered
named transcripts so a home already carrying the stale state self-heals on
its next run.
@talktejas
talktejas merged commit e640c10 into main Oct 7, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant