Skip to content

feat(bin): record per-card worker time in a work ledger and report only drift edges - #8

Merged
matthewstrud merged 2 commits into
mainfrom
fm/fm-work-ledger
Sep 18, 2026
Merged

matthewstrud merged 2 commits into
mainfrom
fm/fm-work-ledger

Conversation

@matthewstrud

Copy link
Copy Markdown
Owner

Intent

We need to be able to see whether agent-run work is drifting - whether a card is taking far longer than its difficulty should warrant - while it is still happening, rather than reconstructing it afterwards.
The measurement has to be a hook rather than a habit, and it must not require anyone, human or agent, to remember to record anything.
What is measured is difficulty per active hour together with tasks completed - not lines, and not tasks per hour.
One constraint on the approved design, verbatim: "Let's just make sure that does not ping you too often."

What changed

  • bin/fm-busy-event.sh - inside its existing lock, after the record write (or the retirement) succeeded, arm, apply and retire append one row to <state>/work-ledger/<id>.events. A failed append never changes the exit code or output; it is counted in work-ledger/.errors.
  • bin/fm-spawn.sh - one spawn row per launch (relaunches included) with harness, model, kind, parent, capture=supported|unsupported, and the card's frozen rating read from (rating: ...) / (parent: ...) fields in the backlog title. No rating records rating=none; the card is unrated, never dropped.
  • bin/fm-pr-check.sh, bin/fm-merge-outcome-lib.sh - pr-ready and merged rows, stamped by firstmate's clock. The merged row shares the merge outcome's existing deduplication.
  • bin/fm-work-ledger-lib.sh (new) - single owner of the row format and the never-fails-a-turn append.
  • bin/fm-work-ledger.sh (new) - copy, check, arm, disarm. Copies local homes' ledgers into data/work-ledger/<lane>/, keeps a cursor, and prints only edges. arm writes the static state/work-ledger.check.sh shim, registers it through bin/fm-check-register.sh, and refuses in a second mate's home.
  • bin/fm-teardown.sh - retiring a second mate's home copies its ledger first and refuses the removal when the copy fails.
  • A short agent-only work-ledger skill (what to do on each wake, the title fields), one trigger line and three layout lines in AGENTS.md, and a section in docs/configuration.md.

How often it pings

Edge-triggered only. There is no timer and no periodic summary: elapsed time alone never produces output.
Every line is recorded in the cursor before it is printed, and a line is printed only if the cursor landed, so a store that cannot be written goes quiet rather than repeating every sweep.

Edge Fires
over-budget once at 3x and once at 6x per card, only while the card has a worker in flight; a card crossing both in one sweep reports only 6x
capture dead once per episode, after 2 consecutive lagging sweeps for every in-flight task in a home; re-arms on recovery
digest at most one per lane per sweep, each time 5 more rated cards have merged
check error once per distinct failure

Nothing is reported per merge, per harness, per model, or per agent.

Two easy-to-get-wrong points

  • A harness whose turns cannot be observed is recorded capture=unsupported and the card is unmeasured, never zero. Codex children stay hooks-disabled; nothing here tries to enable them. "Supported" is read from whether spawn armed the busy-state contract, so it cannot drift from spawn's own decision.
  • These are turn-bracketed minutes, not the transcript-gap method of the existing 13-card series. Nothing is spliced or backfilled. The inherited 9.9 min/pt median is used only as the over-budget threshold until the store holds 13 of its own rated, complete, merged cards, and every over-budget line labels the median post hoc and says which one it used.

Decisions worth a second look

  • Rows carry an explicit row= kind (arm|turn|retire|spawn|pr-ready|merged), which the report's example row did not have; the reader needs it.
  • The rating lives in the backlog title, placed before (kind: ...). I checked tasks-axi 0.2.5: there the fields survive reads and state transitions, whereas after (kind: ...) they break kind parsing. Spawn reads the title with show --full because plain show truncates long titles.
  • "A turn-closing event with no turn open" marks a card incomplete only for stop, stop-failure and after-agent. An idle event arriving while already idle is otherwise normal - a live Claude turn ends stop then session-end - so a broader rule would have marked almost every card incomplete.
  • The check is not auto-armed. A registered check counts as a standing reason to keep watching, which would change every user's primary home; bin/fm-work-ledger.sh arm is a one-time step in the primary home.

Verification

  • tests/fm-work-ledger.test.sh (new, 17 cases): rows follow the record's seq; a stale incarnation is not appended; a relaunch keeps both incarnations in one file; a failed append is silent, counted, and exits 0; 12 concurrent appends neither interleave nor skip a seq; the check is silent on repeated runs when nothing changed; copy is idempotent and ignores a partial trailing line; over-budget fires once per level; sub-cards fold into the parent by union, not sum; unrated and unmeasured cards never alarm; capture dead needs two sweeps and fires once per episode; a seq gap and a live record running ahead are detected; no per-merge output and no digest before five cards; a second mate's lane is copied without writing to it; arm is primary-only; the retirement copy fails closed; a merge is stamped once.
  • tests/fm-teardown.test.sh: teardown leaves work-ledger/ intact, including the PR-ready row from the real fm-pr-check.sh.
  • tests/fm-secondmate-lifecycle-e2e.test.sh: retirement refuses while the ledger cannot be copied, then keeps it after the home is removed.
  • tests/fm-busy-adapter-wiring.test.sh: a real fm-spawn records the frozen rating and parent, an unrated card, and Codex as capture=unsupported with no turn rows.
  • Live, Claude Code 2.1.276: a real claude -p turn with the same hook shape spawn installs wrote arm, user-prompt-submit, stop, session-end rows in order.
  • FM_LINT_JOBS=1 bin/fm-lint.sh, bin/fm-doc-audience-check.sh, and bin/fm-test-run.sh --check-coverage pass.

Not done here: the live re-check on Pi 0.85.1 and the five-card calibration overlap against the transcript method (report section 6, item 6). Both need real cards to run.

Follow-up work, deliberately not in this change

Any idle alarm; capture for Codex, Cursor, muse, Grok, Rovo, Kimi or agy; the orchestrator's own time; remote second mates; tokens, cost, lines, test or step counts; any per-harness, per-model or per-agent aggregate; trend alarms; any automatic action on a card; the plan-drift consumer.
Also noticed: a local-only landing through bin/fm-merge-local.sh writes no merged row, so those cards never complete in the ledger; and a second mate retiring its own child home copies into its own store, which the primary does not read.

matt added 2 commits September 18, 2026 19:07
…ly drift edges

Append one row per turn boundary, dispatch, PR-ready registration, and merge
to state/work-ledger/<id>.events from the scripts that already run at those
moments, so a card's cost is on record without anyone remembering to write it.
Capture never blocks or fails a turn, and a harness that reports no turn
boundaries is recorded as unmeasured rather than as zero.

bin/fm-work-ledger.sh copies local homes' ledgers into the primary home's
store and is a registered check that prints only edges: a card crossing 3x or
6x its rating's budget, capture going dead for a home, and a five-card lane
digest. A run in which nothing changed prints nothing. Retiring a second mate
copies its ledger first and refuses when the copy fails.
Copilot AI lite review requested due to automatic review settings September 18, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Critical ledger-preservation issues and moderate correctness gaps remain unresolved.

Pull request overview

Adds an edge-triggered per-card work ledger for tracking worker time, lifecycle events, and drift reporting.

Changes:

  • Records turn, spawn, PR-ready, merge, and retirement events.
  • Adds ledger copying, validation, cursor-based reporting, and primary-only arming.
  • Adds lifecycle tests, documentation, and agent guidance.
File summaries
File Description
tests/fm-work-ledger.test.sh Tests ledger behavior and reporting.
tests/fm-teardown.test.sh Tests teardown preservation.
tests/fm-secondmate-lifecycle-e2e.test.sh Tests secondmate retirement handling.
tests/fm-busy-adapter-wiring.test.sh Tests spawn and capture wiring.
docs/documentation-audiences.json Registers documentation audiences.
docs/configuration.md Documents ledger configuration.
bin/fm-work-ledger.sh Copies ledgers and evaluates drift edges.
bin/fm-work-ledger-lib.sh Defines ledger rows and append behavior.
bin/fm-test-run.sh Registers the ledger tests.
bin/fm-teardown.sh Preserves ledgers during teardown.
bin/fm-spawn.sh Records spawn metadata and ratings.
bin/fm-pr-check.sh Records PR-ready events.
bin/fm-merge-outcome-lib.sh Records merge outcomes.
bin/fm-busy-event.sh Records worker lifecycle events.
AGENTS.md Adds ledger workflow guidance.
.agents/skills/work-ledger/SKILL.md Defines ledger wake handling.
Review details
  • Files reviewed: 8/16 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@matthewstrud
matthewstrud merged commit 911c070 into main Sep 18, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants