From 3ac9b3d4b6c547a427bd85dd5a2b99bc4f367a14 Mon Sep 17 00:00:00 2001 From: Tejas Shah Date: Tue, 6 Oct 2026 01:58:12 +0700 Subject: [PATCH 1/3] fix(bin): offer the pause recheck of a worker whose agent has exited to routine-wake triage The pause recheck is reached for an ordinary crew only once its agent is confidently dead, and the triage refused every agent that was not alive, so the recheck it was wired to could never be offered. A worker whose agent has exited beside an open pull request is now offered, the model is told which, and the question reads a wait for the merge word of that pull request as the already reported merge whoever gives it. --- bin/fm-watch.sh | 28 ++++++++++++++++++++-------- docs/configuration.md | 8 +++++--- tests/fm-jev-wake-triage.test.sh | 30 ++++++++++++++++++++++++++---- 3 files changed, 51 insertions(+), 15 deletions(-) diff --git a/bin/fm-watch.sh b/bin/fm-watch.sh index 72484c0037a..039bd233bd2 100755 --- a/bin/fm-watch.sh +++ b/bin/fm-watch.sh @@ -1663,7 +1663,12 @@ surface_nonterminal_stale() { # # jev_triage_enabled refuses while the away daemon or the away-posture record # exists, and jev_triage_task_evidence refuses a secondmate, a task with any # open keyed decision, a declared clearing time that has passed, an endpoint -# whose agent is not proven alive, and a task with no recorded pull request. +# whose agent state is anything but alive or dead, and a task with no recorded +# pull request. A dead agent beside an existing endpoint is offered because that +# is the state pause_state_class routes to the pause recheck for an ordinary +# crew, so refusing it left that recheck with no task it could ever offer; the +# evidence tells the model the agent exited, and the question keeps such a wait +# routine only when it is the merge of the open pull request. # # The recorded pull request (pr= in the task's metadata, which only firstmate # records after reading the worker's ready report) is what shows firstmate @@ -1671,7 +1676,9 @@ surface_nonterminal_stale() { # # surfaces exactly as before. The deterministic read comes first and settles # what it can: the task is offered only while bin/fm-pr-state.sh reads that pull # request as open with no blocker reported, so a merged, closed, conflicting, -# failing, or unreadable one is delivered without asking. +# failing, or unreadable one is delivered without asking. A repository that +# reports no checks at all is not a blocker: only a check that reported and is +# failing or pending is. # # ponytail: one consecutive-absorb counter per subject bounds the model the way # PAUSE_RESURFACE_SECS bounds the deterministic absorbs - after @@ -1681,8 +1688,8 @@ JEV_TRIAGE_MAX_STREAK=6 JEV_TRIAGE_PR_TIMEOUT=30 FM_PR_STATE_BIN="${FM_PR_STATE_BIN:-$SCRIPT_DIR/fm-pr-state.sh}" # shellcheck disable=SC2016 # The backticks are literal markup in the question text. -JEV_TRIAGE_INSTRUCTIONS='A monitoring reminder in `wake` is about to interrupt the supervisor of a fleet of software workers. The supervisor has already seen every status event listed in `wake`, so nothing in it is news. Decide from the words of the evidence what is being waited on. Choose `routine` only when the wait will clear without any action from the supervisor or another person, apart from the already reported merge of an open pull request. When the worker is waiting on a person or on the supervisor to choose, answer, review, approve, or unblock something, or it stopped after a failure, the reminder is how the supervisor remembers that someone still owes the worker a reply, so choose `needs_firstmate`. Choose `needs_firstmate` whenever you are unsure.' -JEV_TRIAGE_CRITERIA='{"needs_firstmate":"The newest status event shows the worker waiting on a person or on the supervisor: to choose between options, answer a question, review or accept finished work, supply or repair a credential or access, or unblock it; or the worker stopped after a failure or an error; or a diagnostic reports anything other than a read that timed out; or it is not clear what the wait is for.","routine":"The newest status event declares a wait on an automatic external event that clears without the supervisor: an open pull request with nothing failing that only awaits its already reported merge, a running CI job, a rate limit or quota reset, a scheduled window, or an upstream release; or one transient read timeout that the monitor retries on its own."}' +JEV_TRIAGE_INSTRUCTIONS='A monitoring reminder in `wake` is about to interrupt the supervisor of a fleet of software workers. The supervisor has already seen every status event listed in `wake`, so nothing in it is news. Decide from the words of the evidence what is being waited on. Choose `routine` only when the wait will clear without any action from the supervisor or another person, apart from the already reported merge of an open pull request. A worker that says only that its open pull request is waiting to be merged, or waiting for the merge word or merge approval of the supervisor, the captain, or anyone else, is waiting on that already reported merge, so that is `routine`. When the worker is waiting on a person or on the supervisor to choose, answer, review, approve, or unblock anything other than that merge, or it stopped after a failure, the reminder is how the supervisor remembers that someone still owes the worker a reply, so choose `needs_firstmate`. When `worker_agent` is `exited`, the worker cannot resume by itself: that changes nothing for a pull request that only awaits its merge, which needs nothing more from the worker and stays `routine`, but every other wait is then `needs_firstmate`. Choose `needs_firstmate` whenever you are unsure.' +JEV_TRIAGE_CRITERIA='{"needs_firstmate":"The newest status event shows the worker waiting on a person or on the supervisor for anything other than the merge of its open pull request: to choose between options, answer a question, review or accept finished work, supply or repair a credential or access, or unblock it; or the worker stopped after a failure or an error; or `worker_agent` is `exited` and the wait is for anything other than the merge of its open pull request; or a diagnostic reports anything other than a read that timed out; or it is not clear what the wait is for.","routine":"The newest status event declares an open pull request with nothing failing that only awaits its already reported merge, whoever gives the merge word and whether `worker_agent` is `running` or `exited`; or, while `worker_agent` is `running`, a wait on an automatic external event that clears without the supervisor: a running CI job, a rate limit or quota reset, a scheduled window, or an upstream release; or one transient read timeout that the monitor retries on its own."}' JEV_TRIAGE_TASK_EVIDENCE= jev_triage_enabled() { @@ -1694,7 +1701,7 @@ jev_triage_enabled() { # 0 when may be offered to Jev, leaving its evidence object in # JEV_TRIAGE_TASK_EVIDENCE; 1 for every task the model must never decide. jev_triage_task_evidence() { # - local task=$1 meta statusf last until w pr pr_out + local task=$1 meta statusf last until w pr pr_out agent JEV_TRIAGE_TASK_EVIDENCE= case "$task" in ''|*[!A-Za-z0-9._-]*) return 1 ;; esac meta="$STATE/$task.meta" @@ -1711,14 +1718,19 @@ jev_triage_task_evidence() { # [ -n "$pr" ] || return 1 w=$(fm_backend_target_of_meta "$meta") [ -n "$w" ] || return 1 - [ "$(fm_backend_agent_state "$(fm_backend_of_meta "$meta")" "$w" 2>/dev/null || true)" = alive ] || return 1 + case "$(fm_backend_agent_state "$(fm_backend_of_meta "$meta")" "$w" 2>/dev/null || true)" in + alive) agent='running' ;; + dead) agent='exited' ;; + *) return 1 ;; + esac pr_out=$(fm_run_timed "$JEV_TRIAGE_PR_TIMEOUT" "$FM_PR_STATE_BIN" "$pr" 2>/dev/null) || return 1 if [ -n "$pr_out" ] && printf '%s\n' "$pr_out" | grep -Eqv '^(CHECKS: |MERGEABILITY: unknown$)'; then return 1 fi JEV_TRIAGE_TASK_EVIDENCE=$(tail -n 6 "$statusf" 2>/dev/null | cut -c 1-400 \ - | jq -Rsc --arg task "$task" --arg pr 'open; no blocker reported' \ - '{task: $task, pull_request: $pr, recent_status_events: (split("\n") | map(select(length > 0)))}' \ + | jq -Rsc --arg task "$task" --arg pr 'open; no blocker reported' --arg agent "$agent" \ + '{task: $task, pull_request: $pr, worker_agent: $agent, + recent_status_events: (split("\n") | map(select(length > 0)))}' \ 2>/dev/null) || return 1 [ -n "$JEV_TRIAGE_TASK_EVIDENCE" ] } diff --git a/docs/configuration.md b/docs/configuration.md index cb1ff918200..93cff28604f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -231,16 +231,18 @@ Only two enumerated wake classes are ever offered, each wired at its own point i No signal is ever offered, so every new status event and every bare turn-end surfaces as it does without the feature. A paused task is eligible only when a pull request is recorded for the task (`pr=` in its metadata) and is still open with no blocker reported; a paused task with no recorded pull request is never offered and surfaces exactly as it does without the feature. -The question asks what the declared wait is for: a wait on an automatic external event may read as routine, while a worker waiting on a person or stopped after a failure is delivered. +The question asks what the declared wait is for: an open pull request that only awaits its merge word, whoever gives it, or a wait on an automatic external event may read as routine, while a worker waiting on a person for anything else or stopped after a failure is delivered. Everything else is refused in code before any call and never depends on the model's answer. That covers captain notes, every new status event, every turn-end, merge and PR-ready outcomes, process-event and Relay wakes, every other check, any task with an open keyed decision, a secondmate, a declared clearing time that has passed, and every wake while the away-mode daemon or the away-posture record exists. -An endpoint whose agent is not proven alive is never routine, so a dead, missing, ambiguous, unreadable, or unverified endpoint is always delivered. +An endpoint whose agent state cannot be established is never routine, so a missing, ambiguous, unreadable, or unverified endpoint is always delivered. +A worker whose agent is running, or has exited while its endpoint remains, is offered, and the model is told which; for an exited agent the question keeps only a pull request awaiting its merge routine, since that worker cannot resume any other wait by itself. The watcher first reads the recorded pull request through `bin/fm-pr-state.sh` and offers the wake only while that read reports the pull request open with no blocker, which is all the model is told about it; a merged, closed, blocked, or unreadable pull request delivers the wake without asking, so an absorbed pause recheck is still presented on its normal cadence once the wait has actually changed. +A repository that reports no checks at all is not a blocker, while a draft, a conflict, a requested change, and a required check that is failing or pending each are. A routine answer advances the same recheck cadence a delivered wake would, so the wait is read again every `FM_PAUSE_RESURFACE_SECS` rather than silenced. One task may receive six routine answers in a row before the next wake is delivered unasked and the count restarts, so the model can never mute a wait indefinitely. -Each call sends the wake's class and reason line, the task id, its last six status lines cut to 400 characters each, and its pull request read, or for a contributions check its diagnostic lines; it sends no brief, pane content, or quota. +Each call sends the wake's class and reason line, the task id, its last six status lines cut to 400 characters each, its pull request read, and whether its agent is running or has exited, or for a contributions check its diagnostic lines; it sends no brief, pane content, or quota. `bin/fm-jev-lib.sh` owns the request, the answer validation, and the key handling shared with typed dispatch resolution, `bin/fm-watch.sh`'s `jev_triage_routine` header owns the exact eligibility gates, and [`verification/jev-wake-triage.md`](verification/jev-wake-triage.md) records the live evidence. ## House-rules check (.env TYPESAFE_API_KEY, config/house-rules.json) diff --git a/tests/fm-jev-wake-triage.test.sh b/tests/fm-jev-wake-triage.test.sh index e62d08f0e33..411d9447e31 100755 --- a/tests/fm-jev-wake-triage.test.sh +++ b/tests/fm-jev-wake-triage.test.sh @@ -308,7 +308,7 @@ jev_triage_pause_routine park "stale: test:fm-park" || fail "a closed decision b pass "never eligible: needs-decision, blocked, failed, done, working, captain-held, or an open keyed decision" # -- never eligible: endpoint, kind, posture, and shape gates ----------------- -for state in dead missing ambiguous unreadable unverified; do +for state in missing ambiguous unreadable unverified; do reset mk_task park "pr=$PR_URL" 'paused: waiting on the upstream release' AGENT_STATE=$state @@ -339,7 +339,26 @@ mk_task park "pr=$PR_URL" 'paused: waiting for the window until 2020-01-01T00:00 rc=0 jev_triage_pause_routine park "stale: test:fm-park" || rc=$? expect_never_asked "$rc" "a declared clearing time that has passed" -pass "never eligible: a dead, missing, or unproven endpoint, a secondmate, the away posture, a passed clearing time" +pass "never eligible: a missing or unproven endpoint, a secondmate, the away posture, a passed clearing time" + +# -- eligible: a worker whose agent has exited beside its open pull request ---- +# pause_state_class routes an ordinary crew to the pause recheck only once its +# agent is confidently dead, so this is the recheck's ordinary case. +reset +mk_task park "pr=$PR_URL" "done: PR $PR_URL" "paused: pull request $PR_URL open, waiting for the captain's merge word" +AGENT_STATE=dead +export FAKE_PR_OUT='CHECKS: none reported yet' +age_status park 500 +handle_paused_stale test:fm-park park h1 +assert_equals 1 "$(calls)" "a due pause recheck of a worker whose agent has exited asks Jev once" +assert_equals 0 "$(queued)" "a routine pause recheck of a worker whose agent has exited queues nothing" +assert_equals 0 "$(woken)" "a routine pause recheck of a worker whose agent has exited does not wake" +assert_equals 'exited' "$(jq -r '.wake.evidence[0].worker_agent' "$U/jev-state.json")" "Jev is told the worker's agent has exited" +reset +mk_task park "pr=$PR_URL" 'paused: PR is open and green, awaiting the merge word' +jev_triage_pause_routine park "stale: test:fm-park" || fail "a running worker with an open pull request was not offered" +assert_equals 'running' "$(jq -r '.wake.evidence[0].worker_agent' "$U/jev-state.json")" "Jev is told the worker's agent is running" +pass "eligible: an exited agent beside an open pull request in a repository with no checks is offered, and Jev is told which" # -- the deterministic pull-request read settles what it can ----------------- reset @@ -347,7 +366,9 @@ mk_task park "pr=$PR_URL" 'paused: PR is open and green, awaiting the merge word jev_triage_pause_routine park "stale: test:fm-park" || fail "an open pull request with nothing blocking was not offered" assert_equals "$PR_URL" "$(cat "$FAKE_PR_CALLS")" "the recorded pull request is read before Jev is asked" assert_equals 'open; no blocker reported' "$(jq -r '.wake.evidence[0].pull_request' "$U/jev-state.json")" "the evidence says no more than the pull-request read established" -for pr_out in 'MERGEABILITY: unknown' 'CHECKS: none reported yet'; do +for pr_out in 'MERGEABILITY: unknown' 'CHECKS: none reported yet' \ + 'CHECKS: no required check has reported; readiness unconfirmed' \ + $'MERGEABILITY: unknown\nCHECKS: none reported yet'; do reset mk_task park "pr=$PR_URL" 'paused: PR is open and green, awaiting the merge word' export FAKE_PR_OUT=$pr_out @@ -355,7 +376,8 @@ for pr_out in 'MERGEABILITY: unknown' 'CHECKS: none reported yet'; do assert_equals 'open; no blocker reported' "$(jq -r '.wake.evidence[0].pull_request' "$U/jev-state.json")" "a pull request reading '$pr_out' is not described as mergeable or passing" done for pr_out in 'STATE: merged at 2026-10-05T10:00:00Z' 'STATE: closed' 'REQUIRED CHECK: test (FAILURE)' \ - 'MERGEABILITY: conflicting' 'REVIEW DECISION: CHANGES_REQUESTED'; do + 'REQUIRED CHECK: test (PENDING)' 'MERGEABILITY: conflicting' 'REVIEW DECISION: CHANGES_REQUESTED' \ + 'DRAFT: pull request is not ready for review' $'MERGEABILITY: conflicting\nCHECKS: none reported yet'; do reset mk_task park "pr=$PR_URL" 'paused: PR is open and green, awaiting the merge word' export FAKE_PR_OUT=$pr_out From 4c03711adcb1b59312098ee0b6b6afe704ee82b3 Mon Sep 17 00:00:00 2001 From: Tejas Shah Date: Tue, 6 Oct 2026 02:11:53 +0700 Subject: [PATCH 2/3] docs: record the exited-agent and merge-word triage behaviour and its live answers --- bin/fm-watch.sh | 1 + docs/verification/jev-wake-triage.md | 69 +++++++++++++++------------- 2 files changed, 39 insertions(+), 31 deletions(-) diff --git a/bin/fm-watch.sh b/bin/fm-watch.sh index 039bd233bd2..3a730d370e5 100755 --- a/bin/fm-watch.sh +++ b/bin/fm-watch.sh @@ -1689,6 +1689,7 @@ JEV_TRIAGE_PR_TIMEOUT=30 FM_PR_STATE_BIN="${FM_PR_STATE_BIN:-$SCRIPT_DIR/fm-pr-state.sh}" # shellcheck disable=SC2016 # The backticks are literal markup in the question text. JEV_TRIAGE_INSTRUCTIONS='A monitoring reminder in `wake` is about to interrupt the supervisor of a fleet of software workers. The supervisor has already seen every status event listed in `wake`, so nothing in it is news. Decide from the words of the evidence what is being waited on. Choose `routine` only when the wait will clear without any action from the supervisor or another person, apart from the already reported merge of an open pull request. A worker that says only that its open pull request is waiting to be merged, or waiting for the merge word or merge approval of the supervisor, the captain, or anyone else, is waiting on that already reported merge, so that is `routine`. When the worker is waiting on a person or on the supervisor to choose, answer, review, approve, or unblock anything other than that merge, or it stopped after a failure, the reminder is how the supervisor remembers that someone still owes the worker a reply, so choose `needs_firstmate`. When `worker_agent` is `exited`, the worker cannot resume by itself: that changes nothing for a pull request that only awaits its merge, which needs nothing more from the worker and stays `routine`, but every other wait is then `needs_firstmate`. Choose `needs_firstmate` whenever you are unsure.' +# shellcheck disable=SC2016 # As above. JEV_TRIAGE_CRITERIA='{"needs_firstmate":"The newest status event shows the worker waiting on a person or on the supervisor for anything other than the merge of its open pull request: to choose between options, answer a question, review or accept finished work, supply or repair a credential or access, or unblock it; or the worker stopped after a failure or an error; or `worker_agent` is `exited` and the wait is for anything other than the merge of its open pull request; or a diagnostic reports anything other than a read that timed out; or it is not clear what the wait is for.","routine":"The newest status event declares an open pull request with nothing failing that only awaits its already reported merge, whoever gives the merge word and whether `worker_agent` is `running` or `exited`; or, while `worker_agent` is `running`, a wait on an automatic external event that clears without the supervisor: a running CI job, a rate limit or quota reset, a scheduled window, or an upstream release; or one transient read timeout that the monitor retries on its own."}' JEV_TRIAGE_TASK_EVIDENCE= diff --git a/docs/verification/jev-wake-triage.md b/docs/verification/jev-wake-triage.md index 46846975272..f774fc54b16 100644 --- a/docs/verification/jev-wake-triage.md +++ b/docs/verification/jev-wake-triage.md @@ -8,49 +8,53 @@ The API shape itself is recorded in [`dispatch-resolve.md`](dispatch-resolve.md) ## Live answers to the shipped question -Run 2026-10-05 against `https://api.typesafe.ai`, model `jev-latest` answering as `jev-1.13.0`, timeout 5 s, floor 0.6. -Each row is one real call made through the watcher's own `jev_triage_routine`, sourced from `bin/fm-watch.sh` at commit 410edad4 with its question text and criteria unmodified, with the key read from a home `.env` by `fm_jev_key_load` and a synthetic wake as evidence. -Every `declared-pause-recheck` row carried the pull request note `open; no blocker reported`, as the class always does: a paused task is eligible only when a pull request is recorded for the task and is still open with no blocker reported. - -Rows whose label ends in `-short` used the plain reason `stale: fm:fm-sample`. -The other `declared-pause-recheck` rows used the reason `stale: fm:fm-sample (paused 14520s, awaiting external - declared pause, rechecked on a long cadence not a wedge; confirm the wait still holds)`. +Run 2026-10-06 against `https://api.typesafe.ai`, model `jev-latest` answering as `jev-1.13.0`, timeout 5 s, floor 0.6. +Each row is one real call made through the watcher's own `jev_triage_pause_routine` or `jev_triage_contributions_routine`, sourced from `bin/fm-watch.sh` at commit 1f534a99 with its question text and criteria unmodified, with the key read from a home `.env` by `fm_jev_key_load` and a synthetic wake as evidence. +Every `declared-pause-recheck` row carried the pull request note `open; no blocker reported`, as the class always does, from a pull-request read that printed `CHECKS: none reported yet`, and the reason `stale: fm:fm-sample (paused 14520s, awaiting external - declared pause, rechecked on a long cadence not a wedge; confirm the wait still holds)`. +The agent column is the `worker_agent` value the row carried: `exited` for an endpoint whose agent state read `dead`, `running` for `alive`. The newest evidence of each row: -- `pr-open-merge-word` and `pr-open-merge-word-short`: `paused: PR is open and green, waiting for the merge`. -- `ci-running` and `ci-running-short`: `paused: CI is running on the pull request, about 25 minutes, resuming on my own when it finishes`. -- `told-which` and `told-which-short`: `paused: two ways to fix the schema, waiting to be told which`. -- `three-failures` and `three-failures-short`: `paused: the test suite failed three times, stopping here`. +- `merge-word`: `paused: pull request open, waiting for the captain's merge word`. +- `merge`: `paused: PR is open and green, waiting for the merge`. +- `ci`: `paused: CI is running on the pull request, about 25 minutes, resuming on my own when it finishes`. +- `release`: `paused: waiting on the upstream release, then I will rebase and finish the follow-up`. +- `told-which`: `paused: two ways to fix the schema, waiting to be told which`. +- `three-failures`: `paused: the test suite failed three times, stopping here`. - `credential-rejected`: `paused: cannot push the follow-up commit, the credential was rejected`. - `pr-open-but-asks`: `paused: PR is open, waiting for you to decide whether the migration should also ship in this PR`. - `review-question`: `paused: a reviewer asked on the pull request whether the old flag should keep working, waiting for an answer before I change anything`. +- `review-wait`: `paused: PR is open, waiting for the captain to review the approach before I continue`. - `gh-timeout`: one diagnostic, `gh pr view timed out after 30s`. -- `gh-401`: one diagnostic, `gh pr view failed: HTTP 401: Bad credentials`. The set was run three times; the table shows run 1. -| Class | Row | Expected | Verdict | Choice | Confidence | `routine` probability | -| --- | --- | --- | --- | --- | --- | --- | -| declared-pause-recheck | pr-open-merge-word | absorb | absorb | routine | 0.81 | 0.91 | -| declared-pause-recheck | pr-open-merge-word-short | absorb | absorb | routine | 0.94 | 0.97 | -| declared-pause-recheck | ci-running | absorb | absorb | routine | 0.91 | 0.96 | -| declared-pause-recheck | ci-running-short | absorb | absorb | routine | 0.96 | 0.98 | -| declared-pause-recheck | told-which | deliver | deliver | needs_firstmate | 1.0 | 0.0 | -| declared-pause-recheck | told-which-short | deliver | deliver | needs_firstmate | 1.0 | 0.0 | -| declared-pause-recheck | three-failures | deliver | deliver | needs_firstmate | 0.99 | 0.0 | -| declared-pause-recheck | three-failures-short | deliver | deliver | needs_firstmate | 1.0 | 0.0 | -| declared-pause-recheck | credential-rejected | deliver | deliver | needs_firstmate | 1.0 | 0.0 | -| declared-pause-recheck | pr-open-but-asks | deliver | deliver | needs_firstmate | 0.99 | 0.0 | -| declared-pause-recheck | review-question | deliver | deliver | needs_firstmate | 1.0 | 0.0 | -| contributions-observation-timeout | gh-timeout | absorb | absorb | routine | 0.91 | 0.95 | -| contributions-observation-timeout | gh-401 | deliver | deliver | needs_firstmate | 0.86 | 0.07 | +| Class | Row | Agent | Expected | Verdict | Choice | Confidence | `routine` probability | +| --- | --- | --- | --- | --- | --- | --- | --- | +| declared-pause-recheck | merge-word | exited | absorb | absorb | routine | 0.97 | 0.99 | +| declared-pause-recheck | merge-word | running | absorb | absorb | routine | 0.98 | 0.99 | +| declared-pause-recheck | merge | exited | absorb | absorb | routine | 0.99 | 0.99 | +| declared-pause-recheck | merge | running | absorb | absorb | routine | 0.97 | 0.99 | +| declared-pause-recheck | ci | exited | deliver | deliver | routine | 0.12 | 0.56 | +| declared-pause-recheck | ci | running | absorb | absorb | routine | 0.92 | 0.96 | +| declared-pause-recheck | release | exited | deliver | deliver | needs_firstmate | 0.58 | 0.21 | +| declared-pause-recheck | release | running | absorb | absorb | routine | 0.89 | 0.95 | +| declared-pause-recheck | told-which | exited | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | told-which | running | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | three-failures | exited | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | three-failures | running | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | credential-rejected | running | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | pr-open-but-asks | exited | deliver | deliver | needs_firstmate | 0.98 | 0.01 | +| declared-pause-recheck | pr-open-but-asks | running | deliver | deliver | needs_firstmate | 0.97 | 0.02 | +| declared-pause-recheck | review-question | running | deliver | deliver | needs_firstmate | 1.0 | 0.0 | +| declared-pause-recheck | review-wait | running | deliver | deliver | needs_firstmate | 0.98 | 0.01 | +| contributions-observation-timeout | gh-timeout | - | absorb | absorb | routine | 0.91 | 0.95 | Every row answered as `jev-1.13.0`. No verdict changed in any of the three runs, and every row matched its expectation. -The lowest absorbed confidence was 0.81, on `pr-open-merge-word`. -The lowest confidence on a delivered `needs_firstmate` answer was 0.81, on `gh-401` in run 2. -Latency was 297 to 427 ms. -The `gh-401` row is also refused in code before any call, because the contributions class admits only diagnostics that say a read timed out. +The lowest absorbed confidence was 0.88, on `gh-timeout` in run 3. +The narrowest delivery was `ci` with an exited agent, which the model kept choosing as `routine` at a confidence of 0.10 to 0.12 and a `routine` probability of 0.56, so only the floor delivered it. +A contributions diagnostic that is not a read timeout is refused in code before any call, so it has no row. ## The rejected wording @@ -71,10 +75,13 @@ In that row set `credential-rejected` read `paused: cannot push, the credential In that run `told-which-short` (routine, confidence 0.3) and `three-failures-short` (routine, confidence 0.54) were delivered only because the confidence fell below the floor. +The wording shipped before the current one read a wait on a named person's merge word as a wait on a person. +Measured 2026-10-06 over three runs, it answered the `merge-word` row `needs_firstmate` at a confidence of 0.62 to 0.69 with a running agent, so the commonest wait the class exists for was always delivered. + ## Offline behavior `tests/fm-jev-wake-triage.test.sh` proves the rest without the network. -With `fm_jev_choice` stubbed at the library boundary it proves each eligible class asks once and absorbs on a routine answer, each never-eligible case, a paused task with no recorded pull request at each of the three pause sites, and each pull request that is no longer open, has a blocker, or cannot be read is delivered without asking, every fallback delivers the wake as before, and the consecutive-absorb bound delivers unasked. +With `fm_jev_choice` stubbed at the library boundary it proves each eligible class asks once and absorbs on a routine answer, each never-eligible case, a worker whose agent has exited beside an open pull request in a repository with no checks is offered with that fact in the evidence, a paused task with no recorded pull request at each of the three pause sites, and each pull request that is no longer open, has a blocker, or cannot be read is delivered without asking, every fallback delivers the wake as before, and the consecutive-absorb bound delivers unasked. With the real library, a fake `curl`, and a real watcher process it proves a new `paused:` status signal is delivered without any call, a later stale recheck of a task with an open recorded pull request is absorbed, logged, and never queued, a bare turn-end from a paused worker is delivered without any call, a paused task with no recorded pull request is still delivered without any call while real heartbeats are being absorbed, a `needs_firstmate` answer and a timed-out call deliver the wake, the feature is off without the key, and the key reaches `curl` only on the file-descriptor header and no child environment, argv, or log. ```console From 1efd16a62b0617977269854615d660f6f538ece1 Mon Sep 17 00:00:00 2001 From: Tejas Shah Date: Tue, 6 Oct 2026 03:12:50 +0700 Subject: [PATCH 3/3] no-mistakes(review): docs: cite reachable commit 3ac9b3d in jev triage verification --- docs/verification/jev-wake-triage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/verification/jev-wake-triage.md b/docs/verification/jev-wake-triage.md index f774fc54b16..b947352a068 100644 --- a/docs/verification/jev-wake-triage.md +++ b/docs/verification/jev-wake-triage.md @@ -9,7 +9,7 @@ The API shape itself is recorded in [`dispatch-resolve.md`](dispatch-resolve.md) ## Live answers to the shipped question Run 2026-10-06 against `https://api.typesafe.ai`, model `jev-latest` answering as `jev-1.13.0`, timeout 5 s, floor 0.6. -Each row is one real call made through the watcher's own `jev_triage_pause_routine` or `jev_triage_contributions_routine`, sourced from `bin/fm-watch.sh` at commit 1f534a99 with its question text and criteria unmodified, with the key read from a home `.env` by `fm_jev_key_load` and a synthetic wake as evidence. +Each row is one real call made through the watcher's own `jev_triage_pause_routine` or `jev_triage_contributions_routine`, sourced from `bin/fm-watch.sh` at commit 3ac9b3d with its question text and criteria unmodified, with the key read from a home `.env` by `fm_jev_key_load` and a synthetic wake as evidence. Every `declared-pause-recheck` row carried the pull request note `open; no blocker reported`, as the class always does, from a pull-request read that printed `CHECKS: none reported yet`, and the reason `stale: fm:fm-sample (paused 14520s, awaiting external - declared pause, rechecked on a long cadence not a wedge; confirm the wait still holds)`. The agent column is the `worker_agent` value the row carried: `exited` for an endpoint whose agent state read `dead`, `running` for `alive`.