Repository navigation
feat(bin): add opt-in Jev PR risk level, failed-check sort, and worker commit check - #50
Merged
Merged
Conversation
…to routine-wake triage The pause recheck is reached for an ordinary crew only once its agent is confidently dead, and the triage refused every agent that was not alive, so the recheck it was wired to could never be offered. A worker whose agent has exited beside an open pull request is now offered, the model is told which, and the question reads a wait for the merge word of that pull request as the already reported merge whoever gives it.
…onment, opt-in The sort runs only when its caller passes --sort-failed-checks, sends a failure log only for a repository listed as owner/repo, and never changes a blocker line or an exit status.
…v, opt-in The hook reaches a worker only on a project listed in config/jev-code-projects, acts only on a commit to the task's own worktree, sends the message and the staged file names and no file content, and stops a commit only for an added line in a recognised credential format.
The house-rules check kept its own opt-in list in config/house-rules.json while every other check that sends a project's code asks fm_jev_code_allowed. It now asks the same function, so config/jev-code-projects is the one place a project is named, config/house-rules.json holds only optional rules, and one docs/configuration.md section owns the list's format and its owner/repo rule.
This was referenced Oct 5, 2026
fix(bin): let routine-wake triage close pause rechecks of merge-word waits after the agent exits
#48
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes
This one pull request carries the final content of four that could not land one by one, and replaces them:
Intent
A set of Jev (TypeSafe decision-only model) integrations was ordered built ("implement" on each row of a ten-row proposal table) and then ordered merged: "merge all jev items". Five have landed on main. Four more are finished - each passed its own review, tests, documentation and lint steps - but cannot get onto main one by one:
Each touches the same shared files (bin/fm-jev-lib.sh, docs/configuration.md, AGENTS.md), so each one that lands puts the next into conflict, and GitHub's hosted runners are cancelling jobs unstarted tonight, so every separate run costs several re-runs. The standing delivery rule for a multi-part effort is one integration branch that merges once: "You should be putting everything in a different work tree. Check everything in a different work tree. When everything is ready to be merged, do so only then."
Rulings given during those four reviews, which the combined result must still satisfy: nothing a worker typed in its shell is sent to TypeSafe; no project's source or diff is sent unless that project is explicitly named; for a caller keyed by repository an entry matches only as owner/repo exactly, and a bare name matches only a caller passing a firstmate project name; a line that looks like a credential is never sent; the commit check touches a worker only on a named project and acts only on the task's own worktree; the failed-check sort never runs unless its caller asks for it; the risk level is rated once at registration and never during a merge; Jev advises and never becomes the only thing that blocks, merges or approves anything.
What Changed
TYPESAFE_API_KEY:bin/fm-pr-check.shprints onerisk: low|medium|high|not ratedline after arming the merge poll (bin/fm-pr-risk-lib.sh).bin/fm-pr-merge.shre-registers with the new--no-riskflag, so the level is never rated during a merge.bin/fm-pr-state.sh --sort-failed-checksadds aFAILED CHECK SORT:line per failed required check, labelling it code bug, flaky, environment, or unknown (bin/fm-check-sort-lib.sh). The sort runs only under that flag, and the watcher's triage read does not pass it.bin/fm-commit-check.shis installed bybin/fm-spawn.shas acommit-msghook, throughGIT_CONFIG_PARAMETERSin the worker's pane only, for ship and scout workers on a listed project. It acts only on commits to the task's own worktree. It refuses a commit whose added lines match a recognised credential format, and otherwise asks Jev about the message and staged file names alone.bin/fm-watch.sh: a paused task whose agent has exited is now offered to triage, withworker_agentin the evidence, instead of always being delivered. The reworded question treats a wait for the merge of the open pull request as routine. A repository that reports no checks is no longer counted as a blocker.bin/fm-jev-lib.sh:config/jev-code-projectsis now the single opt-in for every code-sending check. It is read fromFM_CONFIG_OVERRIDEwhen set, and matched as a whole line:<owner>/<repo>for repository-keyed callers, a firstmate project name for the rest.bin/fm-house-rules-check.shnow gates on that list instead of theprojectsarray inconfig/house-rules.json, which now holds only optional rules.fm_jev_secret_pathkeeps secret-named files out of the content callers send.docs/configuration.md, thedocs/verification/notes, andAGENTS.mdare updated to match.Risk Assessment
✅ Low: Both earlier fixes hold when traced through the code: every string in the state of all eight Jev callers passes the single withhold filter in _fm_jev_ask (including a key tail cut from its BEGIN line), the project gate and the house rules read the same config directory, and every new check stays opt-in and advisory, with only a rare unreachable-in-practice mid-key slice left unfiltered.
Testing
Ran the nine targeted suites for the changed Jev callers (all pass), then drove the real bin/ scripts by hand with real git, real jq and real installed commit hooks in a throwaway home. The only replacements were
curl(a recorder that saves each request body and answers in the API's shape) andgh(a fixture forge). The drive covered real commits through the installed hooks, real pull request registrations, real fm-pr-state.sh reads with and without --sort-failed-checks, adversarial private-key inputs to the one Jev caller, and a house-rules run over a key split across two 80-line blocks; all 54 checks passed. The watcher's routine-wake fix, the finished check's "no shell command is sent" rule and the merge path were exercised only through their existing suites, not by a separate manual drive, so the watcher and finished-check scenarios are reported as untested rather than as live passes. Nothing was sent to the real TypeSafe endpoint, so real model answers are untested. No UI surface is involved, so there is no visual evidence; the transcript is the evidence.tests/fm-pr-merge.test.shtest_merge_registration_never_rates_riskbash tests/fm-jev-wake-triage.test.sh(which passed) on a fake terminal backend with a fake curl…bash tests/fm-finished-check.test.sh(which passed), not by a separate manual drive of the finis…Evidence: Manual drive transcript (54 checks, with the exact state sent to Jev for each caller)
Source: Manual drive transcript (54 checks, with the exact state sent to Jev for each caller)
Evidence: Drive script that produced the transcript
Source: Drive script that produced the transcript
Evidence: What reached Jev when credentials were planted
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
bin/fm-pr-risk-lib.sh:181- Intent ruling: "a line that looks like a credential is never sent". Three paths in the combined change send text to typesafe.ai with no credential-line filter. (1) bin/fm-pr-risk-lib.sh:181 sends the first 30000 bytes of the pull request diff asdiff_startfor a listed project; nothing drops secret-shaped files (.env, *.pem) or lines matching a credential format, so a PR that addsAWS_KEY=AKIA...in an ordinary file has that line sent at registration. (2) bin/fm-check-sort-lib.sh:123 sends up to 4000 characters of the failed job's log aslog_tailwith no filter. (3) bin/fm-commit-check.sh:215 sends the commit message; the credential stop scans only added lines of staged files, so a token pasted into the message is sent. The commit check's FM_CC_CREDENTIALS patterns exist but are applied only to staged added lines. Policy decision needed: does the ruling cover the risk-level diff excerpt, the failed-check log tail and the commit message, or only staged content in the commit check? If it covers them, the smallest remedy is to drop lines matching the existing credential patterns before each of those three payloads is built.bin/fm-house-rules-check.sh:78- Commit ed0692e ("gate the house-rules check on the shared Jev project list") is not one of the four items the intent names and is not in the commit lists of PRs 45 or 49 (47 and 48 were not checked). It changes an already-landed feature's opt-in in both directions: a home that opted a project in throughprojectsin config/house-rules.json silently loses the check (that key is now ignored), and any project listed in config/jev-code-projects for another check (for example the review finding sort) now has its changed source lines sent by the house-rules check with no separate opt-in. The project is still explicitly named, so the "explicitly named" ruling holds, but no intent requirement needs this component; recommend removing the commit from the batch unless the author confirms it. Side effect of the same change: the gate reads🔧 Fix applied.
2 issues (1 warning, 1 info) still open:
bin/fm-jev-lib.sh:165- The fix round'swithholdfilter recognises a private key block only from its BEGIN line, and tracks the open block per string. A caller that cuts text before handing it over can deliver the tail of a block without its BEGIN line, and then the key body lines and the-----END ... PRIVATE KEY-----line are sent unchanged (line 166-168: no BEGIN, and no pattern matches a base64 body line or the END marker). Concrete path: bin/fm-house-rules-check.sh cuts each hunk into 80-line blocks (house_rules_blocks,count >= max), one request per block. A listed project adds a file not named like a secret (e.g. tests/fixtures.py) holding an embedded PEM key at added lines 60-110: block 1 has BEGIN and is withheld to its end, block 2 starts with ~30 key body lines plus the END line and is sent whole. The same happens for a -U3 hunk whose leading context lines are the end of an existing key, and for bin/fm-check-sort-lib.sh'stail -c 4000when it cuts off the BEGIN line. This contradicts the library's own contract ("a private key block replaced whole") and docs/configuration.md. Smallest fix, inside the one filter: when an END line is met with no block open, replace everything already kept from that string, and the END line, with the single placeholder. Add one case to the library-boundary test in tests/fm-jev-wake-triage.test.sh (a string starting with body lines and an END line).bin/fm-watch.sh:1753- Side effect of filtering at the one caller, noted as a tradeoff: when a worker's newestpaused:status line matches a credential pattern or the generic secret literal (e.g.paused: deploy needs API_KEY="abcd1234efgh" rotated, who owns it?), routine-wake triage now asks Jev over[line withheld: looks like a credential]in place of that line, and aroutineanswer at or above the floor still absorbs the reminder; the fix round's own test (tests/fm-jev-wake-triage.test.sh, "a status line holding a credential is withheld") asserts that absorb. The harm is bounded: the status line itself was already delivered unasked, only the later recheck reminders are absorbed, JEV_TRIAGE_MAX_STREAK caps it at 6 in a row, and the question tells the model to chooseneeds_firstmatewhen the wait is unclear. Making code refuse to absorb when a line was withheld would need a new output from the library, which is beyond what the fix was asked to do.🔧 Fix applied.
1 info still open:
bin/fm-jev-lib.sh:169- Residual limit of the round-2 fix, noted as a tradeoff: the filter recognises a private key block by its BEGIN or END line, so a string holding only body lines (neither marker) is sent as is. That needs a caller's cut to land on both sides of one key: a key body over 80 lines covering a whole middle block in bin/fm-house-rules-check.sh (an ordinary RSA 4096 key is about 51 lines, so it spans at most two blocks and both are now withheld), or a key printed unterminated at the very end of a failed job's log with its BEGIN line beyondtail -c 4000in bin/fm-check-sort-lib.sh. I could not construct a sequence that intended usage actually performs, and base64 body lines cannot be told from ordinary text without a new heuristic, so no change is recommended.✅ **Test** - passed
✅ No issues found.
tests/fm-pr-merge.test.shtest_merge_registration_never_rates_riskbash tests/fm-jev-wake-triage.test.sh(which passed) on a fake terminal backend with a fake curl…bash tests/fm-finished-check.test.sh(which passed), not by a separate manual drive of the finis…bash tests/fm-commit-check.test.shbash tests/fm-pr-risk.test.shbash tests/fm-pr-state.test.shbash tests/fm-jev-wake-triage.test.shbash tests/fm-house-rules-check.test.shbash tests/fm-finding-sort.test.shbash tests/fm-pr-merge.test.shbash tests/fm-dispatch-resolve.test.shbash tests/fm-finished-check.test.shbash ~/.no-mistakes/evidence/01M473YB6E12KYY47NVN9RMKJ7/jev-batch-drive.shrun from the worktree root: section Abin/fm-commit-check.sh --installplus realgit commitruns under the exported GIT_CONFIG_PARAMETERSsame script, section B:bin/fm-pr-check.sh task-a <url>with the project listed, unlisted, Jev answering yes to everything, Jev timing out, and--no-risksame script, section C:bin/fm-pr-state.sh [--sort-failed-checks] <url>with list entrieso/r,r,x/r,O/R,o/r-web,# o/r, with Jev unreachable, and with no keysame script, section D:fm_jev_choicefrombin/fm-jev-lib.shwith a key tail missing its BEGIN line, a nested whole key block, a block with no END line, and one line per credential kindsame script, section E:bin/fm-house-rules-check.sh hrover a file with a private key spanning two 80-line blocks✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.