Repository navigation
feat(bin): add five opt-in advisory Jev integrations - #51
Merged
Merged
Conversation
…reaches the captain bin/fm-escalation-screen.sh sorts one question's text into a fixed kind through the shared Jev caller and prints one advisory line: yours, captain's, or by hand. Code decides a merge, approval, destructive or security question, a review gate, and an over-long question without a request. Off without TYPESAFE_API_KEY; every failure is by hand; it writes nothing.
…er hand-offs with Jev, opt-in With TYPESAFE_API_KEY and the project listed in config/jev-code-projects, a ship or scout Claude worker gets a PreToolUse hook on its helper-agent tool. Code decides first (a named model or a non-inheriting helper type is left alone); otherwise one fixed-choice question, mechanical or judgement, and a confident mechanical runs that helper on sonnet. Every other outcome keeps the model the worker asked for. The hook states no permission decision and each asked hand-off is recorded in state/.helper-model.log. Without the key or the listing the worker's settings file is unchanged.
…Jev, opt-in bin/fm-intake-route.sh asks one fixed-choice request through the shared caller which registered project a request or bug report is about and which second mate's scope covers it. The choices are read at call time from data/projects.md (bin/fm-project-mode.sh --list, which leaves out an entry marked finished) and data/secondmates.md. Off without TYPESAFE_API_KEY; advice only, printed for firstmate's own intake resolution, and by hand on any failure or low confidence.
…tate read bin/fm-worker-health.sh prints the bin/fm-crew-state.sh line unchanged and, only with TYPESAFE_API_KEY present and only when that read comes from the pane or status-log fallback for a local ship or scout, asks one fixed-choice question through bin/fm-jev-lib.sh: working, stuck, waiting, or finished. An answer at or above the shared confidence floor adds one advice-only health line; every other outcome leaves the crew-state line alone. It sends status lines, ages, and an unacknowledged-instruction count, never terminal content, steering text, the brief, or project files. Firstmate runs it at the start of stuck-crewmate-recovery's live-endpoint escalation; nothing runs it automatically.
…tions bin/fm-exists-search.sh asks one yes/no question of a project's functions and prints a ranked file:line list, and with --change compares each function a change adds with the existing ones closest to it. It is on only with TYPESAFE_API_KEY and the project named in config/jev-code-projects, calls Jev only through bin/fm-jev-lib.sh, and is advice only: every outcome exits 0. An opted-in project's brief gains the search under "Before you write any code"; every other brief is byte-identical. bin/fm-house-rules-check.sh's base pick moves into house_rules_base so the search reuses it.
AGENTS.md's config/jev-code-projects entry lists every check keyed by a firstmate project name; the helper model pick is one of them, as docs/configuration.md "Jev code projects" already says.
…pick The pick moves a helper to sonnet, which is a lowering only when the worker's own model is a stronger one. A worker launched with no model named runs on the account default, which firstmate cannot see and which may itself be sonnet or haiku, so that launch now gets no hook and its settings file is the keyless file. The pick can therefore never raise a helper's model.
… verification sentence
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
A set of Jev (TypeSafe decision-only model) integrations for firstmate was ordered built ("implement" on the rows of a ten-row proposal table, and "yeah incorporate above 5 too" on an earlier five-row table) and ordered merged: "merge all jev items". Eight have landed on main. Five more are now finished - each built and tested on its own local branch, not pushed, with no pull request:
They were deliberately not landed one by one: every Jev item touches the same shared files (bin/fm-jev-lib.sh, docs/configuration.md, AGENTS.md, bin/fm-test-run.sh, docs/documentation-audiences.json), so each landing put the next into conflict. The standing delivery rule for a multi-part effort is one integration branch that merges once: "You should be putting everything in a different work tree. Check everything in a different work tree. When everything is ready to be merged, do so only then."
Rulings every Jev item must satisfy: nothing a worker typed in its shell is sent to TypeSafe; no project's source or diff is sent unless that project is named in config/jev-code-projects (a repository matches only as owner/repo, a bare name only a firstmate project name); every line that looks like a credential is withheld before anything is sent, inside the one Jev caller; Jev advises and never becomes the only thing that blocks, merges, approves or discards anything; a small answer never adds or removes a stop for the captain's approval; a feature adds nothing beyond what its row describes.
What each of the five items is, so this reads without the branches: (1) the escalation screen is one opt-in command that sorts a question firstmate is about to put to the captain into a fixed kind and prints one advisory line (yours, captain's, or by hand), with merges, approvals, destructive and security questions decided as captain's by code without any request; (2) the helper model pick lets a Claude worker's helper hand-off that names no model run on a cheaper model when the work is judged mechanical, only for a project named in config/jev-code-projects, only ever lowering the model and never stating a permission decision; (3) intake routing is one opt-in command that prints an advisory project and second mate for an incoming request from the two registries, never dispatching anything; (4) worker health is one opt-in command that prints a task's deterministic current-state line and may add one advisory health line (working, stuck, waiting, or finished) built from status records only, never terminal content; (5) the already-exists search is an opt-in command, only for a project named in config/jev-code-projects, that asks a yes-or-no question of a project's functions and prints the likely matches so a worker finds what exists before writing a second copy.
What Changed
bin/fm-escalation-screen.sh(prints one advisory line for a question before it goes to the captain),bin/fm-intake-route.sh(prints an advisory project and second mate for an incoming request),bin/fm-worker-health.sh(prints a task'sfm-crew-state.shline and may add onehealth:line), andbin/fm-exists-search.sh(asks a yes-or-no question of a project's functions and prints likelyfile:linematches, with a--changerun over the worker's own change).bin/fm-helper-model.shand wires it intobin/fm-spawn.sh: a ship or scout Claude worker gets aPreToolUsehook onAgent|Taskonly when the pick is enabled for that home, project and launch model; otherwise the hooks file is written as before.bin/fm-brief.shlikewise adds the already-exists search step to a brief only when that search is enabled for the project.bin/fm-project-mode.shgains--listand afinishedregistry token that leaves a project out of that list,bin/fm-house-rules-check.shexposes its base lookup ashouse_rules_basefor the search to share,bin/fm-test-run.shregisters and maps the five new test files, andAGENTS.md,docs/configuration.md,docs/scripts.md,docs/documentation-audiences.jsonand two skills document the new commands.The five items
bin/fm-escalation-screen.sh: screens a question before it is put to the owner.bin/fm-helper-model.sh: a cheaper model for mechanical side jobs inside a worker.bin/fm-intake-route.sh: routes an incoming request to its project and second mate.bin/fm-worker-health.sh: working, stuck, waiting or finished.bin/fm-exists-search.sh: "does this already exist?" before building.Decisions confirmed during review
finishedregistry token is kept on purpose. The instruction given for this feature was "for project maintain a seperate list so when u have new project or complete work on some project u can update that", so a project marked finished stops being offered by intake routing, and the project-management skill now says to set and clear the mark.--changemode is kept on purpose. The accepted row reads "Run it at the start of a task, and again on the new code in a change", so the second run is part of the row.state/.helper-model.log) was removed: nothing in its row needs a record.🤖 Generated with Claude Code
Risk Assessment
✅ Low: Both fix-round changes are verified against the code (the helper-model log, its arguments, docs and tests are fully removed with the spawn hook command and usage consistent; the widened captain word list matches every added test case and only ever errs toward the captain), and all five features remain opt-in, advisory, exit-0 paths that send only what their rows allow through the one Jev caller.
Testing
Ran the five feature test files (all passed), then drove the real bin/ scripts by hand in an isolated fixture home: a real git project, real registries, and a real task on a private tmux server read by the real fm-crew-state.sh. The TypeSafe endpoint was a stand-in
curlon PATH that recorded each request and returned a chosen answer, because no TYPESAFE_API_KEY is available here. All 75 checks in the transcript passed, but the checks that rest on a stand-in answer are reported as untested scenarios, not live passes. Live passes cover: off without a key, code-decided outcomes with zero requests, project opt-in (a bare name not matching an owner/repo entry), credential lines withheld, the key kept off argv and child environments, usage errors, and nothing written into the home. One real Claude Code 2.1 session (worker on opus) ran the real hook and its helper ran on sonnet. Real model answers from api.typesafe.ai and a real worker spawn are untested. No UI surface, so no screenshots; evidence is CLI transcripts. The fixture was removed and the worktree is clean.screen: captain's (... not asked)with requests-made=0--enabled exit codestable (0 only for key + listed project + opus)bash tests/fm-helper-model.test.sh, which runs the real bin/fm-spawn.sh on a fake tmux and passed. A real spawn launches a…fm-project-mode.sh --listprints storefront, ledger, catalogue-scrape;fm-project-mode.sh oldblogstill prints direct-PRfm_brief_exists_search_stepoutput for proj (lines naming the command and --change) and for otherproj (empty)Evidence: Transcript of every hand-driven run (commands, stdout, stderr, exit code, requests made, what was sent)
Source: Transcript of every hand-driven run (commands, stdout, stderr, exit code, requests made, what was sent)
Evidence: Real Claude Code session: worker on opus, helper lowered to sonnet by the hook
Source: Real Claude Code session: worker on opus, helper lowered to sonnet by the hook
main session assistant-message models: 2 claude-opus-5-5 helper (subagent) assistant-message models: 5 claude-sonnet-5-5 hand-off the hook sent: {"helper":{"description":"Find parse_rate callers","type":"general-purpose","prompt":"List every file under src/ that calls parse_rate and report file and line. Do not change anything."}}Evidence: Stand-in endpoint: the recording curl shim
Source: Stand-in endpoint: the recording curl shim
Evidence: Stand-in endpoint: the answer program
Source: Stand-in endpoint: the answer program
Evidence: Escalation screen: code-decided and withheld-credential excerpt
Evidence: Worker health: what was sent for a real fixture task
Evidence: Already-exists search: opt-in gate and matches
Evidence: Feature test file results
Source: Feature test file results
escalation-screen, helper-model, intake-route, worker-health, exists-search: each file exited 0 with nonot okline (logs test-*.log in the same directory)Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
🔧 **Review** - 5 issues found → auto-fixed ✅
bin/fm-escalation-screen.sh:56- ESCALATION_SCREEN_CAPTAIN_WORDS lists every inflection of merge but only the base forms of the other words, and omits common destructive/security terms. Checked with the samegrep -Eiqw: "Has the captain approved shipping this?", "Should the old rows be deleted?", "Should the stale branch be dropped?", "Should I remove the legacy table?" and "Rotate the API token now?" all miss the code rule, so the question is sent to Jev and a confidentcheap-to-reverseanswer printsscreen: yours. The intent requires "merges, approvals, destructive and security questions decided as captain's by code without any request". Fix: add the missing inflections (approves/approved/approving, deleted/deletion, dropped/drops, discarded, destroyed, wiped, overwritten/overwrites, forced) and the obvious missing terms (remove/removed/removal, token, key, revert, reset), and extend the test loop at tests/fm-escalation-screen.test.sh:91 with those cases.bin/fm-project-mode.sh:81- Simplification: the change introduces a newfinishedtoken in the data/projects.md bracket grammar (parsed in --list at line 81, tolerated by the main parser at line 115, documented in docs/configuration.md "Intake routing"). The intake-routing row only requires printing "an advisory project and second mate for an incoming request from the two registries"; no requirement needs a new registry marker for hiding completed projects, and the intent says "a feature adds nothing beyond what its row describes". Recommend removing thefinishedtoken and having --list print every entry.bin/fm-exists-search.sh:266- Simplification:fm-exists-search.sh <project> --changeis a second mode that compares each function the change already added against existing ones (up to 25 new x 16 candidates), plus the ship-brief line that tells the worker to run it (bin/fm-dod-lib.sh:278) and the house_rules_base extraction in bin/fm-house-rules-check.sh that exists only to serve it. The row describes a command "that asks a yes-or-no question of a project's functions and prints the likely matches so a worker finds what exists before writing a second copy" - the question mode alone satisfies that; --change runs after the copy is written and exceeds the row. Recommend removing --change, its brief line, and the house_rules_base extraction.bin/fm-helper-model.sh:83- Simplification: the helper model pick adds a new durable record, state/.helper-model.log (_fm_hm_record: one JSON line per asked hand-off with the description's first 80 characters, size-capped rotation, AGENTS.md state-layout entry). The row only requires letting a no-model helper hand-off "run on a cheaper model when the work is judged mechanical ... only ever lowering the model and never stating a permission decision"; no requirement needs a persisted log, and the other four items write no record. Recommend removing the log (or confirm it is wanted as the evidence trail the ponytail comment at line 56 refers to).bin/fm-exists-search.sh:93-git grepwith no pathspec searches and reports paths relative to the current directory, while the --changegit diffreports repo-root-relative paths, so a --change run from a subdirectory of the worktree silently finds 0 added functions (and a question run covers only that subtree). The brief and header say to run it in the worktree root, so intended usage is unaffected; noted only as a tradeoff.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Test** - passed
✅ No issues found.
screen: captain's (... not asked)with requests-made=0--enabled exit codestable (0 only for key + listed project + opus)bash tests/fm-helper-model.test.sh, which runs the real bin/fm-spawn.sh on a fake tmux and passed. A real spawn launches a…fm-project-mode.sh --listprints storefront, ledger, catalogue-scrape;fm-project-mode.sh oldblogstill prints direct-PRfm_brief_exists_search_stepoutput for proj (lines naming the command and --change) and for otherproj (empty)bash tests/fm-escalation-screen.test.shbash tests/fm-helper-model.test.shbash tests/fm-intake-route.test.shbash tests/fm-worker-health.test.shbash tests/fm-exists-search.test.shbin/fm-escalation-screen.sh "<question>"by hand: no key, 8 merge/approval/destructive/security questions, review-gate line, over-4000-character question, stand-in answers at high and low confidence, endpoint failure, unknown answer, no argumentbin/fm-helper-model.sh --enabled <home> <project> <model>over 6 gate combinations, and--hook <home> <project>with hook JSON on stdin: mechanical, judgement, low confidence, model already named, Explore helper type, unlisted project, bare name vs owner/repo, no key, endpoint failure, non-JSON stdinbin/fm-project-mode.sh --listandbin/fm-project-mode.sh oldblogon a registry with afinishedentrybin/fm-intake-route.shwith the request on stdin: no key, routed request, local-only override, low confidence, off-list answer, empty request, 20001-byte request, endpoint failurebin/fm-worker-health.sh <task>on a real fixture task (realbin/fm-crew-state.sh, private tmux server,bin/fm-busy-event.shidle/busy): no key, each of the four health kinds, low confidence, unknown answer, endpoint failure, worktree gone, secondmate, bad task idbin/fm-exists-search.sh <project> "<question>" [<path>],--change, and--enabledin a fixture git repo: no key, unlisted project, bare name vs owner/repo, matches, path limit, low confidence, endpoint failure, usage errorfm_brief_exists_search_stepfrombin/fm-dod-lib.shfor an opted-in and a not-opted-in projectclaude -p ... --model opusin a scratch repo whose.claude/settings.local.jsonPreToolUse hook runsbin/fm-helper-model.sh --hook, then read the session and helper transcripts for the models usedFile-hash snapshot of the fixture home before and after every run🔧 **Document** - 1 issue found → auto-fixed ✅
.agents/skills/project-management/SKILL.md:22- Thefinishedbracket token for data/projects.md is documented only in docs/configuration.md "Intake routing" and bin/fm-project-mode.sh's header. The project-management skill, which is the agent-runtime owner firstmate reads when maintaining the registry, never tells it to addfinishedwhen a project's work is complete, so the list may not get updated in practice. I left it unedited because adding it changes firstmate's operating contract rather than fixing a stale fact; a one-line pointer from that skill to docs/configuration.md "Intake routing" would close it if wanted.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.