feat(review-gate): gate agent commits on a clean code review - #27
Conversation
30c8b7b to
b0d6b5b
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The new gate’s commit detection can be bypassed by valid command forms (e.g., env -i … or git.exe …), which undermines enforcement in block mode.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR introduces a new “review gate” mechanism that can be installed as a PreToolUse hook to prevent (or warn on) agent-driven git commit actions unless the staged index has a matching clean review receipt, and wires the receipt/lock plumbing into the existing review loop tooling.
Changes:
- Add
bin/review-gatehook implementation plus a comprehensive BATS test suite covering parsing, receipts, nonce behavior, and platform-specific behaviors. - Extend
lib/lib-review-loopandbin/code-review-loopto record review receipts and maintain a lock so the gate can safely exempt commits during review-loop runs. - Update setup/docs/prompts/config to document, install, and operationalize the gate (including Windows CI coverage for BATS).
File summaries
| File | Description |
|---|---|
| test/review-gate.bats | New end-to-end BATS coverage for review-gate behavior (nonce, receipts, parsing, exemptions, formats). |
| test/lib-review-loop.bats | Adjust is_inside_dir tests to handle Windows symlink limitations and add conditional symlink test. |
| setup | Adds optional Claude Code hook wiring for review-gate and installs the script with the review-loop suite. |
| README.md | Documents review-gate behavior, modes, installation, escape hatches, and Windows caveats. |
| prompts/code-review-response.md | Adds explicit “do not stage/commit/push” guidance to reviewer response prompt. |
| prompts/code-refinement.md | Adds explicit “do not stage/commit/push” guidance to refinement prompt. |
| lib/lib-review-loop | Adds REVIEW_GATE config loading, recursion guard env export, receipt writing, and lock helpers. |
| bin/review-gate | New hook script implementing parsing + tiered allow/deny/warn behavior, receipts/nonces, and context output. |
| bin/code-review-loop | Acquires/releases gate lock and records a review receipt at loop completion. |
| .kimi-code/skills/code-refinement/SKILL.md | Mirrors “do not stage/commit/push” guidance for Kimi skill prompt. |
| .copilot/skills/code-refinement/SKILL.md | Mirrors “do not stage/commit/push” guidance for Copilot skill prompt. |
| .codex/skills/code-refinement/SKILL.md | Mirrors “do not stage/commit/push” guidance for Codex skill prompt. |
| .claude/commands/code-refinement.md | Mirrors “do not stage/commit/push” guidance for Claude command prompt. |
| .antigravity/skills/code-refinement/SKILL.md | Mirrors “do not stage/commit/push” guidance for Antigravity skill prompt. |
| .gitignore | Ignores root-anchored PLAN-*.md working artifacts. |
| .github/workflows/lint.yml | Adds a Windows GitHub Actions job to run the BATS suite under Git Bash. |
| .gitattributes | Marks generated skill/prompt outputs as linguist-generated for classification by review-gate. |
Review details
- Files reviewed: 14/17 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Copilot's review on #27 flagged `env -i` and `git.exe`. Both were real, and testing the shape turned up three more: `env -u NAME`, `command`, and `sudo`. Every one is an ordinary commit an agent could type without trying to evade anything, and in block mode the gate waved them all through. Two causes. The command word was matched as `git|*/git`, so anything carrying a .exe suffix or a backslash path missed, which is exactly the form a Windows invocation takes. And the prefix loop broke on the first option it saw, so any wrapper with a flag ahead of the command word fell out of the scan. Now the command word is reduced to a bare lowercased name before matching, and the prefix loop skips wrappers that take a command of their own along with their options. The wrapper list stays short on purpose: every entry runs the rest of the segment as a command, so skipping one lands on the real command word rather than on an argument. Also stops folding backslashes away when the command came from a PowerShell tool, where a backslash is a path character and the escape is a backtick. Without that, `& "C:\Program Files\Git\bin\git.exe" commit` tokenized into a word matching nothing. Nine tests, one per form, plus `command -v git`, which is a lookup rather than a commit and must still pass clean.
There was a problem hiding this comment.
🔵 Needs a closer look
The Claude hook wiring in setup has correctness issues (dangling symlink accepted; hook command not robust to spaces in $HOME) that can break hook execution.
Review details
Suppressed comments (2)
Previously missed (1) — in code that hasn't changed since the last review.
setup:1130
configure_review_gate_hooktreats any symlink at~/.local/bin/review-gateas sufficient, even if it is broken.[[ -L ... ]]is true for a dangling symlink, which would cause setup to wire a hook pointing at a non-existent executable and likely break Claude hook execution.
Prefer requiring an executable file (or an executable symlink target) via -x only.
This issue also appears on line 1180 of the same file.
local gate="$HOME/.local/bin/review-gate"
local settings_file="$HOME/.claude/settings.json"
[[ -x "$gate" || -L "$gate" ]] || return 0
setup:1186
- The hook
commandstring is written as$HOME/.local/bin/review-gate --format=claudewithout quoting the path. If$HOMEcontains spaces (common on Windows profiles and possible elsewhere), the hook runner will split the command and fail to execute the gate.
Quote the executable path when building the command string so it remains a single argv element under shell parsing.
local updated
updated=$(jq --arg cmd "$gate --format=claude" '
.hooks = (.hooks // {})
| .hooks.PreToolUse = ((.hooks.PreToolUse // []) + [{
matcher: "Bash|PowerShell",
hooks: [{type: "command", command: $cmd}]
}])' "$settings_file") || {
- Files reviewed: 14/17 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
bin/review-gate currently allows a practical bypass via env -S/--split-string where git commit is embedded in the split-string argument but not detected.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
test/review-gate.bats:226
- There’s no test covering the
env -S/--split-stringbypass case (where the command is embedded in the split-string argument). Given this is a realistic way to wrapgit commit, the suite should pin thatenv -S "git commit ..."is still detected and denied.
@test "env -u NAME git commit is detected" {
setup_gate; mkrepo; stage_code
run gate 'env -u FOO git commit -m "add app"'
assert_denied
}
- Files reviewed: 14/17 changed files
- Comments generated: 0 new
- Review effort level: Lite
…ring Three findings from Copilot's second and third passes on #27. `env -S "git commit -m x"` packs a whole command into one argument, and the wrapper scan skipped that argument as if it were an option's value, stepping straight over the commit. It is now tokenized and analyzed as its own segment, so the staging flags inside it are read like any other command's: `env -S "git commit -am x"` reports committing beyond the index, as the bare form does. Confirmed as a live bypass first; `env -S` is real here, GNU coreutils 8.32. setup accepted a dangling symlink as an installed gate: `-L` is true for a symlink whatever its target, so a broken link wired a hook pointing at nothing. `-x` alone is both necessary and sufficient, since it follows the link. Not reproducible on Windows, where ln -s cannot make a symlink at all, so this one rests on the test semantics rather than a run. And the hook command embedded an unquoted path. A Windows profile directory such as "C:\Users\First Last" would split into two arguments and execute nothing, silently, which is the failure mode this gate is least able to tolerate. Quoted now, in the README example too.
There was a problem hiding this comment.
🔵 Needs a closer look
The env -S/--split-string parsing currently ignores arguments appended after the split-string value, which can cause the gate to miss commits or staging flags in valid env -S forms.
Review details
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
test/review-gate.bats:2
- The header comment calls this a “pre-commit” gate, but it’s a harness PreToolUse hook (not a git pre-commit hook). This is likely to confuse readers and future maintainers when debugging why it runs.
# Tests for bin/review-gate: the pre-commit code review gate.
- Files reviewed: 14/17 changed files
- Comments generated: 0 new
- Review effort level: Lite
There was a problem hiding this comment.
🔵 Needs a closer look
bin/review-gate’s token iteration/copying uses unquoted array expansions that can split quoted tokens containing spaces (e.g., commit messages), breaking the core quote-aware parsing guarantees.
Review details
- Files reviewed: 14/17 changed files
- Comments generated: 0 new
- Review effort level: Lite
The reviewer-side prompts have always carried an explicit "Do not stage, commit, or push" line. The two editor-side prompts did not: code-review-response said nothing about git at all, and code-refinement mentioned commits only in passing, as a place to discover the lint command. Both run with full write access in every harness (claude --allowedTools includes Bash, codex --sandbox workspace-write, copilot --yolo, agy --dangerously-skip-permissions, kimi -p), so nothing stopped an editor agent from deciding to commit its own fixes. The review loop stages what it needs on its own; the commit is the developer's call.
Adds bin/review-gate, a PreToolUse hook that stops a coding agent from
committing code nobody reviewed, plus the receipt that makes "has this been
reviewed" answerable in the first place.
The hook is sub-second git plumbing and string parsing; it never runs
code-review-loop itself. Three tiers, cheapest first:
1. Deterministic allow: nothing staged, a clean receipt for this exact index
and base, a history rewrite in progress, an index tree identical to
ORIG_HEAD or HEAD@{1}, a conflicted index, or a bypass.
2. Deterministic deny carrying the staged diff, per-category line counts,
receipt status, a single-use nonce, and the classification rubric.
3. The agent classifies the change and either re-runs the commit with the
nonce, saying why it judged it trivial, or asks the user.
No harness hook can render a multi-choice menu, so the decision tree renders
one layer up: the hook denies with a structured reason and the agent takes it
from there.
Details worth calling out:
- The nonce is read from the command string, not the environment. A PreToolUse
hook is spawned before the shell runs the command, so an inline
AI_REVIEW_GATE=<nonce> assignment never reaches the hook's own environment.
Getting this wrong ships an infinite deny loop, so it is the first test.
- Command parsing is quote-aware. A naive split on ; && || corrupts on the most
ordinary input there is: git commit -m "fix: sanitize input; also docs &&
tests" is one command, not three, and the -a inside a message is not a
staging flag.
- -a, --all, -o, --only, -i, --include, and pathspec forms are checked before
every index-derived rule. Otherwise git commit -a with a clean index sails
through the empty-index check and lands unreviewed work.
- Unparseable commands that mention git and commit fail closed. A spurious
prompt is recoverable; a silent miss is not.
- Allow is silence plus exit 0, never an affirmative "allow", which would skip
the user's own permission rules and auto-approve every shell command.
- Headless detection is explicit, never a TTY check. Every harness spawns hooks
with pipes on all three descriptors, so keying off [ -t 0 ] would degrade
every interactive run and quietly turn the gate off while still looking
installed.
- The recursion guard is two independent mechanisms: run_agent exports
AI_REVIEW_GATE=off, and code-review-loop holds a lock file that the gate
validates by PID liveness and age rather than merely observing, since an EXIT
trap does not run on SIGKILL and a stale lock would disable the gate forever.
- git-dir resolution uses --absolute-git-dir. --git-dir returns a relative path
normally but an absolute one in a linked worktree, and on Git for Windows
that is C:/..., which no leading-slash test recognizes.
Ships in warn mode. The rubric is untested against real commits and a wrong
block is far more annoying than a wrong warn; switch to REVIEW_GATE=block once
a few weeks of warn output shows it is not crying wolf.
setup wires it for Claude Code only. The script speaks every harness's output
shape via --format, but Copilot fails closed on a parse error, so writing an
unverified schema into a user's config would deny every shell call in the
session. The other four are documented for manual wiring instead.
Detecting the commit is the whole job, so the command word is normalised before
matching and wrappers are stepped through. Matching a bare `git` alone enforces
nothing: `git.exe`, an absolute path, `env -i git`, `env -u NAME git`, `command
git`, `sudo git`, and `env -S "git commit"` are all ordinary commits, and each
was allowed at some point during development. env's --split-string needs the
most care, since it carries a command inside one argument and appends whatever
operands follow it, so it is tokenized and analyzed as its own segment with the
tail attached. Losing that tail is not merely a missed commit: `env -S "git
commit" -a` would otherwise be denied with the staging flag unread, and a
receipt describing the index would then vouch for a working-tree commit.
Windows takes three accommodations, each learned by running the gate there.
Those harnesses expose a PowerShell tool alongside the Bash one, so the hook
matcher is Bash|PowerShell; a matcher of Bash alone lets every commit made
through the other tool past without even a warning, and setup widens an
existing Bash-only entry in place rather than reporting it already configured.
PowerShell has no VAR=value command prefix, so the gate reads the
$env:AI_REVIEW_GATE = "..." statement form too and renders the nonce and the
escape hatch in whichever syntax matches the tool the commit came from; without
that the deny message told the agent to run a hard parse error, which in block
mode left no working escape hatch at all. And a backslash is a path character
there rather than an escape, so folding it away turned `& "C:\Program
Files\Git\bin\git.exe" commit` into a word matching nothing.
The hook command is written with the path quoted, since a profile directory
containing a space would otherwise split into two arguments and execute
nothing, and setup requires -x rather than accepting any symlink, so a dangling
link cannot wire a hook that points at nothing.
164 bats tests, green on Linux and Windows, one per command form that can reach
a commit.
None of this ran on Windows before, which is how the PowerShell gaps in review-gate survived a full review. GitHub-hosted standard runners are free on public repos, so the job costs wall-clock time only. Tests alone here; shellcheck and markdownlint are platform-independent and stay on the Linux job. One test had to move first. is_inside_dir's symlink case built a link with `ln -s`, which Git Bash turns into a copy unless Developer Mode is on, leaving nothing to resolve. It is now its own test that skips when the platform cannot make a real symlink, detected by trying rather than by testing the platform name, so a Windows box that can still runs it.
The loops' other working artifacts were already ignored; the plan document the user hands plan-review-loop was not, so a scoping doc rode along into a commit on this branch by accident. Anchored to the repository root. Windows and macOS check out with core.ignorecase set, where an unanchored PLAN-*.md also matches prompts/plan-review.md and its neighbours, which are prompts this repo ships.
8df9f9c to
22a232f
Compare
What
Adds
bin/review-gate, aPreToolUsehook that stops a coding agent from committing code nobody reviewed, plus the review receipt that makes "has this been reviewed" answerable in the first place.code-review-loopends onWaiting for manual review.by design, and until now nothing connected that to the commit. An agent could run the loop, ignore the verdict, and commit anyway, or skip the loop entirely.How it works
The hook is sub-second git plumbing and string parsing. It never runs
code-review-loopitself: that takes minutes, spawns nested agent CLIs, and is designed to hand back to a human. Three tiers, cheapest first, and most commits never reach the third.Tier 1, deterministic allow. Nothing staged or an
--amendthat only rewords; a clean receipt for exactly this index on exactly this base; a history rewrite in progress (rebase, interactive rebase, cherry-pick, revert, merge, including every--continuestep); an index tree identical toORIG_HEADorHEAD@{1}, which catches a rewrite whose markers are already cleaned up; a conflicted index; or a bypass. A twelve-commit rebase must not stop to ask twelve times.Tier 2, deterministic deny with context. The reason carries the staged stat, the changed paths with generated ones flagged, added and deleted lines split into code / tests / docs / generated, why any receipt went stale, a single-use nonce, and the rubric.
Tier 3, agent judgment. No harness hook can render a multi-choice menu, so the decision tree renders one layer up: the agent classifies the change and either re-runs the commit with the nonce, saying out loud why it judged it trivial, or calls
AskUserQuestionwith the options. A line-count threshold cannot tell a one-line permission check from a two-hundred-line rename, so line counts are evidence handed to the agent, never the decision.Details worth reviewing closely
The nonce is read from the command string, not the environment. A
PreToolUsehook is spawned before the shell runs the command, so an inlineAI_REVIEW_GATE=<nonce> git commit ...never reaches the hook's own environment. Getting this wrong ships an infinite deny loop, so it is the first test in the suite.Command parsing is quote-aware. A naive split on
;,&&,||corrupts on the most ordinary input there is:git commit -m "fix: sanitize input; also docs && tests"is one command, not three, and the-ainside a message is not a staging flag.-a,--all,-o,--only,-i,--include, and pathspec forms are checked before every index-derived rule. Otherwisegit commit -awith a clean index sails through the empty-index check and lands unreviewed work. There is a test for exactly that ordering.Unparseable commands fail closed. A spurious prompt is recoverable; a silent miss is not.
Allow is silence plus exit 0, never an affirmative
"allow", which would skip your own permission rules and auto-approve every shell command the agent runs.Headless detection is explicit, never a TTY check. Every harness spawns hooks with pipes on all three descriptors, so keying off
[ -t 0 ]would degrade every interactive run and quietly turn the gate off while still looking installed. There is a test pinning the inverse: no TTY on any descriptor, no headless marker, still blocks.The recursion guard is two independent mechanisms.
run_agentexportsAI_REVIEW_GATE=off, andcode-review-loopholds a lock file the gate validates by PID liveness and age rather than merely observing, since anEXITtrap does not run onSIGKILLand a stale lock would disable the gate forever in exactly the situation where nobody thinks to look.The command word is matched after normalisation, and wrappers are stepped through. A gate that only recognises a bare
gitenforces nothing:git.exe,env -i git,command git,sudo git, andenv -S "git commit"are all ordinary commits. Each has a test.Also in here
The reviewer-side prompts have always said "Do not stage, commit, or push." The two editor-side prompts did not, and they run with full write access in every harness. First commit fixes that. It is worth doing regardless of the gate, and becomes load-bearing once the gate exists.
A
chore:commit also gitignores root-anchoredPLAN-*.md, the working plan documentsplan-review-loopoperates on. The anchor matters: macOS and Windows check out withcore.ignorecaseset, where an unanchored pattern would also swallowprompts/plan-review.mdand its neighbours, which this repo ships.Defaults and scope
Ships in warn mode: prints the reason, lets the commit through. The rubric is untested against real commits and a wrong
blockis far more annoying than a wrongwarn. Switch toREVIEW_GATE=blockin~/.ai-coding-setup.confonce a few weeks of warn output shows it is not crying wolf.setupwires it for Claude Code only. The script speaks every harness's output shape via--format, but Copilot fails closed on a parse error, so writing an unverified schema into a user's config would deny every shell call in the session. The other four are documented for manual wiring, with their formats, once you confirm the paths against current docs.Two deliberate omissions, both for the same reason: I would not ship a config schema or an environment marker I could not verify. Headless detection wanted per-CLI non-interactive markers; I could only verify
CI, so the list isAI_REVIEW_HEADLESS=1plusCI, with the gap noted in a comment.Known blind spot
A commit made inside a script the agent invokes is invisible, because the gate only ever sees the command the agent typed. Accepted for v1. Nothing short of a git-level hook closes it, and a git-level hook cannot ask a question, so it would only ever warn after the fact.
Testing
164 bats tests, all passing on Linux and Windows;
shellcheck -xclean;markdownlintclean;tools/generate --checkin sync. The suite covers a realgit rebase -ifixture, a linked worktree, an unborn HEAD, a corrupt receipts file, nonce reuse and cross-tree rejection, stale locks by dead PID and by age, each--formatoutput shape, and one case per command form that can reach a commit.Bugs caught by dogfooding the gate on this change:
sedhas no\|alternation in a basic regex, which silently made every generated file classify asdocs.Bugs caught by review, all reproduced against the gate before being fixed:
env -iandgit.exeslipped past commit detection. Testing the shape turned up three more:env -u NAME,command, andsudo.env -S "git commit"was skipped as if it were an option's value, and once that was fixed, the operands after the split string were dropped.env -S "git" commitwas a live bypass, andenv -S "git commit" -awas worse: denied, but with the staging flag lost, so a receipt describing the index would have vouched for a working-tree commit.setupaccepted a dangling symlink as an installed gate, and wrote the hook command with an unquoted path, which a profile directory containing a space would split into two arguments.One review finding was a false positive and is documented rather than "fixed": the token arrays are expanded with
${arr[@]+"${arr[@]}"}, whose alternate value is quoted, so spaced commit messages are not split. There is now a test pinning that.Windows
Verified on Windows, and covered by a
windows-latestCI job running the full suite under Git Bash. The gate began as macOS-only in practice; this is what it took to make it real there.Claude Code runs hook commands through Git Bash on Windows by default, falling back to PowerShell only when Git Bash is absent, so the Git Bash style path
setupwrites resolves as written.jqis the one dependency not bundled with Git for Windows, and without it the gate allows every commit rather than failing, so confirm it is onPATH.PowerShelltool alongsideBash, and a matcher ofBashalone lets every commit made through the other one past without even a warning.setupwritesBash|PowerShell, and widens an existingBash-only entry in place when re-run.AI_REVIEW_GATE=off git commit ...prefix is a parse error in PowerShell, so the gate reads the$env:AI_REVIEW_GATE = "..."statement form too and renders the nonce and escape hatch in whichever syntax matches the tool the commit came from. Either way it has to ride on the same command as the commit: the hook is a separate process spawned before the command runs, so it never inherits a variable set in an earlier call.& "C:\Program Files\Git\bin\git.exe" commitbecame a word matching nothing.Cost worth knowing: the hook fires on every shell call, and Git Bash process startup is slow. Roughly 160ms warm against 55ms for a bare
bash -c true, and over a second on a cold file cache. Milliseconds on macOS and Linux.setupfalls back to copying whenln -scannot make a real symlink, which is the default on Windows without Developer Mode. That works, but the installed script is then a snapshot rather than a link, so re-runsetupto pick up changes.