Skip to content

fix(review-loop): accept a verdict that closes a summary paragraph - #30

Merged
rlorenzo merged 1 commit into
mainfrom
fix/review-clean-sentence-final-verdict
Sep 1, 2026
Merged

fix(review-loop): accept a verdict that closes a summary paragraph#30
rlorenzo merged 1 commit into
mainfrom
fix/review-clean-sentence-final-verdict

Conversation

@rlorenzo

@rlorenzo rlorenzo commented Sep 1, 2026

Copy link
Copy Markdown
Owner

The bug

prompts/ and lib/ disagreed about where the verdict goes.

The prompt asked the reviewer to "state in the Summary: **Verdict: good to go**". The parser required a line of its own:

^[[:space:]]*([-+][[:space:]]*)?verdict[[:space:]]*:[[:space:]]*good to go[[:space:]]*[.!]?[[:space:]]*$

A reviewer that follows the prompt and ends its summary paragraph with the verdict produces a report that reads as unclean.

Observed on a real run. An antigravity review closed with:

… Finding counts: High 0, Medium 0, Low 0. Verdict: good to go.

test_review_clean never matched, so the loop could not break early. It ran all five cycles against an already-clean diff, spent 7m54s re-reviewing work no agent had touched, printed MAX ITERATIONS REACHED - some issues remain, and wrote a needs-review receipt for bin/review-gate to act on. Every one of the six passes had reported High 0 / Medium 0 / Low 0.

The fix

lib/lib-review-loop — the verdict may now open a line or close a sentence at the end of one. Both guards the anchors existed for survive by construction:

  • Quoting prose — "the report says Verdict: good to go" — still fails, because says is not a sentence boundary.
  • Hedged prose — "good to go, but …" — still fails, because the trailing anchor allows only an optional . or !.

The leading marker class also repeats now, so a ## Verdict heading counts; it did not before.

Promptscode-review and code-review-followup now say where the verdict goes instead of leaving it to the reviewer. The loop drives five agent CLIs and none can be relied on to place it, so this is defence in depth — the parser is the fix. The code-review wording is generated, so the edit is in .claude/commands/code-review.md; tools/generate carried it to prompts/ and the four vendor SKILL.md copies.

Test plan

  • ./test/run182/182 pass (was 181 + 1 stale-generation failure caught mid-work by generate --check).
  • Four new regression tests in test/lib-review-loop.bats: the paragraph-closing verdict that failed, a ## heading verdict, and two pinning the guards against hedged and quoted verdicts.
  • Replayed the real agent-code-review.md from the failing run through both regexes — old: no match (confirms root cause); new: clean.

The prompts and the parser disagreed about where the verdict goes. The prompt
asked the reviewer to "state in the Summary: **Verdict: good to go**"; the
parser required it to be a line of its own. A reviewer that followed the prompt
and ended its summary paragraph with the verdict produced a report that read as
unclean.

That is not hypothetical. An antigravity review closed with "... Finding
counts: High 0, Medium 0, Low 0. **Verdict: good to go**." on the end of the
summary paragraph. test_review_clean never matched, so the loop could not break
early: it ran all five cycles against an already-clean diff, spent eight minutes
re-reviewing work no agent had changed, printed MAX ITERATIONS REACHED, and
wrote a needs-review receipt for bin/review-gate to act on. Every cycle had
reported High 0, Medium 0, Low 0.

The verdict may now open a line or close a sentence at the end of one. Both
guards the anchors were there for survive by construction: quoting prose ("the
report says Verdict: good to go") still fails because "says " is not a sentence
boundary, and hedged prose ("good to go, but ...") still fails because the
trailing anchor allows only an optional "." or "!" after the phrase. The
leading marker class repeats so an "## Verdict" heading counts, which it did
not before.

Both prompts now say where the verdict goes rather than leaving it to the
reviewer, since the loop drives five agent CLIs and none of them can be relied
on to place it. That is defence in depth; the parser is the fix. The
code-review wording is generated, so the edit is in .claude/commands and
tools/generate carried it to prompts/ and the four vendor SKILL.md copies.
@rlorenzo
rlorenzo merged commit 20150a2 into main Sep 1, 2026
5 checks passed
@rlorenzo
rlorenzo deleted the fix/review-clean-sentence-final-verdict branch September 1, 2026 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant