Skip to content

fix: close the verify-gate bypass, reach the last two gates from MCP, correct the docs - #3

Merged
bhyi4 merged 2 commits into
mainfrom
fix/audit-gate-and-docs
Aug 4, 2026
Merged

fix: close the verify-gate bypass, reach the last two gates from MCP, correct the docs#3
bhyi4 merged 2 commits into
mainfrom
fix/audit-gate-and-docs

Conversation

@bhyi4

@bhyi4 bhyi4 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

An audit of every doc claim against the code. Three things mattered; the rest was hygiene.

1. The verify gate could be bypassed by deleting the thing it keys on

verify-gate matched - [x] … verify: \CMD``. An item ticked after its verify clause was removed
matched nothing, so the gate skipped it and the box survived unverified:

- [x] item with verify deleted   →  exit 0   (passed, not reverted)
- [x] item. verify: `false`      →  exit 1   (caught, reverted to [ ])

ralph's eligibility check only looked at unchecked items, and only once, before the loop —
so nothing re-checked it mid-run. The documented guarantee ("a checkbox survives only if its verify
command actually passes when the harness runs it") did not hold on that path.

Fix: --require-verify treats a checked item with no verify clause as a failure and reverts it.
ralph passes the flag and refuses a TODO where any item — checked or not — lacks the clause, which
is what makes the flag safe without exemption bookkeeping. The default stays lenient so a mixed
TODO (loop items + manual items) can still be scanned standalone.

2. The two strongest gates had no MCP surface

arc-prereg (link a seal so arc-close injects the kill-condition verbatim) and verify-gate were
shell-only. An agent driving Yeoul through MCP — the setup the README prescribes — could run neither,
while docs/BOOTSTRAP_PROMPT.md told it the verify gate existed "as a tool, not just instructions".

Added arc_prereg and verify_gate. Tool surface 10 → 12, CI assertion updated.

3. Docs

  • docs/METHODOLOGY.md referenced bin/yeoul-graduate; the command is bin/graduate.
  • The verify-gate claims in README, README_KO, METHODOLOGY and the bootstrap prompt now say what is
    enforced and what is not: the harness owns the re-run and the presence of the clause, but a verify
    command that cannot fail is still the author's problem.
  • Backend A's obligation to run the gate itself is explicit instead of a parenthetical.
  • CHANGELOG records this batch plus the two previously merged fixes.

Tests

5 new assertions covering the bypass, the compat path, and no-false-positives. Verified they fail
against the previous scripts
— 3 of 5 fail (numbered above); the other 2 are the compat and
no-false-positive controls, which pass on both by design. pre-publish-check.sh and examples/demo.sh
still pass.

Note on that check: its over-claim scan looks for marketing superlatives, so it reported clean
throughout — it cannot catch a claim that contradicts the code. That is why this audit was manual.

🤖 Generated with Claude Code

Mother Seara and others added 2 commits August 5, 2026 06:41
… correct the docs

An audit of every doc claim against the code turned up three things that mattered.

1. The verify gate could be bypassed by deleting the thing it keys on. `verify-gate` matched
   `- [x] … verify: \`CMD\``, so an item that was ticked *after* its verify clause was removed
   matched nothing and passed silently. `ralph`'s eligibility check only looked at unchecked
   items, and only once, before the loop. So the documented guarantee — "a checkbox survives
   only if its verify command actually passes when the harness runs it" — did not hold on that
   path. Fixed with `--require-verify`: a checked item with no verify clause is a failure and is
   reverted; `ralph` passes the flag and now refuses a TODO where any item, checked or not,
   lacks the clause. Default stays lenient so a mixed TODO can still be scanned standalone.

2. The two strongest gates had no MCP surface. `arc-prereg` (link a seal so `arc-close` injects
   the kill-condition verbatim) and `verify-gate` were shell-only, so an agent driving Yeoul
   through MCP — the setup the README prescribes — could run neither, while the bootstrap prompt
   told it the verify gate existed as a tool. Added both; 10 tools → 12, CI assertion updated.

3. Docs. `docs/METHODOLOGY.md` pointed at `bin/yeoul-graduate`, which does not exist (`bin/graduate`
   does). The verify-gate claims in README, README_KO, METHODOLOGY and the bootstrap prompt now
   state what is enforced and what is not: the harness owns the re-run and the presence of the
   clause, but a verify command that cannot fail is still the author's problem. Backend A's
   obligation to run the gate itself is now explicit rather than a parenthetical.

Tests: 5 new assertions for the bypass and its controls. Verified they fail against the previous
scripts (3 of 5; the other 2 are the compat and no-false-positive controls, which pass on both).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The added continuation line was dedented out of the `run: |` block, which made the
workflow file itself invalid — the run failed before any job started. Validated the
YAML parses and the embedded python compiles before pushing this time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@bhyi4
bhyi4 merged commit 4e3bc84 into main Aug 4, 2026
3 checks passed
@bhyi4
bhyi4 deleted the fix/audit-gate-and-docs branch August 4, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant