fix: close the verify-gate bypass, reach the last two gates from MCP, correct the docs - #3
Merged
Merged
Conversation
… correct the docs An audit of every doc claim against the code turned up three things that mattered. 1. The verify gate could be bypassed by deleting the thing it keys on. `verify-gate` matched `- [x] … verify: \`CMD\``, so an item that was ticked *after* its verify clause was removed matched nothing and passed silently. `ralph`'s eligibility check only looked at unchecked items, and only once, before the loop. So the documented guarantee — "a checkbox survives only if its verify command actually passes when the harness runs it" — did not hold on that path. Fixed with `--require-verify`: a checked item with no verify clause is a failure and is reverted; `ralph` passes the flag and now refuses a TODO where any item, checked or not, lacks the clause. Default stays lenient so a mixed TODO can still be scanned standalone. 2. The two strongest gates had no MCP surface. `arc-prereg` (link a seal so `arc-close` injects the kill-condition verbatim) and `verify-gate` were shell-only, so an agent driving Yeoul through MCP — the setup the README prescribes — could run neither, while the bootstrap prompt told it the verify gate existed as a tool. Added both; 10 tools → 12, CI assertion updated. 3. Docs. `docs/METHODOLOGY.md` pointed at `bin/yeoul-graduate`, which does not exist (`bin/graduate` does). The verify-gate claims in README, README_KO, METHODOLOGY and the bootstrap prompt now state what is enforced and what is not: the harness owns the re-run and the presence of the clause, but a verify command that cannot fail is still the author's problem. Backend A's obligation to run the gate itself is now explicit rather than a parenthetical. Tests: 5 new assertions for the bypass and its controls. Verified they fail against the previous scripts (3 of 5; the other 2 are the compat and no-false-positive controls, which pass on both). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The added continuation line was dedented out of the `run: |` block, which made the workflow file itself invalid — the run failed before any job started. Validated the YAML parses and the embedded python compiles before pushing this time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
An audit of every doc claim against the code. Three things mattered; the rest was hygiene.
1. The verify gate could be bypassed by deleting the thing it keys on
verify-gatematched- [x] … verify: \CMD``. An item ticked after its verify clause was removedmatched nothing, so the gate skipped it and the box survived unverified:
ralph's eligibility check only looked at unchecked items, and only once, before the loop —so nothing re-checked it mid-run. The documented guarantee ("a checkbox survives only if its verify
command actually passes when the harness runs it") did not hold on that path.
Fix:
--require-verifytreats a checked item with no verify clause as a failure and reverts it.ralphpasses the flag and refuses a TODO where any item — checked or not — lacks the clause, whichis what makes the flag safe without exemption bookkeeping. The default stays lenient so a mixed
TODO (loop items + manual items) can still be scanned standalone.
2. The two strongest gates had no MCP surface
arc-prereg(link a seal soarc-closeinjects the kill-condition verbatim) andverify-gatewereshell-only. An agent driving Yeoul through MCP — the setup the README prescribes — could run neither,
while
docs/BOOTSTRAP_PROMPT.mdtold it the verify gate existed "as a tool, not just instructions".Added
arc_preregandverify_gate. Tool surface 10 → 12, CI assertion updated.3. Docs
docs/METHODOLOGY.mdreferencedbin/yeoul-graduate; the command isbin/graduate.enforced and what is not: the harness owns the re-run and the presence of the clause, but a verify
command that cannot fail is still the author's problem.
Tests
5 new assertions covering the bypass, the compat path, and no-false-positives. Verified they fail
against the previous scripts — 3 of 5 fail (numbered above); the other 2 are the compat and
no-false-positive controls, which pass on both by design.
pre-publish-check.shandexamples/demo.shstill pass.
🤖 Generated with Claude Code