Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,8 @@ jobs:
from yeoul_mcp import server
tools = [n for n in dir(server) if n in (
'yeoul_new','arc_open','arc_ticket','loop_guard_init','loop_guard_tick',
'arc_close','build_handoff','ralph_gate_check','status','arc_list')]
assert len(tools) == 10, tools
print("yeoul-mcp OK — 10 tools")
'arc_close','build_handoff','ralph_gate_check','arc_prereg','verify_gate',
'status','arc_list')]
assert len(tools) == 12, tools
print("yeoul-mcp OK — 12 tools")
PY
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,23 @@ Initial extraction of the Yeoul harness (de-personalized structure only).
cross-linked row (verdict · stop · sealed claim · one-line conclusion · source) to `KNOWLEDGE_INDEX.md`.
Hands-off, sourced only from the sealed _SUMMARY — it can't rot the way a hand-maintained wiki does.

### Audit fixes (2026-08-05 — found by auditing docs against code)
- **Verify-gate bypass closed.** The gate keyed on the `verify:` clause, which lives in a file the agent
edits: deleting the clause while ticking the box made the item invisible to the gate and it survived
unverified. `verify-gate --require-verify` now treats a checked item with no verify clause as a failure,
`ralph` passes the flag and refuses a TODO where *any* item (checked or not) lacks the clause. The docs
that claimed "a checkbox survives only if its verify command passes" were corrected to state what is
actually enforced — and what is not (a verify command that cannot fail is still on the author).
- **The two strongest gates reached MCP.** `arc_prereg` (link a seal so `arc_close` injects the
kill-condition verbatim) and `verify_gate` (re-run a round's verify commands) had no MCP tool, so an
agent driving Yeoul purely through MCP — the setup the README prescribes — could run neither. Added;
tool surface 10 → 12.
- **Docs**: `bin/yeoul-graduate` → `bin/graduate` (the referenced command did not exist); backend A's
obligation to run the verify gate itself made explicit in METHODOLOGY and the bootstrap prompt.
- **`index-append`**: numbered lists no longer index an empty conclusion, and a failed extraction warns
instead of passing silently (see below).
- **`mcp<2` pin**: an unbounded floor let CI install a breaking major (`mcp.server.fastmcp` moved).

### Review fixes
- **Portability**: replaced GNU-only `sed -i` with a portable temp-file edit (was silently failing the
In-Progress→Closed status update on macOS/BSD sed, which the README targets).
Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,9 @@ New to the method? Read [`docs/METHODOLOGY.md`](docs/METHODOLOGY.md), then paste
- **KILL-defense 5-check** — a "failed" verdict cannot be sealed until anchor-reproduction, ≥2 independent
angles, implementation-defect ruled out, catalog cross-check, and verbatim kill-wording are all recorded.
- **Verify-gated dev loop** — each automated development round advances one TODO item and may only check it
off after its own machine verification command exits 0. Items without a verify command are refused.
off after its own machine verification command exits 0, re-run by the harness rather than reported by the
agent. Items without a verify command are refused — including checked ones, so deleting the clause is not
a way out. (A verify command that cannot fail is still a hole: that part is on the author.)

## Layers

Expand Down
4 changes: 3 additions & 1 deletion README_KO.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,9 @@ tests/test_gates.sh # 모든 게이트의 단언 기반 스모크 테스트
- **KILL 방어 5항 체크** — "실패" 판정은 앵커 재현·독립각도 ≥2·구현결함 배제·도감 대조·kill 문언 일치가 전부
기록되기 전엔 봉인되지 않는다.
- **검증 게이트 개발 루프** — 각 자동 개발 라운드는 TODO 한 항목을 진행하고, 그 항목의 기계 검증 명령이 exit 0일
때만 체크할 수 있다. 검증 명령 없는 항목은 거부된다.
때만 체크할 수 있다. 판정은 에이전트 보고가 아니라 **하네스 재실행**이다. 검증 명령 없는 항목은 거부되며
**체크된 항목도 마찬가지** — 절을 지우고 체크하는 우회가 막힌다. (실패할 수 없는 검증 명령은 여전히 구멍이고,
그건 작성자 몫이다.)

## 계층

Expand Down
14 changes: 9 additions & 5 deletions bin/ralph
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,13 @@ for a in "${@:2}"; do case "$a" in
--agent-cmd=*) AGENT_CMD="${a#*=}" ;;
esac; done

# Gate: any unchecked item without a `verify:` command → refuse the whole loop (form = eligibility).
BAD="$(grep '^- \[ \]' "$TODO" | grep -v 'verify:' || true)"
# Gate: any item without a `verify:` command → refuse the whole loop (form = eligibility).
# Checked items are included: the loop hands the file to an agent every round, and a checked item
# with no verify clause is exactly what the clause-deletion bypass produces. Requiring the clause on
# every item is what lets verify-gate run with --require-verify below (no exemption bookkeeping).
BAD="$(grep -E '^- \[( |x)\]' "$TODO" | grep -v 'verify:' || true)"
if [ -n "$BAD" ]; then
echo "⛔ unchecked items without a 'verify:' command — not loop-eligible (manual/session work):"
echo "⛔ items without a 'verify:' command — not loop-eligible (manual/session work):"
echo "$BAD" | sed 's/^/ /'
exit 3
fi
Expand Down Expand Up @@ -68,8 +71,9 @@ $(cat "$TODO")

# ★ harness enforces the gate: independently re-run verify for whatever got checked off; revert failures.
# The agent's claim that "verify passed" is not trusted — a checkbox survives only if the command
# actually exits 0 when the harness runs it.
"$SCRIPT_DIR/verify-gate" "$TODO" --revert || echo " ⚠ harness reverted a checked item (its verify did not pass on re-run)"
# actually exits 0 when the harness runs it. --require-verify additionally reverts a checked item
# whose verify clause went missing, so deleting the clause is not a way out either.
"$SCRIPT_DIR/verify-gate" "$TODO" --revert --require-verify || echo " ⚠ harness reverted a checked item (verify failed on re-run, or its verify clause was missing)"

TOK=0
if command -v python3 >/dev/null 2>&1; then
Expand Down
33 changes: 26 additions & 7 deletions bin/verify-gate
Original file line number Diff line number Diff line change
@@ -1,21 +1,40 @@
#!/usr/bin/env bash
set -uo pipefail
# verify-gate <TODO.md> [--revert]
# verify-gate <TODO.md> [--revert] [--require-verify]
# Harness-enforced verify gate. For every checked item (`- [x]` … verify: `CMD`), the harness
# RE-RUNS CMD itself (cwd = current dir) — it does not trust the agent's claim that verify passed.
# With --revert, any item whose command exits != 0 is flipped back to `- [ ]`.
# This is what makes "resists self-deception" true rather than requested: a checked box only
# survives if its verify command actually passes when the harness runs it.
# Exit 0 = every checked item's verify passed · non-zero = at least one failed.
#
# --require-verify: a checked item with NO `verify:` clause counts as a failure too.
# Why it exists: without it the gate keys on text the agent itself writes, so deleting the
# `verify:` clause while ticking the box made the item invisible to the gate and it survived
# unverified (observed, not hypothetical). `ralph` always passes this flag — its eligibility gate
# has already established that every item in a loop-driven TODO carries a verify command.
# Default stays off so a mixed TODO (loop items + manual items) can still be scanned standalone.

TODO="${1:-}"; REVERT=0
for a in "${@:2}"; do case "$a" in --revert) REVERT=1 ;; esac; done
[ -n "$TODO" ] && [ -f "$TODO" ] || { echo "usage: verify-gate <TODO.md> [--revert]" >&2; exit 2; }
TODO="${1:-}"; REVERT=0; REQUIRE=0
for a in "${@:2}"; do case "$a" in
--revert) REVERT=1 ;;
--require-verify) REQUIRE=1 ;;
esac; done
[ -n "$TODO" ] && [ -f "$TODO" ] || { echo "usage: verify-gate <TODO.md> [--revert] [--require-verify]" >&2; exit 2; }

FAIL=0; TMP="$TODO.vg.$$"
: > "$TMP"
while IFS= read -r line || [ -n "$line" ]; do
if printf '%s' "$line" | grep -qE '^- \[x\].*verify:[[:space:]]*`[^`]+`'; then
if [ "$REQUIRE" -eq 1 ] && printf '%s' "$line" | grep -q '^- \[x\]' \
&& ! printf '%s' "$line" | grep -qE 'verify:[[:space:]]*`[^`]+`'; then
# checked but carries no verify command — the gate has nothing to re-run, so it cannot vouch
# for this box. Treat as failed rather than skipping it (skipping is how the clause-deletion
# bypass worked).
FAIL=1
echo " ✗ checked item has no verify command: ${line:0:60}…" >&2
if [ "$REVERT" -eq 1 ]; then
line="$(printf '%s' "$line" | sed 's/^- \[x\]/- [ ]/')"
echo " ↩ reverted to [ ] (nothing for the harness to verify)" >&2
fi
elif printf '%s' "$line" | grep -qE '^- \[x\].*verify:[[:space:]]*`[^`]+`'; then
# extract the FIRST verify block via python (sed's greedy `.*verify:` grabbed the LAST block,
# letting an appended decoy `verify: `true`` override a real failing command).
cmd="$(printf '%s' "$line" | python3 -c 'import sys,re; m=re.search(r"verify:\s*`([^`]+)`", sys.stdin.read()); sys.stdout.write(m.group(1) if m else "")')"
Expand Down
4 changes: 3 additions & 1 deletion docs/BOOTSTRAP_PROMPT.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ HARD RULES (do not violate — these are the point of the tool)
- Never seal before pre-registering the kill-condition. Never edit a kill-condition after seeing results
(amend, don't overwrite). Record negatives and retractions indelibly.
- Never automate: pre-registration sealing, PASS/KILL judgment, graduation, external publishing. Those are mine.
- In the dev loop, only check off a TODO item after its verify command exits 0; never edit verify commands.
- In the dev loop, only check off a TODO item after its verify command exits 0; never edit or delete verify
commands. Run `verify_gate` (or `bin/verify-gate <TODO> --revert --require-verify`) after every round —
in backend A that is your job, not the loop's.
- Separate the tool from your judgment: only say "a gate refused X" when a script/tool actually returned an
error you can quote; say "applying the discipline, I suspect X" for your own reasoning. Never borrow the
tool's credibility for a judgment call.
Expand Down
16 changes: 11 additions & 5 deletions docs/METHODOLOGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,13 +87,19 @@ Run the dev loop two ways (Ralph pattern):
**Loop-forbidden (belongs to the human/session):** measurement runs, sealing, PASS/KILL judgment. The loop's
safety comes from the machine-verification gate, not from the loop — it is only as good as your verify commands.

The gate is **harness-enforced, not requested**: after each round the harness runs `bin/verify-gate <TODO> --revert`,
which independently re-runs the `verify:` command of every checked item and flips any that don't exit 0 back to
`[ ]`. `ralph` does this automatically; in backend A, the session must run it too. A checkbox survives only if
its verify command actually passes when the *harness* runs it — the agent's word is not trusted.
The gate is **harness-enforced, not requested**: after each round the harness runs
`bin/verify-gate <TODO> --revert --require-verify`, which independently re-runs the `verify:` command of every
checked item and flips any that don't exit 0 back to `[ ]`. `ralph` does this automatically; **in backend A the
session must run it too** (MCP: the `verify_gate` tool) — otherwise nothing has been verified but the agent's word.

Scope, honestly: the gate keys on the `verify:` clause, which lives in a file the agent can edit. Deleting the
clause while ticking the box used to make the item invisible to the gate, so it survived unverified. That is what
`--require-verify` closes — a checked item with no verify clause is now reverted too, and `ralph` refuses a TODO
where any item lacks the clause. What remains outside the machine's reach is the *content* of a verify command:
a command that does not actually test the item will still exit 0. Write commands that can fail.

## Exits
- **graduate** (success): `bin/yeoul-graduate` → moves the project out of the incubator, leaves a pointer.
- **graduate** (success): `bin/graduate` → moves the project out of the incubator, leaves a pointer.
- **close-project** (retire/reject): `bin/close-project` → archives with a `_CLOSED.md`, freezes open arcs,
optionally seals the closure. History is preserved — no silent deletion.

Expand Down
8 changes: 6 additions & 2 deletions mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,12 @@ Set `YEOUL_BIN` if the `bin/` scripts are not adjacent to the package.

## Tools

`yeoul_new`, `arc_open`, `arc_ticket`, `loop_guard_init`, `loop_guard_tick`, `arc_close`, `build_handoff`,
`ralph_gate_check`, `status`, `arc_list`.
`yeoul_new`, `arc_open`, `arc_ticket`, `loop_guard_init`, `loop_guard_tick`, `arc_close`, `arc_prereg`,
`build_handoff`, `ralph_gate_check`, `verify_gate`, `status`, `arc_list`.

`arc_prereg` and `verify_gate` are the two enforcement halves that used to be reachable only from the shell:
linking a seal (so `arc_close` injects the kill-condition verbatim) and re-running a round's verify commands.
An agent driving Yeoul purely through MCP could not run either — so neither gate held on that path.

Each returns `{exit_code, stdout, stderr}`. A non-zero `exit_code` on `arc_close` (4 = blanks, 5 = KILL-defense)
or `ralph_gate_check` (3 = ungated item) is an enforced gate, not an error to route around.
Expand Down
18 changes: 18 additions & 0 deletions mcp/yeoul_mcp/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,24 @@ def ralph_gate_check(name: str, workspace: str = ".") -> dict:
return {"exit_code": 0, "stdout": "loop-eligible: all unchecked items have verify commands", "stderr": ""}


@mcp.tool()
def arc_prereg(arc_dir: str, claim_id: str, ledger: str = "", workspace: str = ".") -> dict:
"""Link a sealed pre-registration to an arc so arc_close injects its kill-condition VERBATIM instead of
trusting an agent-typed field. Seal the claim first (mirror-stack). Without this, closes are UNSEALED."""
args = [arc_dir, claim_id] + ([ledger] if ledger else [])
return _run("arc-prereg", *args, cwd=workspace)


@mcp.tool()
def verify_gate(todo_path: str, revert: bool = True, require_verify: bool = True,
workspace: str = ".") -> dict:
"""Re-run the `verify:` command of every checked TODO item and revert the boxes that do not pass. This is
the harness half of the dev loop — backend A (in-session) MUST call it each round, or nothing has been
verified but the agent's word. require_verify also reverts a checked item whose verify clause is missing."""
args = [todo_path] + (["--revert"] if revert else []) + (["--require-verify"] if require_verify else [])
return _run("verify-gate", *args, cwd=workspace)


@mcp.tool()
def status(workspace: str = ".", md: bool = False) -> dict:
"""One line per active project: name · latest arc verdict · dev TODO progress."""
Expand Down
16 changes: 16 additions & 0 deletions tests/test_gates.sh
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,22 @@ printf -- '- [x] decoy. verify: `false` verify: `true`\n' > "$T3"
"$BIN/verify-gate" "$T3" --revert >/dev/null 2>&1
grep -q '^- \[ \] decoy' "$T3" && echo " ✓ decoy double-verify reverted (first block wins)" || { echo " ✗ decoy passed (greedy bug)"; FAIL=1; }

# --- clause-deletion bypass: ticking a box after deleting `verify:` must not survive ---
# Regression: the gate keys on text the agent writes, so a checked item with the clause removed used
# to be invisible to it and passed silently. Observed, not hypothetical.
T4="$WS/t4.md"
printf -- '- [x] deleted its own verify clause\n' > "$T4"
"$BIN/verify-gate" "$T4" >/dev/null 2>&1; assert "clause-less checked item passes without --require-verify (compat)" 0 $?
"$BIN/verify-gate" "$T4" --revert --require-verify >/dev/null 2>&1; assert "clause-less checked item fails with --require-verify" 1 $?
grep -q '^- \[ \] deleted its own verify clause' "$T4" && echo " ✓ clause-less box reverted to [ ]" || { echo " ✗ clause-less box survived"; FAIL=1; }
# a genuine passing item must still survive under the same flag (no false positives)
printf -- '- [x] real. verify: `true`\n' > "$T4"
"$BIN/verify-gate" "$T4" --revert --require-verify >/dev/null 2>&1; assert "genuine checked item still passes under --require-verify" 0 $?
grep -q '^- \[x\] real' "$T4" && echo " ✓ genuine box left checked" || { echo " ✗ genuine box wrongly reverted"; FAIL=1; }
# ralph must refuse a TODO whose CHECKED item lacks a verify clause (was only checking unchecked ones)
printf -- '- [x] checked without verify\n- [ ] ok. verify: `true`\n' > "$WS/projects/p/dev/TODO.md"
"$BIN/ralph" p >/dev/null 2>&1; assert "ralph refuses a checked item without verify" 3 $?

# --- index-append: the conclusion must survive any list style, and a miss must be loud ---
# Regression: the extractor used to assume a `- ` bullet, so a numbered list silently indexed
# an empty conclusion while still logging success. Observed in the wild, not hypothetical.
Expand Down
Loading