Repository navigation
docs(readme): describe the evidence rook actually uses - #61
Merged
Merged
Conversation
Four README claims no longer match the shipping CLI (0.1.13): - "watches the filesystem": rook takes no filesystem snapshot. Every scenario records "the filesystem was not observed" unless a collect hook reports what changed. - "tool calls checked against the agent's own tool surface": calls are graded against each scenario's expectations, and only when a hook returns them. - "does not grade the reply": replies are graded, from quotes of what the agent output. What changed is that a claimed action is not proof. - "a verdict can say Pass, and here is what nobody looked at": a scenario with any unchecked criterion rolls up to Unable to Verify, not Pass. The comparison table's tool-call and side-effect rows now say the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
Continuous Review — Round 1 — ✅ PASSHead: Per-reviewer summary
Auto-posted by continuous-review-agent v1.3 (findings now include a suggested fix / code snippet). Contact @vipulkv for feedback. |
gauravBansal06
approved these changes
Oct 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Four README claims don't match the shipping CLI (0.1.13). An independent evaluator's write-up, due out 28–30 Oct, links this README and documents the gap, so readers will compare the two.
runner.tsunobservedNote). A collect hook reports changes instead ("Nothing is hashed off the disk any more").callsexpectations, and only when a hook returns them (judge/calls.ts).rookdoes not grade the reply"judge/checks.tsrollUp).The comparison table's Tool calls and Side effects rows now match. No other doc in this repo repeats these claims.
Not changed here: the testmuai.com Agent Assurance page carries similar copy ("Watches files", "Unverifiable is excluded from the pass rate").
🤖 Generated with Claude Code