[vchamarthi]

Summary

Adds a review guide and a pre-PR self-review checklist, so that most findings are caught before a PR is opened and each review round closes more of them. Text only; no code, CI or gate changes.

 /
+├── REVIEW.md                         # what a defect is here, how to raise and close findings, rounds
 ├── AGENTS.md
+│   └── "Before opening a PR"         # self-review: mutation check, sibling sites, bad inputs, docs match code
 └── .github/
+    ├── copilot-instructions.md       # points Copilot review at REVIEW.md
     └── pull_request_template.md
+        ├── "Revert" section
+        └── "Self-reviewed" checkbox

Why: what the data shows

PR history of intel/dokimi (GitHub API, all 185 PRs, 13 to 23 Oct 2026):

Measure	Value
PRs	146 merged, 32 open, 7 closed
Time to merge	median 13.7 h, p90 64.9 h
Commits per merged PR	median 4, p90 22
Review threads	1,416 on 127 PRs
Threads on the 25 PRs with 15+ commits	603 (43%)
Threads unresolved at merge	346 of 1,141, on 76 of 146 merged PRs

Recurring themes in thread openers (keyword match, overlapping): tests that cannot fail 680, nits 387, docs or docstrings that disagree with code 246, timing or flakiness 161, error handling 121, changelog 101, exit codes 88, Windows paths 35. Several are the same bug class found one command at a time, for example the unreadable-input exit code across #189 to #194, a docstring claiming exit 1 where the code exits 2 (#190), None from os.cpu_count() (#112), and backslash paths in generated config (#116).

One week of agent review on this repo (self-measured audit of 301 review points on 65 PRs, 14 to 20 Oct 2026):

• 57 of 62 must-fix findings came from running code (a repro, a differential against main, a mutation check), not from reading it.
• A pushed fix with a test landed 99% of the time; a comment-only point landed 62%.
• 21 of 83 author fixes needed another round; 5 of 13 must-fix fixes made by authors regressed.
• 3 of the 7 must-fix issues missed were in the reviewer's own fix commits.
• Misses clustered on the largest PRs (repo inspect: static look at what a repository offers an agent (repo_environment.v1) #66, refactor(eval): give answer and eval records one reader (eval/records.py) #80, docs, tests: user-journey catalog (J1–J15) and criterion evidence trace #102, refactor(harnesses): docker-* solver, sandbox and probe on the process runner #104).

How the Claude Code team runs review (Solving code review at Anthropic):

• Target the least human time per change with no rise in reverts. On their repo, merged PRs more than doubled while the share needing a revert fell by more than 40%.
• Verify before the PR: tests, build and a fresh-context review of the diff. Giving the agent a way to check its work is estimated to make results two to three times better.
• A REVIEW.md per repo: say what failure looks like there, cap nits, require file and line as evidence, open with a count and say plainly when there is nothing to fix.
• Converge: re-review the diff only after early rounds, flag the bug class so every instance is fixed, never repeat a posted finding, and after three rounds reconsider the approach. This took PRs passing in one round from 65% to 78% and mean reviews to pass from 2.08 to 1.32.
• Every finding is fixed or closed with a reason. Repeated comments move upstream into a lint rule, skill or instruction file.

Changes

• REVIEW.md: defects defined for this repo (wrong number, verdict or exit code; a test that cannot fail; docs that say what the code does not; schema drift; Windows or None behaviour). Findings name file, line and a breaking input, name the bug class and its other sites, are pushed as fix plus test when short, are marked must-fix or nit (three nits at most), and are not repeated unless the fix did not hold. Bugs in unchanged code are noted once as pre-existing. Later rounds review the new diff and its callers and re-run the original input. After three rounds of must-fix findings, agree the approach in the thread. Threads close with the fixing commit or a reason. Repeated comments become a test, a make ci-local check or an AGENTS.md line.
• AGENTS.md, "Before opening a PR": mutate the code under each test of changed behaviour and revert; fix sibling sites in scope and list the rest; run bad inputs where they apply and say what was not run; check the title, docs and changelog against the code; re-review your own fix commits.
• .github/copilot-instructions.md: Copilot review follows REVIEW.md.
• PR template: "Revert" section and a "Self-reviewed" checkbox.

How this helps later PRs

Seen in the data	What changes
Must-fix found by running code, late	The author runs the mutation and bad-input checks before opening
Same bug found one command at a time (#189 to #194)	Author and reviewer name the class and its sites in one round
Author fixes regress, reviewer fixes miss	Every fix, either side's, is re-run on the original input
387 nit threads, each a commit and a round	Three nits per review at most
346 threads open at merge	Each thread closes with a commit or a reason
Docs and exit codes disagree with code	Checked by the author before review
The same comment on PR after PR	Moved into a test or AGENTS.md

Nothing here adds a required check or blocks a merge.

Risk

Two-way door: text only, reverted in one commit. Blast radius: contributor and reviewer workflow. Agents that load AGENTS.md or REVIEW.md change how they self-review and comment; CI, the package and stored records are unaffected.

Testing

• pytest tests/test_docs_code_refs.py: 40 passed (paths and commands named in AGENTS.md resolve).
• New and changed lines are ASCII.

Revert

git revert of this PR. Nothing is left behind.

Checklist

• Tests added/updated and pytest passes (no code change; doc reference test run)
• ruff, mypy, and schema checks pass (N/A, no code)
• Docs updated (or N/A)
• Self-reviewed per "Before opening a PR" in AGENTS.md
• No ad-hoc run artifacts committed
• New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

[Copilot]

🟢 Approval recommended

The four documentation changes are consistent with the stated scope, with no unresolved defects identified.

No open issues 🎉

What changed in this PR

Adds contributor and reviewer guidance to catch defects before PR submission and reduce repeated review rounds. The changes affect workflow documentation only; application code, CI, and merge checks are unchanged.

Changes:

• Defines actionable findings, review rounds, and thread-resolution expectations.
• Adds pre-PR self-review checks for test effectiveness, related bug sites, bad inputs, and documentation accuracy.
• Connects Copilot to the guidance and adds rollback and self-review prompts to the PR template.

File	Description
REVIEW.md	Defines defects, finding expectations, and review follow-up.
AGENTS.md	Adds the pre-PR self-review checklist.
.github/​pull_request_template.md	Adds a Revert section and self-review checkbox.
.github/​copilot-instructions.md	Points Copilot to review and coding guidance.

_{Review effort: Balanced}

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.86%. Comparing base (cdb58cb) to head (a54f76c).
⚠️ Report is 1 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #206   +/-   ##
=======================================
  Coverage   91.85%   91.86%           
=======================================
  Files         223      223           
  Lines       27712    27712           
  Branches     4563     4563           
=======================================
+ Hits        25455    25457    +2     
+ Misses       1600     1599    -1     
+ Partials      657      656    -1     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[napetrov]

I ran dokimi's own repo eval on this change, with and without the PR. It is safe to merge as text, but at this sample size it doesn't change what agents do, and agents never open REVIEW.md.

Static (dokimi repo budget, no LLM): start context goes from 7,377 to 8,182 chars (+11%), all of it the new AGENTS.md section. REVIEW.md is not loaded when a session starts. Agents reach it only through the one-line pointer.

Agent run (dokimi repo run): 4 .agent-eval tasks × {guidance:raw, this PR} × 3 reps, Sonnet 5.5, 24 cells, $2.18. The task base (16e319f) has only CLAUDE.md, so I ported the change: same REVIEW.md, and the same section appended to CLAUDE.md.

task	without PR	with PR
fix-degrade-pre	3/3 pass, 7.7 turns, $0.122	3/3, 11.3 turns, $0.143
fix-docker-reward	3/3, 10.7 turns	3/3, 11.7 turns
rename-cli-flag	3/3, 4.3 turns, $0.027	3/3, 6.0 turns, $0.042
review-pr-conventions	recall 0.93	recall 1.00 (one extra off-by-one find, noise at n=3)

• regress, ruff, mypy and diffcov were all 100% in both arms.
• repo report says the arm's additions were used in 0/3 cells for every task.
• 1 of 12 cells did a real "test fails without the fix" check (git stash, then rerun the tests). No cell ran a bad input or make ci-local.
• Net effect: +1 to 2 turns per fix/docs task (+3.6 on fix-degrade-pre) and more spend per task (+3% to +55%), with no change in quality.

Suggestions:

1. REVIEW.md is read by review bots (Copilot, the Claude Code Review app) but not by coding agents. If the goal is author-side self-review, the few rules that matter need to be in AGENTS.md itself, or the PR should say plainly that REVIEW.md is for review bots.
2. The checks that can be automated would land more reliably as gates than as prose, which is what REVIEW.md's last section recommends. Examples: docstring exit codes against the real ones, and None/Windows-path inputs for each command. These could go in make ci-local or the test-quality gate.
3. "At most three nits per review" fits a bot, but may suppress real low-severity findings from a person. Maybe: "nits only after must-fix findings; at most three."
4. The numbers in the description (57/62, the theme counts) are self-measured and keyword-matched. Worth saying so next to them.

Caveats: n=3, one model, guidance ported onto an older base. A stronger test would re-pin the tasks to a base that has AGENTS.md and add a task with a test that can't fail. Runs are available if useful.