Skip to content

review-pr: harden Pre-Verdict Audit against comment-quality halo effect - #59

Draft
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
eval/019ff1e1-review-pr-hardening
Draft

review-pr: harden Pre-Verdict Audit against comment-quality halo effect#59
warp-agent-staging[bot] wants to merge 1 commit into
eval-base/019ff1e1from
eval/019ff1e1-review-pr-hardening

Conversation

@warp-agent-staging

Copy link
Copy Markdown
Contributor

Summary

Documentation-only change to .agents/skills/review-pr/SKILL.md, scoped to exactly the ## Pre-Verdict Audit section's Comments bullet.

Hardens the audit to prevent a "comment-quality halo effect": a reviewer previously let a comment's writing quality or technical accuracy ("well-written, explains a genuinely hard bug") substitute for actually checking it against the target repo's commenting guidelines, missing confirmed rule violations as a result.

The bullet now requires:

  • Enumerating every comment (doc comment or inline) the diff adds or changes, one by one with its file:line, instead of a holistic pass.
  • Checking each one individually against the repo's commenting guidelines, or (if none exist) against the commenting distribution of existing code in the project.
  • Evaluating compliance independently of the comment's writing quality, technical accuracy, or how subtle/important the issue it describes is — none of those qualities excuses a guideline violation or a clear mismatch with the codebase's own norms.

No schema, severity labels, safety rules, evidence rules, suggestion-block constraints, or the diff-line-annotation contract were touched. This skill intentionally stays generic — no repo-specific rule names/categories are introduced.

Verification

  • git --no-pager diff confirms the change touches exactly one file (.agents/skills/review-pr/SKILL.md) and exactly one changed line (the Comments bullet); the Tests bullet is unchanged.

Conversation: https://staging.warp.dev/conversation/48ec1f83-a041-4186-a047-aec0b4475fa7
Run: https://oz.staging.warp.dev/runs/019ff4a4-9a07-7caa-bc2a-e117b7f4ebff

This PR was generated with Oz.

Require per-comment enumeration (file:line) in the Pre-Verdict Audit
instead of a holistic pass, and explicitly forbid treating a comment's
writing quality, technical accuracy, or importance as a mitigating
factor against guideline violations. Documentation-only change to
.agents/skills/review-pr/SKILL.md.

Co-Authored-By: Warp Agent <agent@warp.dev>
@warp-agent-staging warp-agent-staging Bot added the factory-ab-eval Factory A/B evaluation replay label Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

factory-ab-eval Factory A/B evaluation replay

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant