Skip to content

docs(agents): say which behavior deserves a test - #93

Merged
Programcode merged 1 commit into
mainfrom
docs/test-diet-rules
Oct 7, 2026
Merged

Programcode merged 1 commit into
mainfrom
docs/test-diet-rules

Conversation

@Programcode

@Programcode Programcode commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Adds a What to test section to AGENTS.md, plus a pointer from docs/testing.md.

  • Proof of a change is using it: lint, typecheck, build, then verify-debatelab. New tests are not a substitute.
  • Tests stay for behavior where a regression is costly or invisible: auth/RLS, billing/quotas, migrations, concurrency, parsers and trust-boundary schemas, fixed bugs.
  • Tests agents should not write: type/shape restatements, "removed feature stays removed", existence checks, static copy, render-only, mock-only, can't-fail.
  • Deleting a test that can't catch a regression isn't weakening tests; deleting one to get a pass still is. Coverage thresholds stay Adam's call.
  • Bug fixes start with a failing test for behavior in that list, otherwise a browser reproduction.

Docs-only. npm run docs:check passes.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Documentation
    • Clarified when regression tests are warranted, which behaviors they should cover, and when to verify behavior in the app.
    • Added guidance to keep test changes small and readable and to require approval before lowering the coverage threshold.

Agents prove changes by using the app and add tests only where a
regression would be costly or hard to notice: auth, RLS, billing,
quotas, migrations, concurrency, parsers and fixed bugs. Lists the
low-value tests not to write and keeps coverage thresholds Adam's call.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 17:55
@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
debate-navigator Ready Ready Preview Oct 2, 2026 5:55pm UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 2e2df86c-4010-499d-adfc-153f1825effb

📥 Commits

Reviewing files that changed from the base of the PR and between 918b58d and 69e69aa.

📒 Files selected for processing (2)
  • AGENTS.md
  • docs/testing.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The PR adds guidance for selecting regression tests, including behaviors that warrant tests and tests to avoid. It also updates the testing documentation with a summary and a link to the guidance.

Changes

Testing Guidance

Layer / File(s) Summary
Test selection guidance
AGENTS.md, docs/testing.md
AGENTS.md adds criteria for when to test, tests to avoid, and constraints on test diffs and coverage thresholds. docs/testing.md links to this guidance and summarizes when to use regression tests or app-based verification.

Estimated code review effort: 2 (Simple) | ~8 minutes

Architecture Summary

Architecture risk: 🔵 Low · up to 69e69

The change affects 2 systems.

Changed systems: AGENTS.md, docs

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — AGENTS.md (service) was modified; 1 changed file maps to changed impact.
  • observed — docs (service) was modified; 1 changed file maps to changed impact.

Before / after behavior

  • observed — Modified behavior in AGENTS.md: Adds test-selection guidance, including when tests are warranted and categories of tests to avoid. Requires test diffs to remain small and prohibits lowering the Vitest coverage threshold without Adam’s approval.
  • observed — Modified behavior in docs/testing.md: Adds a link to AGENTS.md for test-selection guidance and summarizes which regressions warrant test specs versus app-based verification.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@Programcode
Programcode merged commit 0058704 into main Oct 7, 2026
8 checks passed

This branch was successfully deployed

1 active deployment
Preview — 69e69aa1 Deployed Oct 2, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants