From 69e69aa1b94242f0ac91d9fca0ae73e706cd38ec Mon Sep 17 00:00:00 2001 From: Adam Date: Fri, 2 Oct 2026 17:55:15 +0000 Subject: [PATCH] docs(agents): say which behavior deserves a test Agents prove changes by using the app and add tests only where a regression would be costly or hard to notice: auth, RLS, billing, quotas, migrations, concurrency, parsers and fixed bugs. Lists the low-value tests not to write and keeps coverage thresholds Adam's call. Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.md | 23 ++++++++++++++++++++++- docs/testing.md | 2 ++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 5e9990d1b..50a32f8fc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -106,9 +106,30 @@ The skill is maintained code. A change that breaks it or makes it stale must upd Never weaken a recipe's assertions to make it pass. +## What to test + +Prove a change by using it: `npm run lint`, `npm run typecheck`, `npm run build`, then drive the feature in the real app with `verify-debatelab`. New tests are not a substitute for that, and a test-heavy diff is harder to review. + +Write a test only when a regression would be costly or hard to notice by using the app: + +- auth, sessions, RLS and pgTAP policies, and cross-user or cross-debate isolation; +- billing, Stripe webhooks, quotas, daily limits, and cost accounting; +- migrations and data integrity, concurrency, and the collaboration service; +- parsers, validators, schemas at a trust boundary, prompt routing in the fake provider, and stream accounting; +- a fixed bug that could plausibly come back. + +Do not write tests that: + +- restate a TypeScript type or a schema's shape; +- check that a removed feature stays removed, or that a column, file, export, route, or translation key exists (the i18n catalog check already covers keys); +- assert static copy, or only that a component renders; +- only exercise mocks, or can never fail. + +Keep a PR's test diff small enough to read next to the change. Deleting a test that cannot catch a real regression is not weakening tests; deleting or loosening one to make a failing run pass is. Never lower a coverage threshold in `vitest.config.ts` without Adam's approval. + ## Hard boundaries -- Start bug fixes with a failing test or repeatable browser reproduction when a suitable test layer exists. +- Start bug fixes with a failing test when the bug is in behavior listed under [What to test](#what-to-test), otherwise with a repeatable browser reproduction. - Stub sessions prove rendering only. Do not call them authenticated backend proof. - Never put staging or production credentials in jobs, logs, traces, screenshots, commits, comments, or docs. Two scoped exceptions reach jobs only through GitHub Actions secrets: - the protected staging values; diff --git a/docs/testing.md b/docs/testing.md index 7f4b0afb6..3965ffce7 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -48,6 +48,8 @@ npx vitest run src/test/security/url-validation.test.ts For a bug, watch the reproduction fail for the expected reason before implementing the fix. +Which behavior deserves a test, and which tests not to write, is in [AGENTS.md](../AGENTS.md#what-to-test). The short version: test specs where a regression is costly or invisible (auth, RLS, billing, quotas, migrations, concurrency, parsers, regressions of fixed bugs), and prove everything else by using the app. + ## Collaboration service tests ```bash