Skip to content

Latest commit

 

History

History
190 lines (138 loc) · 16.7 KB

write.md

File metadata and controls

190 lines (138 loc) · 16.7 KB

Write Tests (TDD Mode)

Write tests in the TDD cadence: Red (failing test) -> Green (make it pass) -> Refactor (clean up), whether test-first or test-alongside. When uncertain about a testing decision (mock this? output or state test? which quadrant is this code in?), load /tdd:principles (when the tdd plugin is enabled) for Beck and Khorikov's guidance.

Vertical slices, not horizontal layers

Write one test, then the code that passes it, then the next test; each new test builds on what the previous one revealed. Writing every test up front and all the code afterwards (horizontal slicing) yields brittle tests: they check the behavior their author predicted, pin the code's form (data structures, signatures) instead of what a user sees, and fix the test layout before the implementation is understood. They stay green when behavior breaks and go red when it is correct.

WRONG (horizontal):
  RED:   test1, test2, test3, test4, test5
  GREEN: impl1, impl2, impl3, impl4, impl5

RIGHT (vertical):
  RED→GREEN: test1→impl1
  RED→GREEN: test2→impl2
  RED→GREEN: test3→impl3

Pre-coding interface check

Before writing the first test, confirm the public interface design:

  • Which parts of the interface does this change add or alter? When the session is interactive and the change is material (a new public surface, a changed contract), confirm with the user; otherwise state the interface you assume and proceed
  • When you state the assumed interface, add one line on what a test at that seam catches and what it misses
  • Identify opportunities for deep modules: can methods be reduced, params simplified, complexity hidden behind the interface?
  • Design interfaces for testability: prefer returning results over side effects, so output-based tests are possible
  • Proceed once the interface is settled; an autonomous run states its interface assumption in the summary instead of waiting

When invoked from /implementation:implement (plan already approved) or as part of a /testing:write focused on a single function, skip the Q&A loop: state the interface you assume in one line and proceed.

Sequence

  1. Tracer bullet first. Write one test confirming one thing about the system end-to-end, to prove the path before investing in edge cases. Name tests and interfaces in the domain's own words, from the glossary read in SKILL.md Step 1 ("Domain language read?"): its canonical terms and chosen verbs, never an Avoid: word or a verb an Actions: line rejects for that operation. Follow any ADRs that govern the code being changed. Then list the remaining behavior scenarios:

    • Happy path (basic correct behavior)
    • Edge cases (null, empty, boundary values)
    • Error paths (invalid input, missing dependencies, timeouts)
    • For domain logic: business rules, invariants, state transitions

    Focus testing effort on critical paths and complex logic, not every possible edge case.

    When the change has acceptance criteria and a user-visible flow, the resolved first_test setting (SKILL.md, Settings) picks the tracer bullet. user-flow (the default): the critical user-flow test, written before the code. unit: a unit test of the first behavior on the list, with the user-flow test written later, before the task is done. Both: see Outside-in.

  2. Choose the test type. Location and framework come from the consuming project's testing conventions (or its existing test projects when undocumented); the role of each row is universal:

    Behavior Test type Location / framework source
    Pure logic, value objects, domain rules Unit project's unit-test project convention + framework
    HTTP endpoints, middleware, DI wiring Integration project's integration-test location + framework
    Service orchestration, runtime composition Integration (orchestrator) project's orchestrator (Aspire, docker-compose, tilt) + framework
    Layer dependencies, naming, conventions Architecture project's architecture-test project, when one exists
    Critical user journeys end-to-end E2E project's browser-automation tooling (see /testing:run-e2e)
    Parsers, serializers, pure transforms, collections and stateful classes Property, beside examples the project's property library; see property.md
    Behavior to keep through a replace, migrate or legacy refactor Characterization, approval or differential (pins) the project's approval or snapshot tooling; see characterization.md
  3. Write the failing test first (Red), and run it to see it fail. On a bug fix, the first red runs against the existing code and fails on the wrong behavior, not on a missing symbol. The test name is the specification. Use the project's documented naming pattern; when undocumented, mirror the ecosystem's idiom. Illustrative forms (.NET/xUnit), with casing and separators adapted to the target ecosystem:

    • Unit: {Method}_Should{Behavior}_When{Condition}
    • Integration: {Subject}_{Behavior} or {Subject}_{Behavior}_{Context}
    • Architecture: {Subject}_Should{Constraint}
  4. Make it pass (Green). Write the minimum code: don't design, abstract, or tidy, and don't optimize. Run the suite and see it green.

  5. Refactor, when the resolved tdd_refactor_step is on (the default); with off, go from green straight to the next test and write no checkpoint line; structure is then left to the review plugin's design-smell check (the code-reviewer agent, governed by its smell_baseline setting), when that plugin is enabled. Never refactor while RED: start only after a green run, as its own step, never folded into the edit that made the test pass. On a failing test you cannot tell refactor breakage from the original failure. Clean both test and production code, run the tests after each refactor edit, and keep them all green. Refactor within the slice you just wrote; a problem the new code reveals in existing code becomes a follow-up note, not part of this cycle. Candidates beyond extracting duplication:

    • Deepen shallow modules: combine or push complexity behind a simpler interface (Ousterhout: can I reduce methods? simplify params? hide more complexity?)
    • Feature envy (Fowler): logic that sends more messages to another object than its own → Move Method
    • Primitive obsession (Fowler): raw strings/ints representing domain concepts → introduce Value Object
    • Apply SOLID principles where natural. Don't force; let the shape emerge from the tests

    Close each Refactor step with one plain line (no bullet, no bold) that begins exactly Refactor checkpoint: and names what you tidied, or why nothing needed it:

    Refactor checkpoint: extracted _line_total from total_cents; suite still green
    Refactor checkpoint: nothing to tidy, the new method is one expression
    

    Repeat every checkpoint line verbatim in the final summary, so a skipped step shows.

  6. Repeat. Next test scenario from the list

Per-cycle checklist

After each Red→Green→Refactor cycle, verify:

  • Test describes behavior, not implementation
  • Test uses public interface only
  • Test would survive internal refactor
  • One logical assertion per test: one behavioral concept, not one Assert statement
  • Every expected value names its independent source, per testing:test-value; a round trip or identity check (decode(encode(x)) equals x) passes when both directions share a mistake, so pair it with a known encoded fixture
  • Existing coverage does not already catch this regression. Each contract has one primary test at its strongest boundary; another layer needs a risk of its own, such as a transport failure the primary test cannot reach. Extend a table-driven case before adding a near-duplicate
  • A bug fix adds one regression test at the boundary that owns the bug, not one per layer the bug crosses
  • The test needs no export, flag or injection hook that no production caller uses; if it does, test through the real boundary instead
  • Code is minimal for this test
  • No speculative features added
  • With tdd_refactor_step: on, the refactor ran after a green run, as its own step, and the cycle wrote its plain Refactor checkpoint: line
  • Hollow-out run, worked through: const cart = { items: [a, b, c] }; totalCart(cart); expect(cart.items).toHaveLength(3); asserts only on the cart literal, so it is green whatever totalCart does. Swap each function the test pulls from the module under test for one that gives back undefined (None in Python, null in C# or Java), run the test once, and expect red. When every assertion can be satisfied by the test's own fixtures or mocks without the subject running, point them at what the subject returns or changes (here, the total totalCart returns)

Four Pillars Assessment (Khorikov)

Resistance to refactoring is non-negotiable and maintainability stays high; protection against regressions and fast feedback trade off, so no test maxes the first three pillars at once (/tdd:principles):

  • Protection against regressions. Does this test catch real bugs? Tests that only verify trivial behavior (getters, constructors) score low
  • Resistance to refactoring. Will this test break when implementation changes but behavior stays the same? Test behavior (observable output), not implementation (internal steps)
  • Fast feedback. Does this test run quickly? Slow tests get skipped. Rough bar: a unit test well under 100ms, an integration test under 5s (this repository's heuristic; Khorikov's pillar sets no number)
  • Maintainability. Is this test easy to understand and change? No test should be harder to read than the code it tests

Determinism controls

A test that passes or fails by chance teaches the agent running it to retry instead of fix. Build these in when the test is written, not after it flakes:

  • Clock: inject the clock (a parameter, an interface, the framework's time provider) or use the test framework's fake clock. Code under test never reads wall time directly
  • Randomness: seed the random generator per test and print the seed in the failure output, so a failure replays with the same seed
  • Network: blocked by default in unit tests; an integration test opts in to the hosts it needs. External services are faked from a recorded or documented response, not a guessed one
  • No sleep: wait on a condition with a timeout, or advance the fake clock. A fixed sleep is slow when it is long enough and flaky when it is not
  • Order independence: each test builds its own state and leaves nothing another test reads. Run the suite in shuffled order (with a printed seed) to prove it
  • Parallel safety: unique temp directories, ports, database names and file paths per test; no shared mutable singletons

Runner-level switches exist for several of these (a fake clock in Playwright, seeded shuffling and per-test reseeding in pytest-randomly and go test -shuffle, socket blocking in pytest-socket). Read the current options at the source: Playwright clock, pytest-randomly, pytest-socket. Whether to fake a dependency at all follows /tdd:principles (when the tdd plugin is enabled).

Outside-in: the user-flow test first

When the change has acceptance criteria and a user-visible flow (a page, an API call, a CLI command), and the resolved first_test is user-flow (the default):

  1. Write the critical user-flow test before the code, from the acceptance criteria: the steps a user takes and the outcome the criteria state. It fails (Red). One test per critical journey, not one per criterion
  2. Keep it as the oracle for the whole task. Run it after each slice; the task is not done until it passes. Never change its expectation to match what the code does. If the criteria turn out wrong, change them with the user first, then the test
  3. Work inside it with unit and integration tests in vertical slices, as above. The pyramid below still decides how many of each

With first_test: unit, start with a unit test of the first behavior on the list and build outward in vertical slices. Write the critical user-flow test, from the acceptance criteria and never from the code's output, before the task is done; once written it is the oracle of step 2 above.

Name the costs: E2E tests are slower and flakier, and a failure does not say which component broke (poor fault isolation; Wacker, Google Testing Blog, 2015). Offset them: keep the flow count small, retain a trace on failure, and use the failure evidence /testing:run-e2e captures (failing step, expected against actual, trace, console and network errors) so the failure report becomes the next prompt. Fault isolation comes from the unit tests inside.

Basis: LLM-written test oracles were measured to assert what the code does rather than what it should do (three papers, measured on LLM test generators over benchmark repositories; the transfer to an interactive agent is inferred). A test written from acceptance criteria before the code exists cannot copy the code's behavior. Recorded in the agent-self-check research slice, 2026-10-06. Recheck trigger: a measurement on interactive coding agents that contradicts it. For acceptance tests written by a separate agent, see blind-author.md.

Verify through the interface, not around it

A managed database (one only your application touches) is real: reading the row back after the act step is legitimate state verification, as long as the test asserts the fields rather than that a row exists. What couples a test to the implementation is reading internal tables the public contract never exposes when a public read path exists. testing:test-value §3 owns the rule. Illustrative: .NET/xUnit; the principle is ecosystem-agnostic:

// GOOD: state verification of a managed database, asserting the fields
[Fact]
public async Task CreateUser_StoresTheUser()
{
    var user = await _sut.CreateUser(new("Alice"));
    var row = await _db.QuerySingleAsync<UserRow>("SELECT * FROM Users WHERE Id = @Id", new { user.Id });
    Assert.Equal("Alice", row.Name);
}

// BAD: reads an internal index table the contract never exposes, while SearchUsers is the public read path
[Fact]
public async Task CreateUser_IndexesTheName()
{
    await _sut.CreateUser(new("Alice"));
    var key = await _db.ExecuteScalarAsync<string>("SELECT NormalizedName FROM UserSearchIndex");
    Assert.Equal("ALICE", key);
}

When no public read path exists and the stored state is not the contract, reaching around the interface is a design signal: the interface is missing an observable output.

Test Pyramid vs Testing Trophy

  • Backend (domain + application layers). Follow the test pyramid (Fowler): many unit tests, moderate integration, few E2E. Domain logic is well-suited to isolated unit testing
  • Frontend / API boundary (endpoints, middleware, UI). Lean toward the testing trophy (Dodds): weight integration tests more heavily. "Write tests. Not too many. Mostly integration." Component interactions at the boundary are where bugs actually hide
  • Architecture rules. Always run the architecture-rules test suite when the project has one. Cheap, fast, and it catches structural drift before it compounds

When not to write tests

No tests needed for:

  • Pure contracts (interfaces, attributes, records with no logic)
  • Constants. A drift guard, where one is needed, compares the constant to its external source of truth (the spec, the consumer's value), never to a literal copied from the code
  • One-liner delegation methods
  • Configuration wiring tested end-to-end through the repo's E2E orchestrator

Code that does need covering may still not be worth this test. Prefer no new test to a bad one when the only available test would need broad harness setup, brittle mocks, slow end-to-end infrastructure, production-only state, a reproduction nobody can state precisely, or large unrelated fixture churn. A test that mostly exercises its own mocks, encodes today's implementation, or would be deleted the moment it has proved its point costs more to maintain than the confidence it buys.

Declining is not skipping: say which of those made the test impractical, then name the closest executable check you used instead (a targeted script, a reproduction command, a snapshot comparison, a log assertion, a focused integration check). A silent decline is a gap nobody can see.

Commit discipline

  • Failing test committed (optional but valuable). Proves the bug/requirement exists in git history
  • Fix + green test committed together. The fix and its proof are atomic
  • Commit at green, before refactoring. Separate structural from behavioral commits