Han Feedback: automated-test-planning-readability-guidance (2026-07-24)

[mxriverlynn]

Han Feedback — 2026-07-24

Skills used: han-coding:automated-test-planning, han-communication:readability-guidance Agents used: han-core:test-engineer, han-core:edge-case-explorer, han-core:information-architect, han-core:junior-developer, han-communication:readability-editor Context: Asked a narrow yes/no question — whether a new RSpec context ("only enabled recurring deposits") on a money-summing model method should be joined by "only disabled" and "enabled and disabled" sibling contexts. Outcome: A reviewed test plan concluding both proposed contexts are YAGNI, plus one genuine HIGH-priority coverage gap (an untested term of the sum) that both analysis agents found independently.

---

What worked well

• test-engineer and edge-case-explorer independently converged on the same verdict (defer both proposed contexts) and independently flagged the identical real gap — a whole term of the method's arithmetic never exercised at that seam — each with file:line evidence. Two-agent agreement made the verdict trustworthy.
• edge-case-explorer traced beyond the files it was given: it found the method's real external caller (a GraphQL field), discovered the disabled-exclusion scenario was already covered at that integration layer too, and confirmed the untested input path is reachable in production (no eligibility rule excludes it, and the field's own description promises the behavior). That reachability check is what elevated the gap from "symmetry" to "realistic failure mode."
• edge-case-explorer's Dropped Edge Cases section was disciplined: it verified (not assumed) that a factory trait ordering concern was harmless, and dropped boundary cases with citations to the validations that make them unreachable, each with a revisit trigger.
• The information-architect and junior-developer review passes earned their turns: they caught an undefined acronym sitting in the plan's single most important consequence sentence, dead (non-navigable) TP-ID cross-references, and the same test scenario being named two different ways across sections. All were real edits, not filler.

What didn't work

• No scaled-down path for a narrow question. A yes/no question about two spec contexts triggered the full pipeline: 5 subagent dispatches (~225k subagent tokens), a multi-section template document, two review agents, and a rewrite agent — to deliver "no, and here is one test to write instead." The skill's Step 2 always dispatches the full analysis set and Steps 4-5 always produce and review the full template. A lightweight mode for single-question scope would have delivered the same verdict at a fraction of the cost.
• YAGNI verdicts argued from coverage location, not assertion strength. Both agents justified deferring the "only disabled" context by pointing at duplicate coverage elsewhere (the collaborator's own spec, the GraphQL spec). When the user explicitly discounted those layers and pressed the symmetry argument ("if one state gets a test, all viable states must"), the plan had no answer. The decisive justification — the proposed test's zero expectation is overdetermined, so it duplicates the existing mixed context's kill set and passes under a mutant that drops the term entirely — had to be produced manually afterward with a two-mutation experiment. test-engineer's deferral rationale would be much more durable if it reasoned about discriminating power of assertions (what mutants a proposed test kills that existing tests don't), not only about where else the behavior is covered.
• readability-editor violated the voice blocklist it was applying. The writing-voice profile says "No em-dash, '—', anywhere, ever," yet the editor's rewrite introduced em-dashes into a heading and two prose sentences. The final self-check caught and fixed three instances manually. The editor should be running the blocklist over its own insertions.

Overall

For this use case the pipeline's analysis quality was excellent and its economics were poor. The two analysis agents were the right tool: independent convergence, reachability verification, and a real finding the author had not asked about. But the skill treats every invocation as a full-plan engagement, so a question answerable in one paragraph produced a full reviewed document, and the one argument the user actually needed next (assertion strength versus state symmetry) was outside the agents' reasoning repertoire. A "focused question" mode and a kill-set/assertion-strength framing in test-engineer's YAGNI rationale would both have paid for themselves in this session.

---

Rating

Dimension	Score
Output accuracy	5/5
Evidence discipline	5/5
Finding signal-to-noise	4/5
Output length vs. decision count	2/5
Turn efficiency	3/5