Where do acceptance criteria come from for an existing feature with no specification? #631

Activity

1. orioltf

   added

   app:unic-archon-dlc

   wayfinder:grillingWayfinder ticket: decide it by grillingWayfinder ticket: decide it by grilling

   on Sep 30, 2026
2. orioltf

   added a parent issue on Sep 30, 2026

   • wayfinder: e2e specs from acceptance criteria, at a second Consumer #538
3. orioltf commented on Sep 30, 2026

   

   orioltf

   on Sep 30, 2026

   MemberAuthor

   More actions

   Constraint from What replaces RED in /build when an e2e test for an existing feature should pass first time?, 2026-09-30

   • /specs marks each criterion as existing or new, beside its e2e class. The developer confirms the mark at Halt 1. /tickets makes a Characterisation Slice from a criterion that is both e2e-testable and existing. Whatever this ticket decides about where criteria come from, each criterion must be able to carry that mark.
   • The agent that writes a characterisation test cannot read the application source, because a test written from the code records today's bugs as correct. A criterion derived from the code has the same defect. So if this ticket lets /specs derive criteria from the feature, the answer must say how a derived criterion is kept apart from a bug in the feature before Halt 1 confirms it.
4. orioltf commented on Sep 30, 2026

   

   orioltf

   on Sep 30, 2026

   MemberAuthor

   More actions

   Input from QA practice, 2026-09-30

   The maintainer asked a Unic QA engineer this ticket's question in general terms, for no particular project. This is input for the grilling. It decides nothing.

   What QA uses when there is no specification. Testers need a test basis, something to compare the feature against. Without one, they use one of three sources:

   1. The current state of the feature, if they judge it acceptable.
   2. General expectations for that kind of feature. For a search, for example: it finds products, it refuses an empty query, it shows a message when nothing is found, and it works on the devices the product supports. This is close to exploratory testing: a list of what could be tested, written without strict requirements.
   3. The design, when one exists: a screenshot compared with the design file.

   Who confirms the criteria. Usually a product owner or a requirements engineer. The maintainer noted that such a person may not be at hand in an agentic workflow.

   The risk. The QA engineer called a missing specification an unwanted situation. A test derived from current behaviour leads to "it is not a bug, it is how the system works". That is the risk the constraint from #540 names: a criterion derived from the code records today's bugs as correct.

   An observation from test-case generation. When a ticket carries no acceptance criteria, the agent takes criteria from other context it can reach. So a gap in the ticket does not stop an agent. It fills the gap with criteria nobody wrote.

   Questions this raises for the grilling:

   • Does /specs record which of the three sources each derived criterion came from, so Halt 1 can judge it?
   • Who confirms a derived criterion at Halt 1: the developer, as for #551's class, or a person with product authority?
   • Does "general expectations for that kind of feature" belong in the chain at all, or is it exploratory testing, which is outside this map?
5. orioltf

   self-assigned this

   on Oct 1, 2026
6. orioltf commented on Oct 20, 2026

   

   orioltf

   on Oct 20, 2026

   MemberAuthor

   More actions

   Resolution, 2026-10-20

   An existing feature with no specification goes through /specs like any other feature. The run writes the feature's PRD after the fact, from what the developer says, what the running app does, or a design, and never from the application source. /specs gains one rule, one question and one field per criterion. There is no new mode, no new Box and no change to the rest of the chain.

   Which branch, and what the run is

   1. No third value. A feature with no page is source-absent. A feature with a design or a tracker item is source-present. Both branches grill, as commands/specs.md § Step 4 already says, and bootstrap and guard-not-ready do not change. The chain /specs → PRD → /tickets → issues.json → /build stays as it is.
   2. The run is guided, never automated. The person at Halt 1 is the only defence against "it is not a bug, it is how the system works". An automated run has only the app and the code to take criteria from, and item 3 forbids the code.

   Where criteria come from

   3. /specs never derives a criterion from the application source. A criterion comes from what the developer states, from a design, or from the behaviour of the running app. The code serves only the seams of Step 5. This is the rule #540 set for the agent that writes a characterisation test, for the same reason: a criterion and a test that share the code as their oracle prove nothing.
   4. Of the three sources QA named, two enter the chain.
      • The current state enters as observed behaviour. The developer confirms each observed criterion as the intended behaviour, not only as a description of what the app does today.
      • A design is already source-present, and #549 covers it.
      • General expectations for that kind of feature stay out. They are close to exploratory testing, and they join the map's fog entry "What a Practice Pack may carry about acceptance criteria", which already names generic criteria.
   5. An observed behaviour the developer judges a bug becomes a criterion for the intended behaviour, marked new. The app does not do it today, so the What replaces RED in /build when an e2e test for an existing feature should pass first time? #540 mark is new, and /tickets makes an ordinary slice from it: an Acceptance Slice when it is e2e-testable. That slice fixes the bug. No test ever records the bug as correct.

   Telling the developer

   6. At the start of Step 4, on every run, /specs asks once whether the feature already runs in the app. If it does, the agent says, before the first grilling question, that it will not read the application source to find criteria, and that criteria will come from the developer, the running app or a design. When no source specifies the feature, it also says that this run writes the feature's PRD after the fact. The notice comes before the interview because the criteria form during it.

   Who confirms

   7. The developer confirms every criterion at Halt 1, as for the class from #551. The Consumer declares in its context who holds product authority. When it declares nobody, a Consumer ADR accepts the gap "criterion confirmed by the developer, not by product", and the PRD's Confirmations section records it, as it records Is an untestable acceptance criterion refused by context, or by a change to /specs and /tickets? #551's accepted gaps. Requiring a person with product authority would stop every existing feature at the second Consumer.

   What the PRD records

   8. Each criterion carries one Source: field, with one of four values:

      • stated: the developer said it.
      • observed: seen in the running app and confirmed by the developer as intent.
      • design: taken from a design.
      • document: taken from a page or a tracker item.

      A derived criterion is one marked observed. No other mark is needed. Halt 1 sees where each criterion comes from, which is what the QA input asked for. The field sits beside the Is an untestable acceptance criterion refused by context, or by a change to /specs and /tickets? #551 class and the What replaces RED in /build when an e2e test for an existing feature should pass first time? #540 mark.

   Where each rule lives

   9. The rule in item 3, the question in item 6 and the field in item 8 go into commands/specs.md. A rule that protects the oracle must not depend on each Consumer remembering to write it. Who holds product authority (item 7) is Consumer context, as Where does /specs read acceptance criteria from when the ticket only links a specification page? #549 and Is an untestable acceptance criterion refused by context, or by a change to /specs and /tickets? #551 decided for their own questions.

   Left open, on purpose

   • The field's exact name and place in the PRD template, and how document sits beside the ID, page and version that Where does /specs read acceptance criteria from when the ticket only links a specification page? #549 keeps, go to /to-spec.
   • Whether the rule in item 3 needs a plugin ADR of its own, or a line in specs-branch-on-input, goes to /to-spec. The branch-on-input decision itself does not change.

   Implementation

   This is command work for /to-tickets. The ticket edits commands/specs.md, so it carries a blocked_by edge to #556, which edits the same file. #540's tickets also edit commands/specs.md for the existing and new mark, so the two are ordered against each other or written as one ticket.
7. orioltf

   closed this as completedon Oct 20, 2026