Skip to content

Generate an e2e suite from acceptance criteria, with the DLC's shape, at a second Consumer #508

Open
1 of 1 issue completed

Description

@orioltf

Destination

A chain that takes a project's acceptance criteria and produces a Playwright suite living in that project's repository, reviewed by a human before it lands, and a run that reports findings as tickets. The suite is the deliverable. Generating it is the work; operating it is not.

The shape, as the concept board draws it:

Requirements ─┐
              ├─→ /gate ──→ /generate ──→ /verify ──→ /review ──→ PR review ──→ /run
Existing repo ┘   testability  Playwright   test vs AC   code quality   HUMAN      executes the
                  gate         per test-    NOT against  selectors,    approves    suite, findings
                               object       the running   maintain-     or sends    → tickets,
                                            app           ability       back        test report

A second Consumer drives it: a different client, under a different contract, on a tracker of its own. That client is never named here, and no detail that would identify it belongs in this repository.

Why this is not the /qa Box

Superseded on 2026-09-23 by #538. The chain is now the DLC's own line (/specs, /tickets, /build, /pr-review, /qa, then CI), adapted by context, plus an e2e Practice Pack and any Box changes that turn out to be unavoidable. So this map may extend the existing Boxes. The paragraph below is kept as it was written.

The concept board says it in its own open points, and it is the reason this is a map of its own: unic-archon-dlc enters as a structural template, and its QA Box is unit and TDD shaped, not end-to-end. Nothing here extends unic-dlc-qa.yaml.

Amended 2026-09-22 — what was measured, and what it changed

This body was written from a reading of the pilot taken before any of it was verified. It was wrong on every line about that Consumer, and two decisions it presents as open have since been taken. The sections below replace what they correct; the sections above stand.

The detail behind every measurement stays out of this issue on purpose. It is in the seat's own notes, outside git.

The pilot is further along than this issue assumed

What this issue said Measured
No test runner It has one, wired into its own CI
No Playwright Playwright is already a resolved dependency, though not as an end-to-end runner
The pipelines deploy only, so a suite has nowhere to run A test stage exists, publishes results and gates on quality
No end-to-end directory A documented Playwright suite already exists, on a branch, in an open pull request

That last row is the one that matters. A colleague built it: a suite home, a device and language matrix, environment configuration, page objects, fixtures, a naming contract, and a README that reads like the specification for exactly the generator this map wants to build. It is in review, not abandoned.

So the first deliverable is not "the pilot fails the preconditions". It is to adopt, repair or overturn an existing human design, with its author in the room. That is a different job and a more delicate one.

The four preconditions, measured rather than predicted

The first two rows were measured on 2026-09-23 by #546. Its comment carries the counts. In short, most criteria are machine-usable but live on a linked specification page rather than in the ticket, and no run creates or resets test data today. The table below is kept as it was written.

Precondition State
Machine-usable acceptance criteria unmeasured; needs the client's tracker
Stable, resettable test data unmeasured. Setup and teardown are wired in, but the teardown is carried over from another project
A test environment met in shape: three environments, each with its own configuration and an uncommitted secret
An execution target a pipeline exists on the branch but has never run on a merged branch

The first decision had a fourth option, and the home has been decided

This issue framed the first decision as a new Box, a new plugin, or a client-side fork. A fourth existed and this issue did not contain it: converge with what already exists. A QA and test-generation plugin had already been built inside Unic under the same programme code this issue cites, carrying a generator, a test-code reviewer and a reference architecture distilled from the very suite described above.

Subtract that and this map adds two things: the testability gate, which exists nowhere, and orchestration — an ordered chain with a human gate and a cap on iterations, where what exists is invoked conversationally.

A grilling on 2026-09-22 settled the shape. The "how" is a Practice Pack: a family-level, independently installable unit carrying both reference architecture and procedure, so the engine stays ignorant of any one test tool. Packs are plural and per-family, not one shared repository, and the pack goes to a new external repository, location still open. The reason travels: the existing plugin lives in a personal organisation and is private by default, which is no home for something several project teams must install.

Whether that plugin's content is reused, and on what terms, needs its authors. It is not settled here.

Two questions the shape raises belong to the engine and are filed there, not here: how several packs coexist in one Consumer, and how a node proves it reached a capability it declared.

Two environments, with different rules

CI is not prepared to run an agent runner, and this map does not assume one ever runs there. What CI runs is the suite a developer already ran locally after developing it with the chain.

Generation Execution
Where a developer's machine CI
Has agents, tool servers, live sessions none of them

A generated test that quietly depends on anything agent-side passes where it was written and cannot run where it is meant to live. That is a boundary on the generator, not a caveat.

The reviewer question has an incumbent

This issue lists the reviewer as already answered, and the evidence for unic-dlc-pr-review stands. What it did not know is that the pilot's team already took this decision: it runs a public review plugin, enabled in committed settings and documented in its README with the exact invocation, chosen deliberately because integrated review tooling on its tracker was judged too expensive, with the condition for revisiting written into the commit message.

So proposing unic-dlc-pr-review there is displacing an incumbent, not filling a gap — and by this repository's own history that incumbent is the generation of toolkit unic-pr-review was rewritten from. It is a conversation with that team against their stated condition, not a settled point.

What this map still has to settle

Routed on 2026-09-23 to #538. Item 1 goes to #543. Item 2 goes to #549 and #550. Item 3 goes to #549. Item 4 is still open, under #538 § "Not yet specified". Item 5 goes to #548. Item 6 is settled in #538 § "Settled while charting", point 6: the existing gates vocabulary covers the human gate. Item 7 goes to #543. The list below is kept as it was written.

  1. Which conventions the generator must honour, and by which of three transports each one travels: a fact goes to a file a node is instructed to read, a capability goes to a declared skill, and anything executable becomes a check. Anything that fits none of the three does not reach the implementer at all.
  2. The preconditions: which the pilot must meet before generation starts, and what the chain does when one is missing.
  3. Where the acceptance criteria come from, and who guarantees their format.
  4. The numbers the board leaves blank: how many verify iterations before the chain gives up, and what share of review findings is acceptable at the PR-review exit. The iteration cap is a wall-clock question as much as a correctness one — one measured chain run took over half an hour.
  5. The baseline to measure before starting — manual test cases, QA effort per sprint, escaped defects — because without it nothing afterwards is provable.
  6. Whether the chain's Boxes need the human gate the board draws, or whether the existing gates vocabulary already covers it.
  7. Who owns the suite directory once a generator writes into it.

Scope boundary

Superseded on 2026-09-23 by #538. Box changes for this work are tickets in #538, and some edit the same Box files as #373's tickets. The maintainer decides who edits what at dispatch time. A cross-map blocked_by edge records the order, for example #540 and #542 are blocked by #439 and #505. The paragraph below is kept as it was written.

This is a separate orchestrator seat from the one running #373 and #498. It shares the plugin repository and nothing else. It does not edit unic-dlc-build.yaml, unic-dlc-qa.yaml, unic-dlc-pr-review.yaml or unic-dlc-explore.yaml while #425, #439 and #505 are in flight.

Source: the concept board of 2026-09, for the implementation and execution step of Unic's software quality management process, reviewed in the call of that date. Charted on 2026-09-23 as #538.

Activity

  1. orioltf commented on Sep 22, 2026

    @orioltf
    MemberAuthor

    The body was amended on 2026-09-22

    Saying so here, because anyone who read this issue before today read a picture of the pilot that was wrong on every line, and a body changing under a reader with no notice is worse than the error it fixes.

    What changed, in order of how much it moves the work:

    1. The pilot already has a documented Playwright suite, in an open pull request, built by a colleague. So the first deliverable is to adopt, repair or overturn an existing human design with its author present — not to record that the pilot fails its preconditions. It also has a test runner and a CI test stage, both of which this issue said it lacked.
    2. The first decision had a fourth option this issue did not contain: converge with a QA and test-generation plugin already built inside Unic under the same programme code. A grilling on 2026-09-22 settled the shape as a Practice Pack in a new external repository. Two questions that shape raises are filed against the engine rather than here.
    3. CI runs no agent. Generation and execution are two environments with different rules, which is a boundary on what a generated test may depend on.
    4. The reviewer question has an incumbent, chosen deliberately by the pilot's team with the condition for revisiting written down. Proposing a replacement is a conversation, not a settled point.

    The closing list went from six items to seven: the Box-versus-plugin question is closed and replaced by which transport each convention travels by, and "who owns the suite directory once a generator writes into it" is added.

    Every measurement behind this sits in the orchestrator seat's own notes, outside git, because the detail identifies the client.

  2. orioltf commented on Sep 23, 2026

    @orioltf
    MemberAuthor

    The body was amended again on 2026-09-23

    This issue is now charted as #538. Its Notes supersede two sections of this body, so both sections now open with a note that points there:

    • § "Why this is not the /qa Box". The chain is now the DLC's own line, adapted by context, plus an e2e Practice Pack. It no longer runs beside the DLC.
    • § "Scope boundary". This work may now hold tickets against the same Box files as wayfinder: unic-archon-dlc is prose, and proven at a Consumer #373. The order is recorded as cross-map blocked_by edges.

    The original paragraphs are kept below each note, so a reader can see what changed. The last line now says the issue is charted, instead of asking for /wayfinder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions