Skip to content

bug(unic-archon-dlc): goals-check counts, the precheck reads, and only the counter reaches evidence.json #483

Closed

Description

@orioltf

Grilled 2026-10-27 with the maintainer, against origin/develop at 579b4ef4 (plugin 0.31.0).
Every criterion that touches evidence, report or open-pr is written against
#439's criteria, not against today's
tree. #439 merges first: evidence writes evidence.json on every verdict, and a verdict other
than pass cancels the run before any pull request opens. Read #439's body before this one.

Observed

Run 3 build c283e296. implement-review-precheck found 11 defects, one a documented standard breached (an invented icon glyph frozen into three snapshot baselines) and three acceptance criteria broken (13 snapshot on the wrong element, 24 focus ring missing a bound token, 25 no hover story). Five minutes later goals-check marked all 30 criteria COVERED, including those three: it counts that something exists that answers the criterion; the precheck reads what it contains. evidence.json certifies from verification and goals-check only, by the Box's own design ("a prompt that both judges and certifies grades its own homework"), so the run completed with verdict: pass and two empty failure lists. report.md § 5 carries the precheck's findings, marked "not posted — /pr-review posts".

Evidence: findings register § G row G8 at https://github.com/unic/unic-agents-plugins/blob/bcf688a/docs/research/2026-08-31-dlc-run-2-findings.md, and the dispatch record on #457.

The decision

A precheck finding that contradicts a claim the run makes turns the verdict to fail. A precheck that could not review the code turns it to unresolved. A judgement call is a note: it reaches evidence.json and the pull request description, and changes no verdict.

goals-check stays a presence counter. evidence is the only node that reads the precheck's verdict fields, so the fact has one source.

The vendored code-review Method already separates the classes this uses. Step 4 asks the Standards sub-agent to "distinguish hard violations from judgement calls", and says baseline smells are always judgement calls. The Spec sub-agent reports (a) missing or partial requirements, (b) scope creep and (c) requirements implemented wrong. What is missing is a structured field. Today the precheck emits one prose string, findings.

Acceptance criteria

  1. The precheck emits structured fields beside findings. Its output_format gains:

    • status, an enum of ran and blocked;
    • blocked_reason, a string, empty when status is ran;
    • hard_findings, an array of objects { axis, category, citation, summary }, where category is one of documented-standard, spec-missing and spec-wrong;
    • notes, an array of strings.

    All four are required by the schema. findings stays as it is, and report still folds it in verbatim.

  2. The precheck's prompt states the classification inline. A Standards finding that breaches a documented standard, citing the standard's file and rule, is documented-standard. A Spec finding of kind (a) is spec-missing, and of kind (c) is spec-wrong. Each hard_findings entry quotes its citation: the standard's file and rule, or the spec line. A baseline smell and a Spec finding of kind (b) go in notes, never in hard_findings. The vendored Method is not edited; the rule lives in the node's prompt.

  3. The precheck reports blocked whenever it reviewed nothing. That covers the three cases its prompt names today: no merge-base resolves, the Agent tool is unavailable, and the diff against the base is empty. blocked_reason carries the sentence the prompt already prints for each, and findings still carries it too. A blocked precheck still emits every required field, with hard_findings and notes as empty arrays, so the schema does not abort the run.

  4. evidence reads the precheck, and depends on it by name. implement-review-precheck is in evidence's depends_on. evidence reads status, hard_findings and notes from it. The comment block's warning about dollar-prefixed tokens stays (bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439 criterion 15 holds it too).

  5. The verdict counts hard findings, and stays red when in doubt. On top of bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439's rule:

    • A non-empty hard_findings makes the verdict fail. An entry with an empty citation, or a category outside the enum, still counts as a hard finding. It is never dropped.
    • status blocked, or any value other than ran, makes the verdict unresolved when nothing else is fail.
    • The precedence is the one evidence already uses: fail over unresolved, unresolved over pass.
    • No when expression changes. bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439's siblings compare with 'pass' only, so both new outcomes take the cancel branch.
  6. evidence.json carries what the precheck reported. Beside bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439's fields it carries precheck_status, precheck_blocked_reason, hard_findings and precheck_notes, on every verdict. A reader of a cancelled run learns from this file which hard finding turned it.

  7. report.md names the hard findings where a reader looks first. Section 2 leads with a warning naming the number of hard findings when there are any, as it does today for uncovered criteria. Section 3 says in one sentence that COVERED means a test and an implementation exist, not that either is correct, and points to the hard findings. Section 5 keeps the precheck's prose and its label.

  8. The pull request description carries the notes. The node that opens the pull request, as bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439 leaves it, adds a ## Pre-check notes section listing precheck_notes from evidence.json, or "none". It reads the file, not the precheck's prose. hard_findings is empty by construction when a pull request opens, so the section does not list it.

  9. Every sentence that describes the old contract is rewritten. Find each by its claim, not by a line: evidence's header comment, and the verification and goals-check prompts where they name what evidence reads; the precheck's prompt where it says its output only informs the report and the human at build-pr-gate; docs/boxes/unic-dlc-build.md where it describes the precheck and evidence.

  10. The build-pipeline diagram shows the new edge. docs/architecture/20260925-unic-dlc-build-pipeline.json gains the edge from implement-review-precheck to evidence, on top of bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439's changes, and both its PNGs are exported again.

  11. ADR 0034-evidence-gate-deterministic-writer is revised inline by the convention in the plugin's docs/adr/README.md. It states that the precheck is now a gate input, that its hard_findings and status are self-reported by a prompt exactly as passed is for verification and goals-check, and that evidence treats a malformed entry as hard rather than dropping it.

  12. CHANGELOG.md records the change under ### Breaking. A /build run whose precheck reports a hard finding, or cannot run, used to end pass with a pull request open. It now ends cancelled with none.

  13. No precheck text can stop evidence from running. A node-output reference substitutes raw into evidence's script source, and only booleans and arrays land as valid literals. So no precheck value reaches evidence's source as a quoted free-text string. A blocked_reason, a note, a citation or a summary that holds a quote, a backslash, a backtick, a newline or ${ leaves evidence running and writing evidence.json. So does a status outside its enum. A precheck output that evidence cannot read makes the verdict unresolved or fail, and never crashes the node. The proving case is the blocked precheck with no merge-base: its sentence today is cannot resolve base branch '$BASE_BRANCH' via origin or local — no merge-base found, with single quotes, and that run must end unresolved, not failed.

What this does not prove

A Box change is unproven in the Consumer until a Consumer run reads it. A run in this repository does not prove it. Run 4 on #498 is the first read.

The bar for that run:

  • Run 3's case: a build whose precheck reports a broken acceptance criterion ends with verdictState fail, and its evidence.json lists the finding.
  • A build whose precheck is blocked with no merge-base ends with verdictState unresolved, and evidence.json carries the blocked sentence intact.
  • A build whose precheck reports nothing hard and ran ends exactly as it does after bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439, with any notes in the pull request description.

Out of this ticket

  • A fix loop inside /build that lets the build repair what the precheck finds. Decided 2026-10-27: a separate Feature, after run 4. It would rest on this ticket's verdict for the case where its attempts run out, and it must not mark a finding fixed because a second precheck stopped mentioning it (#482's defect).
  • build-pr-gate's on_reject pass pushes a fix without running the precheck or evidence again. That is older than this ticket and not changed here.

Related

  • #439: lands first. Same file, same node, same diagram.
  • #488: also edits unic-dlc-build.yaml.
  • #465: same family. A run that found a breach looks like one that found nothing.
  • #430: the unresolved state this ticket reuses for a blocked precheck.

Filed 2026-09-17 by wayfinder-orchestrator-10 from run 3's measurements.

Activity

  1. added
    bugSomething isn't working
    needs-specsEnough info from reporter; ready to write a spec
    p1High priority
    on Sep 16, 2026
  2. orioltf commented on Oct 27, 2026

    @orioltf
    MemberAuthor

    Stale — do not follow. Superseded on 2026-10-28 by the /afk-loop 498 opener below.

    Opener — How to start this session (written 2026-10-27 by unic-sdlc-orchestrator-7, at dispatch)

    What this session produces

    Approved acceptance criteria in this ticket's body, then the label ready-for-agent. Nothing else. The fix runs later under /afk-loop 498. This ticket blocks #502, run 4.

    Start: the maintainer types /wayfinder 498 #483. This ticket's parent is #498. When the grilling starts, the maintainer types /grill-with-docs. Both carry disable-model-invocation, so the session cannot invoke them itself.

    Do not close this ticket. This overrides /wayfinder step 4. This stream grills a ticket into its own criteria and implements it under the same number. Do not invoke /implement, /tdd or /to-tickets. Keep needs-specs until the maintainer approves.

    Read in this order

    1. This ticket's body. It was filed on 2026-09-17 from run 3, and its two "Not decided" points are the starting questions.
    2. #439's criteria, in full. bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439 is ready-for-agent and runs tonight under /afk-loop 498, before this ticket. It rewrites the evidence node.
    3. apps/claude-code/unic-archon-dlc/.archon/workflows/unic-dlc-build.yaml on origin/develop: implement-review-precheck, goals-check, evidence, report and open-pr.
    4. Map #498 § The destination's third condition. This ticket is one of the six claims the chain makes about its own work: evidence.json certifies "every criterion covered" from a check that counts presence.

    The trap that costs an hour

    Grilling against the evidence gate on develop. This ticket's body asks whether evidence_policy should read the precheck's findings. But evidence_policy checks only that the file exists, never what it holds. And #439, which merges before this ticket runs, changes evidence to write evidence.json on every verdict, and makes a verdict other than pass cancel the run before any pull request opens. So the real question is whether evidence's verdictState counts the precheck's findings. A criterion written against today's withhold-on-red evidence will fight #439's merged tree. Write every criterion that touches evidence, report or open-pr against #439's criteria, and say so in the body.

    What this ticket does not decide yet

    Each of these is a question for the maintainer. None is yours to settle with a plausible default.

    • Where the fix lives. Either evidence reads the precheck's findings, or goals-check reads the precheck before it marks a criterion COVERED, or both. Today evidence depends on verification and goals-check only.
    • Which findings count, and who classifies them. implement-review-precheck emits one prose string, findings, with no severity and no structure. The body says "hard findings", but no such field exists. In run 3 the precheck found 11 defects: one breached a documented standard, and three broke acceptance criteria. Decide which of these turns the verdict, and whether the precheck must emit a structured field for it. A structured field is still self-reported by a prompt. evidence's own header comment names that limit for verification and goals-check.
    • What a counted finding does to the run. After bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439, a verdict other than pass cancels the run before a pull request opens, so /pr-review never sees the code. Decide whether a precheck finding should be fail, a separate state, or a note that leaves pass standing but must appear in the PR description.
    • A precheck that could not run. PRECHECK BLOCKED: … today ends up as findings text and changes no verdict. Decide whether a blocked precheck is unresolved, the way bug(unic-archon-dlc): the config maps commands to nodes and the Boxes name tools, so a check runs by luck or not at all #430 treats a check that could not run.
    • The open-pr description. This is the body's second "Not decided" point: whether the pull request must carry the precheck's findings.

    Other traps

    The bar that proves it

    Run the four reads from docs/agents/dispatching-and-learning.md § Before you post one on the criteria as a set. Then check:

    • Run 3's case: a build whose precheck reports an acceptance criterion broken does not end with verdictState pass, or, if the maintainer chooses the note, the pull request description carries that finding.
    • A build whose precheck reports nothing ends exactly as it does after bug(unic-archon-dlc): open-pr precedes its gate, and a config-value gate has no safe middle #439.
    • Every criterion that reads the precheck names the field it reads, and that field exists in the precheck's output_format after the change.

    A false pass is goals-check told to "consider" the precheck in prose: the counter still reaches evidence.json, nothing structured changes, and run 3 repeats.

    On landing

    1. Rewrite this ticket's body with the approved criteria, and post a comment saying what changed.
    2. Before you apply any label, report to unic-sdlc-orchestrator-7 with the link. The orchestrator reads the criteria as one set and replies.
    3. Then replace needs-specs with ready-for-agent, remove wayfinder:grilling (the decision is made, and this stream does not close the ticket), keep p1 unless the maintainer moves it, and leave it unassigned for /afk-loop 498.
    4. Change no file in the repository.

    Report back

    BEFORE YOU FINISH, AND WHENEVER YOU LEARN SOMETHING THAT WOULD HAVE CHANGED THIS PROMPT,
    report to unic-sdlc-orchestrator-7 with four things and nothing else:
      1. what you expected
      2. what happened
      3. the command or file that proved it
      4. whether it changes only this ticket, or every session in this repo
    Send it when you learn it, not only at the end — a session that dies unreported
    takes its learnings with it.
    Talk only to unic-sdlc-orchestrator-7. Do not message another orchestrator or its
    sessions; tell unic-sdlc-orchestrator-7, who relays it.
    

    Review and handover

    There is no pull request. The maintainer approves the criteria in the session. The orchestrator reads them as one set before the label goes on. Implementation runs under /afk-loop 498, which this seat records as unproven in the Consumer until a Consumer run reads it.

  3. added
    ready-for-agentFully specified, ready for an AFK agent
    and removed
    needs-specsEnough info from reporter; ready to write a spec
    wayfinder:grillingWayfinder ticket: decide it by grilling
    on Oct 27, 2026
  4. orioltf commented on Oct 27, 2026

    @orioltf
    MemberAuthor

    Grilled 2026-10-27 with the maintainer. The body now holds 13 acceptance criteria, written against #439's criteria. The two "Not decided" points are answered.

    What changed:

    • A hard precheck finding makes the verdict fail. Hard means a documented-standard breach, a missing or partial requirement, or a requirement implemented wrong. Smells and scope creep are notes only. They reach evidence.json and the pull request description.
    • The precheck emits structured fields beside its prose. evidence reads them and treats a malformed hard entry as hard.
    • A blocked precheck, or one with an empty diff, makes the verdict unresolved.
    • The fix lives in evidence only. goals-check stays a presence counter, and report.md says what COVERED does not mean.
    • Criterion 13 came from the orchestrator's set-read. A precheck string is substituted raw into evidence's script, so a quote in the blocked sentence would crash the gate. The criterion requires that no precheck text can do that.
    • A fix loop inside /build is out of this ticket. It becomes a separate Feature, after run 4.

    Native dependency added: this ticket is blocked by #439, which edits the same file, node and diagram.

  5. 2 remaining items

  6. added
    resolvedImplemented; ready for a PR
    and removed
    ready-for-agentFully specified, ready for an AFK agent
    on Oct 28, 2026
  7. added
    closedPR has been merged
    and removed
    resolvedImplemented; ready for a PR
    on Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions