Skip to content

Release enforcement ladder 3.0.0 with the verification tier and capability-named rungs - part 2 of 2 #16078

Closed

Description

@melodic-automation-lanes

Part 2 of 2 of #15708, split at intake (owner Q93). The done-when below is copied from #15708 as written.

Done when

  • The two-version compare is non-inferior for each carrier and its record is in the pull request body, with this repository's dogfooding step.

Activity

  1. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 14, 2026
  2. github-actions commented on Nov 14, 2026

    @github-actions
    Contributor

    Size: Small
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: #16324, #16325

    I read this as the closing step for #15708 part 2: the ladder 3.0.0 two-version compare, with its record in the PR body.

    Decision recorded. Owner decision Q171, option (a), "go w/ recommendations", by 2026-11-15T13:34:49Z, after the question was restated in full with three options.

    [quote truncated; the full decision is at the link]

    #16078 (comment)

    That decision settles the open questions. #16246 merges on its record, this issue stays open, and the follow-ups were filed as #16324 (harness-config Sonnet loss) and #16325 (session-flow Sonnet cases not exercised). This issue is blocked until both are delivered, or until the owner amends the done-when. No further question is raised here.

    Merged fix: no merged pull request already satisfies the done-when. #16246 is the open pull request carrying the record.

    Automated triage. A maintainer reviews it.

  3. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 14, 2026
  4. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    ContributorAuthor

    Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  5. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    ContributorAuthor

    {"v":1,"actor_id":337595936,"title_sha256":"1ce3dd93aa0f9a0358b5ab86da09862c71db44149e8e2ae71298673cf73c82fa","body_sha256":"66176a6e9bffb20fe10e01690738001b998470be8258fcaa640c717ca9ee5f07"}

  6. 17 remaining items

  7. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    ContributorAuthor

    The latest triage verdict holds nothing: admission moved the issue to status: confirmed; its comment says why.

  8. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    ContributorAuthor

    Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

    Intended outcome: Close this issue with the compare record (harness-config Sonnet a loss, session-flow Sonnet not exercised) and a follow-up issue for the loss, once the owner says what the result means.

    Related work

    Open questions

    1. Does non-inferior for session-flow require each model to be MET, or only the non-inferiority rule (rule 2)?
      RECOMMENDED: Each model must be MET. Sonnet session-flow had 0 of 24 after-side ladder reads, so the changed text was never exercised and the item is not met. Basis: The Decision recorded comment of 2026-11-15T09:44:37Z, per-rule table: session-flow Sonnet fails rule 4, AFTER ladder-read 0 of 24.
    2. How should this issue close: file a follow-up issue for the harness-config Sonnet loss, rerun the compare, or accept the result?
      RECOMMENDED: File a follow-up issue for the harness-config Sonnet loss (posttooluse-not-a-gate, 6/6 before and 2/6 after) and close this issue with the record. The loss stands under every reading, and the record is complete. Basis: The Decision recorded comment of 2026-11-15T09:44:37Z: harness-config Sonnet is a loss under every reading, and the registration orders a follow-up per losing carrier.
    3. Does the amended done-when item for registration count as met with the compare cut to 8 codify cases, or does it need the 13 rung-grading cases Q152 counted?
      RECOMMENDED: Count it as met on the 8 cases, because the owner's restart choice narrowed codify to 8 and the registration was written before the first launch. Record the 5 omitted cases as not exercised. Basis: The Decision recorded comment of 2026-11-15T09:44:37Z, done-when mapping: two readings of the registration item, and it is unclear which the owner meant.
    4. Is the repository's dogfooding step recorded in the body of PR #​16246, and what is it?
      RECOMMENDED: Have the hub add the dogfooding step to the PR body before close. No session found it in the registration's working files or the progress record. Basis: The Decision recorded comment of 2026-11-15T09:44:37Z, done-when mapping: the dogfooding item is marked UNVERIFIED.
    5. Does the owner approve the two hub judgments, amendment 3 (re-launching batch 9 after a session clear) and dropping the void batch 35 case?
      RECOMMENDED: Approve both. They change no verdict, since the by-the-letter and amendment 3 columns agree in every row. Basis: The Decision recorded comment of 2026-11-15T09:44:37Z: both are marked not owner-approved, and the verdict table shows no verdict differs.

    Sources read

    • /home/runner/work/claude-code-plugins/claude-code-plugins/refine-data/context.json: the issue body, triage and the two Decision recorded comments (Q152 amended done-when; Q161, Q165 and Q168 with the compare record). The record shows the compare not met, with harness-config Sonnet a loss and session-flow Sonnet not exercised.
    • Repository checkout: not read. The open questions turn on owner decisions in the issue comments, not on repository files.

    To answer, post a comment that starts with this line:

    **Decision recorded.** <answer>
    

    Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.

  9. kyle-sexton commented on Nov 15, 2026

    @kyle-sexton
    Member

    Decision recorded. Owner decision Q171, option (a), "go w/ recommendations", by 2026-11-15T13:34:49Z, after the question was restated in full with three options.

    Question. The compare reads not met as registered: harness-config a loss on Sonnet, session-flow not exercised on Sonnet. Ladder 3.0.0 is already on main, and pull request #16246 changes only the new alias eval case. What follows for this issue and that pull request?

    Options shown. (a) Recommended: merge #16246 on its record, keep this issue open, and file the follow-up issue per losing carrier that the registration orders. (b) Keep #16246 held until both follow-ups are fixed and a new Sonnet compare of those two carriers is met. (c) Accept the result as recorded, amend the done-when, merge and close.

    Basis given. The pull request's file list (seven files under plugins/corrections/evals/ and one changelog fragment), and the record's harness-config section: the judge passed and failed replies of the same shape on both sides, and two of the four failing runs never read the changed file. Stated against it: the registered rule calls the Sonnet result a loss, and (a) merges with that loss recorded and unresolved.

    Acceptance criteria for this decision:

    session: GitHub Support

  10. removed
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 15, 2026
  11. kyle-sexton commented on Nov 15, 2026

    @kyle-sexton
    Member

    PR #16246 (the alias eval case, on its recorded compare) merged at 2026-11-15T13:50:10Z as e202c3c under Q171 (a). This issue stays open until the follow-ups #16324 (harness-config, Sonnet) and #16325 (session-flow, Sonnet) are delivered, or its done-when is amended.

    session: GitHub Support

  12. kyle-sexton commented on Nov 15, 2026

    @kyle-sexton
    Member

    Both follow-ups of this compare are delivered, so this closes as the owner decided (Q171 (a)):

  13. removed
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Work status

    confirmed

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions