Skip to content

Release enforcement ladder 3.0.0 with the verification tier and capability-named rungs #15708

Closed
2 of 2 issues completed

Description

@kyle-sexton

Parent

Refs #15703
Source: PLAN Phase 6, topic ecosystem-specifics

Depends on

  • none: on the frontier now

Outcome

The enforcement ladder reads the same in its contract and five copies at version 3.0.0: capability-named rung ids, a verification tier in third place, the proportion in words and the order of work.

Demo: the ladder contract shows ### Tier 3: Verification, and the shared-copy sync check passes.

Pre-flight first: search scripts, workflows, schemas and docs for every reader of the rung ids and of a tier named by number, and list them in the pull request. Expected: four codify eval files and one fixture file name in a docs-hygiene test (kept, it is not a rung id); no script parses an id.

What changes:

  • The ladder contract, to 3.0.0: semgrep-rule becomes syntax-pattern-rule with a capability definition; custom-analyzer and architecture-test lose their tool names; a new Tier 3 "Verification" with runtime-verification and formal-verification (opt-in); Guidance becomes Tier 4 and Human only Tier 5; tests stay in static analysis; a proportion paragraph in words with no digit (most enforcement in the strongest tiers, least in guidance and human review); an "Order of work" paragraph (lint first, clean up, strengthen verification, then automate); an alias line accepting semgrep-rule with a printed notice through 3.x.
  • The ladder changelog: the 3.0.0 entry lists the rename and the tier move with the tier's reason (the talk's pyramid puts verification third; tests stay in static analysis; formal verification is opt-in).
  • The five carrier copies (review, session-flow retro, harness-config automation-gaps audit, architecture improve, corrections codify) are regenerated, each with one major changelog fragment and no plugin version edit.
  • The old conventions redirect stub keeps its headings and points them at the renumbered anchors.
  • Three codify eval cases (and samples) accept both id sets, so the corrections suite stays green until codify itself changes.
  • Carrier skill bodies that write "tier 3" or "tier 4" by number name the tier instead.

Method:

  • Each carrier's cases run as a two-version compare with graders that accept both id sets, plus one new case where consumer instructions still say semgrep-rule.
  • Run budget: 800 runs (43 codify cases and about 23 cases of the four other ladder-reading skills, counted first, on the opus and sonnet aliases, three runs per side, after two measuring runs). Select cases with --case, use --runs 3 and a --max-cost-usd ceiling, and start no batch after a usage-limit error.
  • Eval batches run in the one local eval-runner session in the main checkout. Open the pull request with the do-not-merge label; the compare record releases it.

Decisions

  • D6 (interview Q11, Q12): the ladder is the one representation of the pyramid: verification tier third, proportion in words, her order of work.
  • Q24: the verification tier and the rung rename ship together as 3.0.0: one contract change, one regeneration, five plugin bumps.
  • Q28: semgrep-rule is accepted as an alias, with a printed notice, through 3.x and removed in 4.0.0.
  • D13: public names syntax-pattern-rule, runtime-verification, formal-verification.
  • Owner, 2026-11-12 (Q5): the tier's reason and the renames are stated in the 3.0.0 changelog entry, not in decision records.
  • Her team's numbers are not targets: the proportion rule holds no digit.

Key interfaces

  • "The enforcement-ladder contract gains a verification tier in third position with two rungs, runtime-verification and formal-verification (opt-in). Tests stay in static analysis. The rung id semgrep-rule becomes syntax-pattern-rule."
  • "Verification technique: the ladder's verification tier names two capability rungs and no tool. Formal verification stays opt-in."
  • Conventions followed: the ladder's own version rule (a tier move or a removed id is a major); the contract-plus-synced-copy style.

Done when

  • AC2: the ladder has no tool-named rung id, has a verification tier in third position, states the proportion in words with no numeric target, states the order of work (lint first, clean up, strengthen verification, then automate), and its five copies are in sync.
  • The contract's frontmatter reads 3.0.0 and the changelog's 3.0.0 entry states the verification tier's reason.
  • Outside the corrections plugin and changelogs, semgrep-rule appears only on the alias line.
  • Every link in the redirect stub resolves to a heading that exists.
  • The two-version compare is non-inferior for each carrier and its record is in the pull request body, with this repository's dogfooding step.
  • AC7: nothing is cut here, so no control result is owed.
  • AC10: no new enforcement-placing skill is added here.

Out of scope

  • Codify's finding classes and hand-off (the next item on this path).
  • Selection rules rows for carriers (a later item).

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 12, 2026
  2. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    No work-resolve-issue run started in the 1 hour since agent-ready was last added. work-unblock removed and added agent-ready back to start a fresh run on main's workflow files. If this run ends the same way, it re-fires the issue again, waiting longer each time.

  3. removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 13, 2026
  4. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Large
    Complexity: Medium
    Uncertainty: Known
    Flags: none
    Blocked by: none

    I read this as a request to release the enforcement-ladder contract at 3.0.0. That means renaming semgrep-rule to syntax-pattern-rule, adding a verification tier in third place, regenerating the five carrier copies, and running the two-version eval compare. The outcome, the decisions and the done-when items are exact, so I have no open questions for you.

    The 800-run eval budget and the local eval-runner session are not something an unattended lane can do. Admission routes those steps separately.

    I found no open duplicate and no merged pull request that already does this. The related open issues are #15703 (the parent) and #15728 (codify routing, which is out of scope here).

    Automated triage. A maintainer reviews it.

  5. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane yet: sub-issue #16078 is open.

  6. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    Contributor

    Split at intake (owner Q93). This issue is Size Large, so its done-when is split into 2 sub-issues, each holding its task-list lines as written.

    Automated split by the lanes App. Admission waits on this issue while a sub-issue is open; work-unblock closes it as completed once every sub-issue is closed as completed by a merge or a listed human and its done-when is covered.

  7. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    Contributor

    Every sub-issue of this split is closed as completed by a listed human or a merge (#16077, #16078), so the lanes App closes it as completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Work status

    ready

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions