Skip to content

Classify the remaining guidance rows blind and count the cuts per plugin #15730

Open

Description

@kyle-sexton

Parent

Refs #15703
Source: PLAN Phase 14 (first work item), topic ecosystem-specifics

Depends on

Outcome

The classification rules for model-read ecosystem specifics are shown to decide the same way for two independent readers, the rows to cut are counted per plugin, and one cut item per plugin is filed in the planned order.

Demo: the classification record on this item shows zero split rules (or the owner's ruling on each), and the per-plugin cut items exist under the container with their row counts.

Steps:

  • Blind classification: two fresh agents each classify a stratified sample of about 50 inventory rows (at least 5 per kind) under the design's classification rules R1 to R11, without seeing the other's answers.
  • A rule is split when they disagree on any sampled row that either placed under it. Reword a split rule and re-classify the disagreeing rows plus ten fresh rows, at most three rounds. A rule still split after three rounds goes to the owner; label this item needs-human at that point.
  • Count the rows to cut per plugin and state the number on this item.
  • File one cut item per plugin under the container, in this order: the discovery explore fallback table; the review ecosystem-specialist agent's generic defaults; mutation-testing's per-tool tables; user-interface's framework pages with their concerns index and its tests; planning's TypeScript idioms page with the type-discipline contract's sentence; then every other plugin with rows under rule R11, the plugin with the most rows first. A shipped default that picks a library goes with its plugin's cut.
  • Each cut item says: list every item with its rule; back every R5 or R11 call with a search showing whether a script reads the file; freeze content-dependent cases first; carry the sensitivity-control result for each whole-page deletion and the detection-control result for each partial cut; run no more than 400 runs per plugin (the changed skills' cases on the opus and sonnet aliases, three runs per side, plus control sides on one model).
  • Each cut item blocks the gate-widening item; add those edges when filing.

Decisions

  • Q27: a classification rule is split when the two blind classifiers disagree on any sampled row that either placed under it; at most three rounds of rewording, then the owner decides.
  • Q28: the remaining cuts go in the design's named order, then by plugin with the most rows to cut first.
  • D3: each plugin's cut is its own topic and pull request.
  • Interview Q32 (D4 of the Brief): bundled defaults that pick a tool, command or library are retired wherever they appear.
  • Bulk-mutation standard: count first, then act; each pull request states its row count.

Key interfaces

  • "The consumer's ecosystem (language, framework, tools): abstract now. Seam: the ecosystem-resolution contract and the consumer's own records."
  • Conventions followed: a cut ships with eval evidence (the skills-and-agents review rules).

Done when

  • The classification record shows zero split rules, or names the owner's ruling on each.
  • The per-plugin row counts are stated on this item.
  • One cut item per plugin exists under the container, each blocking the gate-widening item, and their number is stated before filing.
  • AC1 and AC7 are carried into each cut item's Done when (no generic skill ships a model-read tool, library or command choice outside the named exceptions; each whole-page deletion carries a sensitivity-control result and each partial cut a detection-control result).
  • AC10: no new enforcement-placing skill is added here.

Out of scope

  • Cutting any guidance (the per-plugin items do that).
  • Widening the portability gate.

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 12, 2026
  2. added theissue type on Nov 12, 2026
  3. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Knowable
    Flags: none
    Blocked by: #15709

    Understood: this item runs a blind two-reader classification of a stratified sample of inventory rows under rules R1 to R11, rewords any split rule for up to three rounds, states the per-plugin row counts, and files one cut item per plugin in the stated order, each blocking the gate-widening item (#15732 looks like that item).

    Nothing is missing. The issue lists #15709 as a dependency and it is still open, so this one waits on it. Parent: #15703. I found no duplicate among the open issues.

    Automated triage. A maintainer reviews it.

  4. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane yet: blocker #15709 is not closed by a listed human or a merge.

  5. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"9d79b74de8166e30535bbccff3f3fc28f2e061133b75b36cc237ef795d7c3808","body_sha256":"9828144936142295b079e265a2acb47574f048184c5eeae6c50ed72b8b726179"}

  6. github-actions commented on Nov 15, 2026

    @github-actions
    Contributor
    No description provided.
  7. added
    host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.
    on Nov 15, 2026
  8. melodic-automation-lanes commented on Nov 15, 2026

    @melodic-automation-lanes
    Contributor

    The work-resolve-issue lane stopped on this issue (host-only): the run reports the work needs a host run the lane cannot make.
    It added host-run and set Work status to confirmed: an agent run on a host builds it, and it waits on no decision from a person.

    In the run's words:

    host-only: every done-when item of #15730 is tracker work this lane cannot do, and the inputs are not in the checkout. It asks for (a) two blind classifications of about 50 inventory rows under the design's rules R1 to R11, but neither the design nor the inventory is in the repository (no file names R1 to R11 for ecosystem specifics, and git history has nothing for #15703); (b) the per-plugin row counts stated on the issue; (c) one cut item per plugin filed under container #15703, each blocking the gate-widening item; and (d) a needs-human label if a rule stays split after three rounds. This lane only writes a patch and has no tracker write, and the issue's Out of scope bars cutting guidance, so there is nothing to commit. Run it from an attended or tracker-capable session with the #15703 design and inventory to hand.

    What the run says it did not do:

    • Blind classification by two fresh agents of about 50 inventory rows under R1 to R11: the design and inventory are not in the checkout
    • Per-plugin row counts stated on #15730: needs the classification, and this lane cannot comment on the issue
    • Filing one cut item per plugin under #15703, each blocking the gate-widening item: this lane has no tracker write
    • Tidy, review, verify: skipped because no file changed

    Run: https://github.com/melodic-software/claude-code-plugins/actions/runs/43770041352

    To hand it back to the lane, remove host-run and set Work status to agent-ready again.

  9. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Classification record for the blind-classification step (done-when item 1). Result: zero split rules after three rounds. The count and the filing have not started; two questions for the owner are at the end.

    Method: the inventory's 1106 rows were parsed by script (584 model-read, 424 script-read, 98 both; the totals match the design's). A seeded stratified draw gave 50 sample rows, 5 per kind, and 30 reserve rows disjoint from them. Each round used two fresh agents that saw only the rule table, their rows and the plugins tree at the inventory's commit, never the design text, the counts or the other reader's answers. A rule is split when the two disagree on any row either placed under it (Q27).

    Round Rows Agree Disagreeing rows (reader 1 / reader 2) Split rules
    1 50 sample 45 5: R3/R5, R5/R2, R3/R5, R6/R2, R3/R6 R2, R3, R5, R6
    2 those 5 + 10 fresh 13 2: R1/R3, R1/R5 R1, R3, R5
    3 those 2 + 10 fresh 12 none none

    What was reworded (R5 and R6 were split only as the other side of a disagreement and were not changed):

    • R2 (security and safety floor), after round 1: the test now says the harm must be a security or safety one, that a quality, portability or durability finding is not, and that the row must be the list or check itself, not a pointer cited inside one.
    • R3 (declared scope), after round 1: the test no longer asks whether the capability is "inherently" bound. A set the plugin name or skill description names, where what ships works only for that set, is enough, and it covers every row of that skill.
    • R1 (out of scope), after round 2: a copy of this repository's own file kept in step by the shared-copies sync is not a vendored upstream copy; it is placed as its canonical file would be.

    Every row that disagreed in an earlier round was placed the same way by both readers of the next round, except one that needed the R1 rewording and agreed in round 3. All five rules involved in a split have a "stays" disposition, so no disagreement moved a row between kept and cut.

    Limits of this record:

    • In rounds 2 and 3 each reader saw a 12 or 15 row set, so the reworded rules are tested on 20 fresh rows plus the rows that had disagreed, not on a second full sample.
    • Three readers reported that a file read showed them the fields of one to three rows outside their set; none classified those rows.
    • Readers still named wordings they found unclear without disagreeing on a row: R6 against R7 for model-read contract ids, R9 against R10 for an example inside a distilled-source page, R4's "picks a tool", and whether a plugin-level file counts as a row of the skill that uses it. They are listed for the owner, not acted on.
    • The reworded R1, R2 and R3 tests are not yet written back into the design.

    Not done, waiting on the owner:

    1. Which exception set AC1 means. ADR 0101 names four kept classes (executed adapter, security or safety floor, declared-ecosystem surface, source citation). The design's rule table also keeps rows under R6 (mechanism record), R8 (measured fact) and R10 (illustration), which have no ADR class. A row those three rules keep is a review finding under the ADR as merged. The per-plugin cut count depends on the answer.
    2. What the count covers. The filing order speaks of rows under R11. R4 rows that are not an overridable default and R8 rows with no measured drop also go. A 50-row sample cannot give per-plugin counts: that needs every model-read candidate row classified (335 of kinds K2 to K5 plus 162 of K7).

    The cut items' recipe in this issue's steps ("no more than 400 runs per plugin ... three runs per side") predates the owner's decision of 2026-11-16 on the parent, which drops the per-cut no-loss compare and keeps each cut's control. Cut items will follow that decision. Their number is stated here before any is filed; none is filed without the owner's yes.

  10. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Correction to the record above: R3 is not settled, and it goes to the owner.

    The round 2 rewording of R3 changed what the rule means, not only how it reads. The original test asks for a capability "inherently bound" to a fixed set, which follows docs/plugin-philosophy.md ("genuinely and inherently locked"). My rewording dropped that word and said a set named in the plugin name or skill description is enough, for every row of that skill. Two readers then agreed, but they agreed on a wider rule.

    Evidence: the first 100 rows of a full pass under the reworded rules (the count step, started and now stopped) placed 44 under R3 and 2 under R11. The R3 rows include model-read guidance that the filing order in this issue treats as cuts or close to it, for example 13 code-tidying rows (the per-tool dead-code lanes among them) and 14 code-metrics rows (collector pages among them), kept because a description lists the tools.

    So:

    • The table above stands as the record of what was run. Its "zero split rules" holds for R1 and R2 as reworded. For R3 it holds only under the wider wording, which the owner has not accepted.
    • R3's round 1 disagreement was between two readings: a skill that names its formats and ships a reader for only those is in declared scope (R3), or the capability is language-neutral and only the executed reader stays (R5). Both keep the script-read rows. They differ on that skill's model-read guidance: R3 keeps it, the other reading sends it on to R6 to R11.
    • The full classification is stopped at 100 of 528 rows until R3 is ruled on; the two finished chunks are kept as evidence only. No count is stated and nothing is filed.

    Adding needs-human, as this issue's steps say for a rule the rounds do not settle.

    For the owner, R3: does a skill that lists its supported tools or formats in its description put its model-read guidance for those tools in declared scope?

    • A: no. Declared scope needs a capability that cannot exist without that set (a tool-branded plugin, an artifact for one platform, a reader that is the product). Guidance for tools a general capability happens to support is judged by the later rules. This matches the philosophy doc's wording and the cuts this issue names.
    • B: yes, as reworded in round 2. Far fewer rows are cut.
  11. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Nov 16, 2026
  12. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Addendum to the record: R6 and R7 agree only on a shared non-literal reading. Both round 1 readers reported, independently, that the rule order read literally lets R6 (mechanism record) match every model-read contract id first, so R7 (contract value) would never apply. Each then placed those rows under R7 anyway, with R6 as runner-up, which is why the comparison shows no split there. A reader who follows first-match exactly would place them under R6. The dispositions differ (R6 keeps one copy; R7 renames ids that name a tool for a mechanism class), so this is not a label-only question. R6 and R7 were not reworded in rounds 2 and 3. Those rounds' fresh rows held one model-read contract id (reserve row V13, round 3), and both readers placed it under R3, an earlier rule, so the R6 against R7 order was not tested again.

    For the owner, with the R3 question above: state in R6 that an ecosystem-specific id other skills or config write or match is judged by R7, not R6 (recommended; it is the reading both readers chose), or keep first-match as written and drop R7's model-read case.

  13. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded.

    Owner, 2026-11-16, asked whether a skill that lists the tools or formats it supports in its description thereby puts its model-read guidance for those tools in declared scope (rule R3), with the recommendation "no": "go w/ recommendations" (said in the hub session, relayed to this one).

    • R3 as decided: a description that lists supported tools or formats does not make that skill's model-read guidance declared scope. Declared scope needs a capability that cannot exist without the set (a tool-branded plugin, an artifact for one platform, a reader that is the product). Guidance for tools a general capability supports is judged by the later rules.
    • What this undoes: the round 2 rewording of R3. The 100 rows classified under it are not used. R3 needs a wording that states the decision, and one more blind round on rows that test it, before the full pass restarts.
    • Still with the owner: which exception set AC1 means (ADR 0101's four classes, or the design's R6, R8 and R10 as well), what the per-plugin count covers, and the R6 against R7 order in the addendum above. This issue keeps needs-human until those are answered.
  14. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded (AC1 exception set), with what stays open.

    Read against the texts in a fresh context, then each cited line re-read:

    • "Named exceptions" in AC1 means the four kept classes of ADR 0101 ("The named exceptions are the last four classes", docs/adr/0101-resolve-ecosystem-specifics-from-the-consumer-repository.md:49).
    • Both texts limit the cut to a choice. AC1: "no generic skill ships a model-read tool, library or command choice". ADR 0101:43 defines the cut class as "text a model reads to choose what to run, recommend or build".
    • R10 (illustration) stays, outside AC1. Its own test is that it "does not sit in a step that selects a tool", so it is not a choice.
    • R6 (mechanism record), first two clauses, stays, outside AC1. A file, key, rule id, script or behavior of this marketplace, and the upstream source of a kept script, are not a choice made for the consumer.
    • These rows fit none of the five classes, while ADR 0101:38-39 says each named tool "is one of these". That is the ADR revisit trigger at line 96 ("A class of shipped content that fits none of the five classes above"): the ADR is incomplete here, it does not order these rows cut. My earlier comment that such a row "is a review finding under the ADR as merged" was too wide and is withdrawn for R10 and these two R6 clauses.

    Not settled by the texts, left for the owner (needs-human stays):

    • R6, third clause ("how to install, run or read the output of a tool a kept adapter or kept default drives"): this is text about what to run, and a kept default has no class in the ADR ("Ship no bundled default", line 35).
    • R8 (measured fact): a measurement "that contradicts a reasonable default" exists to change what the model runs. One row in the current output carries it.

    For each: either the ADR gains a kept class for it, or the rule changes from stays to goes. The full pass does not restart until that, the count scope and the R6 against R7 order are answered.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.host-runNeeds an unattended run on a host; the host runner takes it, not a person or the issue lane.needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.

    Type

    Fields

    Work status

    confirmed

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions