Skip to content

Have implementation:implement read the consumer's own idioms for the language it detects #15835

Description

@kyle-sexton

Parent

Refs #15661 (the map decision this implements).

Depends on

Outcome

When /implementation:implement settles a type choice, it uses the consumer's own idioms for the language it is working in, found in the consumer's repository, instead of only the language-neutral rules.

Demo: in a repository that declares an idioms file for its language, implement a change that needs a cast or a parse; the run cites that file. In a repository that declares none, the run says so and uses the neutral rules.

Source: Lauren Tan's pstack (cursor/plugins, MIT), pstack/skills/typescript-best-practices/SKILL.md: a per-language rule table read while building. Her TypeScript rules are not taken.

Decisions

  • Owner, 2026-11-12: a skip must be validated. The earlier reason ("it conflicts with shipping no language specifics") hid a real gap: the type-discipline rules say "Each skill or agent that reads this file says where its own language idioms live, if it has any", and implement names none.
  • The marketplace ships no idioms file for this. The consumer declares one per language, resolved through the ecosystem-resolution procedure; with none declared, implement reports that and continues on the neutral rules.
  • No fallback to a shipped per-language page: the approved ecosystem-specifics plan removes the one that exists.

Key interfaces

  • The ecosystem-resolution contract (the consumer's records, then inference) is how the language and its idioms file are found.

Done when

  • The implement skill states where its language idioms come from: a consumer-declared file for the detected language.
  • With no declaration, the run names the missing idioms source and proceeds; it reads no shipped language page.
  • One new eval case covers each branch; the skill's existing implement cases are non-inferior on a two-version compare (opus and sonnet aliases, three runs per side, --case filter, a --max-cost-usd ceiling, no new batch after a usage-limit error), run in the one local eval-runner session; the pull request carries do-not-merge until the record lands.
  • The pull request body names this repository's dogfooding step.

Out of scope

  • Writing idioms for any language.
  • The design skill's own idioms handling.

Activity

  1. added
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 13, 2026
  2. removed
    agent-readyFully specified and briefed; eligible for autonomous pickup from the frontier.
    on Nov 13, 2026
  3. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Small
    Complexity: Low
    Uncertainty: Knowable
    Flags: none
    Blocked by: none

    I understand this as: /implementation:implement should read a consumer-declared idioms file for the detected language. With no declaration, it should say so and use the neutral rules. Pull request #16086 merged with Refs:, so the issue stayed open. The implement skill and its evals already mention idioms, so part of this may be done. Nobody has yet checked the "Done when" items against what #16086 left: the skill wording, the two eval cases, the two-version compare record, and the dogfooding note. Please confirm which items remain, or close the issue if all are met.

    Automated triage. A maintainer reviews it.

  4. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    Not admitted to the issue lane: merged pull request #16086 names it with Refs: after its newest decision or refinement comment, so its work may already be done: close the issue if it is, or post a comment whose first line is **Decision recorded.** saying what is left, which sends it back through intake. Admission takes the issue once each of those is met, when intake triages it again (.github/actions/resolve-config/admission-rules.json).

  5. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    Contributor

    {"v":1,"actor_id":337595936,"title_sha256":"5866b93483c856b5a3b10dcbd50ffd9f0e09f54ced5a6b67166f4ec477e257b4","body_sha256":"88e8af13fb5cac1f7ebb3f9da6997688bdaa7acdbaec61a4c232f9909a4fbb85"}

  6. github-actions commented on Nov 14, 2026

    @github-actions
    Contributor
    No description provided.
  7. melodic-automation-lanes commented on Nov 14, 2026

    @melodic-automation-lanes
    Contributor

    Pull request #16086 merged with Refs:, so this issue stays open and nothing starts the lane on it again. work-unblock removed status: ready and agent-ready, so the hourly triage sweep sends it back to intake triage. Admission then parks it at status: confirmed for a check of what that pull request left, until a **Decision recorded.** comment says what is left or someone closes it.

  8. github-actions commented on Nov 14, 2026

    @github-actions
    Contributor

    The triage sweep dispatched intake triage for this issue (dispatch 1; 2 runs that reach intake, or 6 dispatches, hand it to a maintainer).

  9. kyle-sexton commented on Nov 16, 2026

    @kyle-sexton
    MemberAuthor

    Session note (agent, not an owner decision). Eval compare result: not measured. The implementation implement cases cannot show whether #16086 kept behavior or whether cases 45 and 46 pass, so this issue still owes a measurable compare.

    What ran (claude plugin eval, model opus, judge opus, 3 runs per case):

    • Base 6fa15a3 and head e4055f4 (feat(implementation): read the consumer's own language idioms in implement's type decisions #16086's merge commit), each converted from skills/implement/evals/evals.json with the evals plugin's convert-evals-json.py and trimmed to the same 7 cases: implement-bugfix-mode-incremental-cadence, implement-contract-change-runs-propagation-sweep, implement-feature-fifth-branch-names-domain-structure, implement-no-suppression-to-pass-lint, implement-refactor-subtraction-first-commit, and the two new cases implement-type-decision-cites-consumer-idioms-file (45) and implement-type-decision-no-idioms-declared-proceeds-on-neutral-rules (46). Cases 45 and 46 also ran on base as the detection control.
    • Both batches finished (partial: false, no run errors), and both are verdict: INVALID under run-validity.py: permission denials of Glob/Grep calls aimed outside the run's working directory (case 46 on base; cases 45 and 46 on head).
    • The underlying fault: every run in both versions and both arms stopped before the task ("the working directory is empty"). These are narration: true cases. The converted suite gives them no fixtures and only Read, Glob, Grep and Skill, so a run starts in an empty sandbox and cannot make the edits the expectations grade. No case is comparable, so this record quotes no score, and the non-inferiority margin (0.05) and the detection control are undecided.
    • Sonnet was not run: it would have repeated the same empty-workspace stop.

    Caveats that would apply even to a valid run: three runs per side is a small sample, and the judges are uncalibrated (the converted samples are unlabeled expected_output text).

    What is left:

    1. Make cases 45 and 46, and the regression cases chosen for the compare, runnable under claude plugin eval. Give each the repository its prompt describes (context.add_dirs or a scaffold_script holding tsconfig.json, src/webhooks/carrier.ts, docs/standards/README.md with the typescript row and typescript.md for 45; pyproject.toml, worker.py and an empty standards setup for 46), and grant Write and Edit, or rewrite the code-change expectations to grade the narrated plan.
    2. Rerun the base versus head compare on opus and sonnet, 3 runs per side, margin 0.05, with 45 and 46 on base as the detection control.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Work status

    confirmed

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions