Skip to content

feat(perps): distribute generated review checklists - #163

Open
abretonc7s wants to merge 5 commits into
mainfrom
feat/explicit-review-template
Open

abretonc7s wants to merge 5 commits into
mainfrom
feat/explicit-review-template

Conversation

@abretonc7s

@abretonc7s abretonc7s commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Distribute the generated Perps static review checklist for Mobile, Extension and Core.

  • This PR contains only Perps review content, its generator, parity tests and documentation.
  • It is independent of installer PR feat(installer): support explicit-only skill invocation #174. The skill can be invoked directly from the checked out source; installer explicit-only controls are a separate follow-up in feat(installer): support explicit-only skill invocation #174.
  • --analyzer-out <path> --client <client> optionally exports inlined context for an existing PR risk analyzer. It does not enable an analyzer integration.
  • Validation: the repository test suite passes, including per-client parity and install/reinstall checks.

Farmslot validation ran the Core template with Astra medium, registered all 30 child checklist rows, saved three actionable findings plus line comments, and posted the selected review to MetaMask/core#10294 after the human gate.

@aganglada aganglada left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three fixes before merge:

Comment thread test/perps-review-template.test.mjs Outdated
Comment thread tools/install Outdated
Comment thread README.md Outdated
@abretonc7s

Copy link
Copy Markdown
Contributor Author

Pre-review pass (17 Sept)

A full read of the end state of this branch before asking for review.

Checked, nothing to change:

  • No run-mode language (runMode, autonomous/interactive, --mode) in the generated skill, overlays, criteria, templates, installer or tests. The only "interactive" hits are the README's domain picker and a testID rule about interactive UI elements.
  • No control-plane mentions in the Perps skill, tools/install or the README.
  • The three requested changes are in 3b031d1 and answered inline: committed skill/template equality test, user-scope agents/openai.yaml written only for explicit-only skills and removed only when it carries the installer banner, README naming the public check and who regenerates. The client-agnostic static-perps.md the test mentions is asserted absent; the per-client set replaced it in 6e453f8.
  • Word counts: skill.md 855 (main 622: the base grew by the phase structure and the Signal-over-noise row; the repo-specific families moved out to repos/metamask-mobile.md 265, repos/metamask-extension.md 630, repos/core.md 427; 30 criteria files, largest 451 words, read on demand).

Fixed in 5003f6d: materialize-review.mjs --check against the library reported the committed files stale because the library main had moved from c991d06 to cac185c (a recipe fix; review/ unchanged). Regenerated: 5 files, 5 lines, revision stamps only, which also shows the committed output is reproducible from a clean library revision.

Not changed on purpose: nothing else was found that is a factual or text error; structural or test suggestions are out of scope for this PR. PR description refreshed to describe the small body + overlays + per-client templates.

@abretonc7s

Copy link
Copy Markdown
Contributor Author

91a4cd3: tools/install now prunes every retired installed name on sync (mms-recipe-pr-qa-review, mms-recipe-evidence, mms-recipe-fixbug, mms-recipe-quality), once before the install loop, keeping the managed-banner guard. Before this, only the QA review name was removed, so the renames in Consensys/skills#74 left the old directories installed beside the new ones on every checkout that had synced earlier; an agent could still invoke /mms-recipe-fixbug. yarn skills execs into this installer, which is why the same list in the Consensys copy did not take effect. Explicit-invocation test passes.

@aganglada aganglada left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Requesting changes for one installer bug. The per-client split (shared skill.md, overlays, on-demand criteria, byte-identical templates) is the right shape, and the three comments from the earlier review look addressed. CI is green. GitHub still shows CHANGES_REQUESTED from the review on 47362fe; this is a re-review of 91a4cd3.

Must fix

User-scope Codex default_prompt is not interpolated. write_user_codex uses a different escape than write_agents. In bash double quotes, \${out_name} writes the literal ${out_name}; project Codex skills get Use $mms-workflow for this task. Inline comment on tools/install. test/explicit-invocation.test.mjs only asserts allow_implicit_invocation: false on the user yaml, so this would not fail CI.

Should fix

Every client still receives every other client’s criteria and templates. copy_bundle_dirs copies all of references/. The install test documents that the files all ship and only the checklist names rows. A Mobile install therefore includes static-perps.extension.md, static-perps.core.md, and references/criteria/{extension,core}/. That undercuts the “1,100–1,450 words, not 6,854” goal: the SKILL body is small, but a model that glob-reads references/ still sees the other clients. Filter the bundle per --repo, or keep other-client templates out of the installed skill.

Local reviewers still get farm worker protocol. Verdict rows tell a Cursor/Claude user to write artifacts/review.md, line-comments.json, satisfy worker-terminal-contract.json, and mark complete --mark-last. The README also says a control plane worker follows a complete per-client template, which contradicts the 17 Sept pre-review note that control-plane language was gone from the README. Either keep that only in the farm templates, or say clearly that local /mms-perps-review-pr ignores those rows.

Suggestions

  • Implicit routing is gone on purpose. Description collapsed from “review my perps PR / perps review / QA static step” to “when explicitly invoked by name.” After Mobile/Extension bump @metamask/skills, agents will not auto-attach this skill. That matches disable-model-invocation, but consumers need the invocation name in AGENTS.md / farm wiring before the bump.
  • summary() uses the first body line. Families whose first line is a bullet (e.g. Charts/CTAs) get that bullet as the checklist one-liner, not the thesis paragraph. Prefer the first non-list paragraph when one exists.
  • Retired-name prune is project-only (CLAUDE_DIR / CURSOR_DIR / AGENTS_DIR). User-home copies of mms-recipe-fixbug would remain. Fine if those names were never user-scoped; worth a one-line comment or extending the loop if they were.
  • CHANGELOG Unreleased bullet has no (#163) link, unlike neighboring entries.
  • Tests are strong on merge equality, --check drift, analyzer --client, and “don’t delete hand-authored skills.” Add the default_prompt assertion; optional: assert a Mobile install does not copy references/templates/review-pr/static-perps.extension.md.

What looks good

  • Per-client templates asserted equal to skill.md + overlay as tools/install merges them; client-agnostic static-perps.md asserted absent.
  • overlay_body() so frontmatter-less overlays are not emptied by body_after_frontmatter.
  • User agents/openai.yaml written only for explicit-only skills, removed only when MANAGED_YAML_BANNER is present.
  • Cursor explicit-only as nested .mdc without description/globs, with RULE.md.mdc swap.
  • Generator refuses dirty library sources, stamps revision + sha256, --check catches stray files, --analyzer-out requires --client and inlines rules.
  • Retired-name prune before install so rename leftovers cannot sit beside the new skill.

Comment thread tools/install
echo 'interface:'
printf ' display_name: '; yaml_quoted "$name"; echo
yaml_field ' short_description' "$description"
printf ' default_prompt: '; yaml_quoted "Use \${out_name} for this task."; echo

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not match write_agents (Use \$${out_name} for this task.).

In bash double quotes, \${out_name} is the literal ${out_name}, so user-scope Codex yaml gets Use ${out_name} for this task. instead of Use $mms-personal for this task.

Use the same \$${out_name} interpolation, and assert the expanded prompt in test/explicit-invocation.test.mjs (that test currently only checks allow_implicit_invocation: false).

@abretonc7s
abretonc7s force-pushed the feat/explicit-review-template branch from 91a4cd3 to 410b87c Compare September 21, 2026 12:56
@abretonc7s abretonc7s changed the title feat(perps): share an explicit static review checklist feat(perps): distribute generated review checklists Sep 21, 2026
@abretonc7s
abretonc7s changed the base branch from main to feat/explicit-skill-invocation September 21, 2026 12:57
@abretonc7s
abretonc7s force-pushed the feat/explicit-review-template branch from 410b87c to 85f46b9 Compare September 21, 2026 13:19
@abretonc7s
abretonc7s changed the base branch from feat/explicit-skill-invocation to main September 21, 2026 14:19

@aganglada aganglada left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 190ac690. Installer work (including the Codex default_prompt bug) is out of this PR and belongs in #174.

The per-client split is the right shape. One blocking drift from 190ac690:

Comment thread domains/perps/skills/perps-review-pr/scripts/materialize-review.mjs
@abretonc7s
abretonc7s dismissed aganglada’s stale review September 22, 2026 08:12

comments addressed

aganglada
aganglada previously approved these changes Sep 22, 2026
NicolasMassart
NicolasMassart previously approved these changes Sep 22, 2026
@abretonc7s
abretonc7s enabled auto-merge (squash) September 23, 2026 01:11
abretonc7s added a commit that referenced this pull request Sep 23, 2026
Add explicit-only workflow metadata to the skill installer and emit each
supported runner's native invocation controls. Preserve repository
overlays and custom files during install and reinstall.

- Contains installer/schema changes, installation tests and
documentation.
- Perps review content is isolated in #163, which is stacked on this
branch.
- Validation: explicit-invocation tests passed on this branch; all 75
repository tests passed on the stacked Perps branch.
- The published installer is still 0.3.1. Adoption requires a new
installer release and consumer dependency updates.
Co-authored-by: Cursor <cursoragent@cursor.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants