[findscripter]

Problem and resulting behavior

A skill collection can pass its own index/build checks while a consumer omits intended skills or exposes archived and template entries. Comparing total counts can hide both kinds of error.

This adds skill-distribution-audit for maintainers investigating release/discovery mismatches. It reconciles the intended public source set, the applicable distribution manifest, and actual consumer discovery output at a specified revision and consumer version.

What this provides

• Explicit public-scope and identity checks, standard YAML/schema compatibility checks, and consumer-specific filtering semantics.
• A dependency-free Node.js helper that reports missing, unexpected, and duplicate identities separately. It reads two captured JSON arrays and never launches an installer.
• A small worked example showing that equal totals can still contain an omission and a template leak.
• Clear limits: discovery coverage does not establish live loading, helper execution, credentials, or service availability.

Concrete motivating case

In a collection I maintain, skills@1.7.4 discovered 1,076 entries against an intended active set of 1,093: 38 active skills were missing, while 20 archived skills and one template were included. The net difference of 17 hid the two directions of failure. Fixes addressed standard YAML compatibility and that consumer version's internal-entry filtering; the recorded follow-up discovery matched 1,093 unique intended identities.

• Fixed historical receipt, including source revision, before/after counts, and verification scope
• One malformed frontmatter example at the original source revision

These are distribution observations, not a benchmark showing an improvement over baseline Copilot. This update replays existing captured discovery data; it does not claim a fresh installer run. Consumer-specific fields are not presented as universal Copilot requirements.

Validation

• node --test skills/skill-distribution-audit/scripts/compare-discovery.test.mjs: 7 tests passed, including equal-count mismatches, duplicate identities, input preservation, error exit codes, and linked installation directories.
• Historical capture replay: 38 missing / 21 unexpected before the repair; an exact unique match after it.
• Independent skill-guided audit of an isolated synthetic collection identified an omission, template leakage, and duplicate output without modifying the source inputs. This was an agent forward test, not a Copilot A/B evaluation.
• npm run skill:validate: all 449 skills passed.
• Vally lint: spec-compliance and valid-refs both passed.
• npm run build, skill-creator validation, required line-ending normalization, and git diff --check passed.

Affiliation: None

[github-actions]

🔒 PR Risk Scan Results

Scanned 4 changed file(s).

Severity	Count
🔴 High	0
🟠 Medium	0
ℹ️ Info	2

Severity	Rule	File	Line	Match
ℹ️	skill-script-touched	skills/skill-distribution-audit/scripts/compare-discovery.mjs	1	skills/skill-distribution-audit/scripts/compare-discovery.mjs
ℹ️	skill-script-touched	skills/skill-distribution-audit/scripts/compare-discovery.test.mjs	1	skills/skill-distribution-audit/scripts/compare-discovery.test.mjs

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

[github-actions]

🚦 Submission status: 💬 Review in progress

Risk tier: merge-risk:high — Privileged execution, automation, or review-policy change
Required to merge: passing submission-gate checks plus 1 approval from reviewers with write access, including a maintainer with admin or maintain permission.

Why this tier

• skills/skill-distribution-audit/scripts/compare-discovery.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
• skills/skill-distribution-audit/scripts/compare-discovery.test.mjs is a high-risk path (automation, scripts, MCP config, hooks, or review policy)
• Spawns processes or evaluates code in skills/skill-distribution-audit/scripts/compare-discovery.test.mjs

Automated checks

Check	Status	Details
Line endings	✅ Passed	Passed · logs
Spelling	✅ Passed	Passed · logs
Generated README consistency	✅ Passed	Passed · logs
Skill validation	✅ Passed	Passed · logs
Skill lint (vally)	✅ Passed	Passed · logs
Risk scan	✅ Passed	Passed · logs
Contributor reputation	✅ Passed	Passed · logs
Duplicate resource scan	✅ Passed	Passed · logs
PR quality signal	⏭️ Skipped	Skipped by its workflow · logs

Review

• Approvals: 0/1
• Assigned reviewer: not assigned yet — comment /request-review to ask for one
• Review target date: not set
• Still needed: 1 more approval(s); an approval from a maintainer with admin or maintain permission
• The core-maintainers pool is not staffed yet; an approver with admin or maintain permission is required instead.

Commands

Command	Who	What it does
/rerun-checks	PR author, maintainers	Re-runs failed or incomplete checks and re-evaluates this gate
/request-review	PR author, maintainers	Asks the review rotation to assign a reviewer (adds needs-reviewer)

_{Updated for 11a11ef · gate run · This comment is maintained automatically — see submission gate docs.}

[github-actions]

🟡 Contributor Reputation Check: MEDIUM risk

Check	Risk
Profile	MEDIUM
Credential audit	NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

[github-actions]

🔍 Vally Lint Results

✅ All checks passed

Scope	Checked
Skills	1
Agents	0
Total	1

Severity	Count
❌ Errors	0
⚠️ Warnings	0
ℹ️ Advisories	0

Summary

Level	Finding
ℹ️	✅ skill-distribution-audit (2/2 checks passed)
ℹ️	✓ [spec-compliance] All 1 skill(s) are spec-compliant.
ℹ️	✓ spec-compliance: All spec checks passed.
ℹ️	✓ [valid-refs] All file references across 1 skill(s) are valid.
ℹ️	✓ valid-refs: All file references resolve to existing files within the skill directory.
ℹ️	1 skill(s) linted, 1 passed

Full linter output

### Linting skills/skill-distribution-audit
✅ skill-distribution-audit (2/2 checks passed)
    ✓ [spec-compliance] All 1 skill(s) are spec-compliant.
        ✓ spec-compliance: All spec checks passed.
    ✓ [valid-refs] All file references across 1 skill(s) are valid.
        ✓ valid-refs: All file references resolve to existing files within the skill directory.

1 skill(s) linted, 1 passed

[aaronpowell]

I'm struggling to understand how this uplifts Copilot's capabilities.

Copilot is already very capable of discovering agent capabilities (such as skill, mcp servers, etc.) since the tooling will load those.

[findscripter]

Thanks @aaronpowell, that's a fair point. I've narrowed the contribution in 11a11ef to a skill distribution audit for collection maintainers.

The concrete motivating case was a collection with 1,093 intended active skills where the installer listed 1,076 entries: 38 active skills were missing, while 20 archived skills and one template were included. Comparing totals hid the omissions and leaks. The historical receipt records the source revision, consumer version, and before/after results.

The updated skill reconciles the intended public source set, the applicable manifest, and actual discovery output, with a deterministic helper for missing, unexpected, and duplicate identities. I removed the generic inventory, onboarding, and extension-selection guidance. Seven helper tests pass, along with validation, lint, and build checks.

The intended value is a repeatable release-validation workflow. Discovery coverage is explicitly separate from runtime loading or successful execution, and I haven't claimed a measured improvement over baseline Copilot.