Repository navigation
ci(skills): validate agent skill integrity in branch checks #4921
Description
User Story
As a first-time OpenShell contributor onboarding with the repository-provided contributor skills, I want skill metadata, inventories, and repository links to be validated automatically so that contributor workflows remain discoverable and do not direct contributors or their tooling to invalid paths.
Problem Statement
OpenShell's sync-agent-infra skill defines a comprehensive manual consistency check, but the repository has no deterministic branch check for skill frontmatter, visibility metadata, inventory drift, or local links.
This has already allowed a broken repository link into .agents/skills/create-github-pr/SKILL.md. The link target .github/PULL_REQUEST_TEMPLATE.md is interpreted relative to the skill directory, where that path does not exist, instead of resolving to the repository's actual PR template.
Impact / Why This Matters
Invalid contributor instructions can make a contributor or coding harness miss required project templates, use stale workflow names, or expose internal skills through ordinary public discovery. Manual review becomes increasingly fragile as the public and internal skill collections grow.
The current workaround is to execute the full sync-agent-infra checklist manually and notice each mismatch during review. That is useful for semantic consistency, but it does not reliably prevent deterministic drift.
Proposed Design
Provide an offline repository validation command for the public and internal skill collections. The command should report actionable file and line information and be included in scoped branch checks when contributor skills, skill inventories, or related guidance change.
The validation surface should cover:
- required
SKILL.mdfrontmatter and unique skill names; - internal visibility metadata for contributor skills;
- consistency between discovered skills and the inventories in
CONTRIBUTING.md; - repository-local Markdown links and referenced files;
- the intended public/internal discovery boundary.
The existing semantic sync-agent-infra review should remain in place; the automated check would cover only deterministic invariants.
Suggested UX
mise run agent-skills:checkExample failure:
.agents/skills/create-github-pr/SKILL.md:167: local link does not exist: .github/PULL_REQUEST_TEMPLATE.md
Acceptance Criteria
- Missing or malformed skill frontmatter fails validation with a file and line reference.
- Duplicate skill names fail validation.
- Contributor skills without the required internal metadata fail validation.
- Inventory drift between skill files and
CONTRIBUTING.mdis reported. - Broken repository-local links and referenced files are reported.
- Public skill discovery does not expose internal contributor skills.
- Unit tests cover valid and invalid fixtures without network access.
- Relevant branch checks run the validator.
- The existing broken PR-template link is corrected.
Alternatives Considered
Markdown linting already checks formatting, but it does not validate skill metadata, discovery boundaries, inventory consistency, or whether repository-local targets exist. Running npx skills add . --list manually confirms public discovery, but it requires an external tool and does not validate internal skills or local references. Keeping only the manual sync-agent-infra checklist would leave deterministic errors unenforced.
Repository Evidence
- Verified against
mainatc4688aa8715ea75d6c39fe468361dbbc16c691fa. - The documented discovery command currently lists the expected four public skills.
- No existing skill-integrity validation was found in
tasks/, repository tests, or branch workflows. - No matching open or closed issue or pull request was found for skill metadata, inventory, discovery, or local-link validation.
If a maintainer accepts this scope, I would like to implement it.
Activity