Repository navigation
ci(skills): ratchet stack-specific code fences in skill bodies and amend ADR 0101 - #16649
kyle-sexton wants to merge 2 commits into
Conversation
…end ADR 0101 Shipped skills carry no ecosystem-specific examples. ADR 0101's Source citation row still kept any documentation link, so it is amended in place: a link to one ecosystem's tool documentation, or an example written for one stack, is no longer a sanctioned citation. scripts/skill-stack-fences.py counts, per file, the code fences in tracked plugins/*/skills/**/*.md and plugins/*/agents/**/*.md whose tag is not a shell, data or prose format, diagram, pre-compute block, or the sql and regex skills run against their own data. Eval, test, fixture and vendor directories are out of scope. A per-file skill-stack-fences row in .performance/ratchets.json freezes today's 174 fences in 39 files; the existing ratchet step in check-plugins checks it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3f3b2fcd75
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Lane run report
Failed GitHub API calls
|
|
claude-review reviewed claude-review: incremental review of What changed: the counter now strips container prefixes (any mix of Finding (1, inline): claude-code-plugins/scripts/skill-stack-fences.py Lines 91 to 99 in fc3ee2f What I checked:
Not verified by me: the PR body says 12 of 12 tests pass and the repository count stays at 174 fences in 39 files. Those are the author's claims; I have no shell, so I ran nothing. Settled thread: the old-line-60 thread was answered by the owner with this fix. I raise nothing from it again. The finding above is a different problem, which the fix itself introduced. Closing issue: none ( Spend: three more short unittest cases on each CI run. That is negligible and not a hold. Verifier dropped 0 of 1 candidates. I checked the candidate myself by tracing it by hand; no separate verifier subagent ran. |
|
claude-security-review reviewed Security review: PR #16649 (incremental since 3f3b2fc), no security issues foundScope. I read every hunk of the 2 files changed since What changed
Checks
Earlier posts: the prior summary comment (#16649 (comment)) reported 0 findings, so nothing carries forward. critical-findings: 0 |
…fixes The opening-fence pattern allowed only whitespace before the delimiter, so a fence inside a blockquote (`> ```python`) or right after a list marker (`- ```python`, `1. ```python`) was missed, and the missed opener's closer then read as an untagged opener that hid the next real block. Any mix of `>` markers, list markers and their indentation now comes off before an opener or a closer is matched. Three regression cases cover the blockquote form, the list-marker form and the hidden next block. Today's count is unchanged: 174 fences in 39 files, so the ceilings stand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| CONTAINER_PREFIX = r"(?:\s*(?:>|[-*+](?=\s)|\d{1,9}[.)](?=\s)))*\s*" | ||
| OPENING = re.compile(rf"^{CONTAINER_PREFIX}(`{{3,}}|~{{3,}})(.*)$") |
There was a problem hiding this comment.
Closers now strip container prefixes the open fence never had, so a top-level fence can close early and hide every fence after it.
OPENING is used for openers and closers alike (count_fences, lines 91-99). So inside an open top-level fence, a content line such as `> ```` now counts as its closer. Under CommonMark it does not: a closing fence may have only indentation before it. A blockquote marker on that line belongs to a container the fence is not inside, so the line is code content.
Reproduce with a skill body that shows a blockquoted code block inside a markdown example, then has a real stack fence:
```markdown
> ```python
> x = 1
> ```
```
```python
y
```
I traced this by hand; I did not run it:
- line 1 opens a
markdownfence (excluded tag). - `> ```` on line 4 now matches with an empty info string and closes it.
- the real closer on line 5 then opens an untagged fence.
- that fence swallows the
pythonopener on line 7 (its info string is not empty, so it is not a closer) and closes on line 9.
count_fences returns 0 where CommonMark gives 1. On the base head this input counted 1, because ^\s* did not match > ````. So this push introduced the under-count. It is the same miss as the one test_list_item_opener_does_not_hide_the_next_block` guards against, but in the other direction. Any new fence later in that file passes the ratchet uncounted.
Fix: strip container prefixes only when matching an opener, and keep the opener's container. For example, record how many > markers the opener had, and accept a closer only when its prefix has the same > depth and no list marker. Add the input above as a regression case.
claude-lane: claude-review
Refs: #16634
No related issue: owner decision recorded in docs/plugin-philosophy.md (no per-stack examples in shipped skills)
Summary
Shipped skills carry no ecosystem-specific examples: no code sample written for one stack, and no link to one tool's documentation. This PR makes two follow-ups to that rule. ADR 0101's Source citation row still kept any documentation link, so it is amended. A count ratchet stops new stack-specific code fences from landing in skill and agent bodies while the existing ones are cut.
Fix
docs/adr/0101-...md: the status line, the Source citation row and a dated## Amendment (2026-11-16)section. The amendment states the rule and its reason, points at the plugin philosophy's "Hardcoded consumer specifics", and names the ratchet. This amends the record in place, following the repository's own convention: ADRs 0018, 0028 and 0075 each carry anaccepted; amended <date>status line with a dated amendment section. That convention wins over/architecture:record-decision's default of a new record.scripts/skill-stack-fences.py: a per-file counter. It prints<path> fences=<n>for each trackedplugins/*/skills/**/*.mdandplugins/*/agents/**/*.mdfile holding a counted fence, and exits 2 when git fails or no file is in scope, so a broken scope can't read as 0.scripts/test_skill_stack_fences.pyandscripts/skill-stack-fences.test.sh(thepython_probewrapper) are shaped like thejs-complexity-sharecounter's. Twelve cases cover counted and excluded tags, tags with attributes, list-indented and tilde fences, fences behind blockquote and list-marker prefixes (and the next block a missed list-item opener would hide), fences nested inside another fence, closing-fence length, scope, sorted output, and exit 2 both with no file in scope and outside a repository..performance/ratchets.json: a new per-fileskill-stack-fencesrow with 39 file ceilings and 174 fences in total, recorded byratchet.py add --per-file --break-glass off. The existingCheck performance counter ceilingsstep incheck-pluginsrunsratchet.py check --file .performance/ratchets.json, so no workflow edit is needed.No plugin is touched, so no plugin needs a version bump or changelog fragment.
The counting rule
A fence counts when it opens a code block in a skill or agent body and the first word of its info string (lowercased,
{,.and}stripped) is not in the excluded set. Every other tag counts, so a new language (rust,kotlin) is counted without an edit. Untagged fences do not count. A fence nested inside another fence is content and does not count.plugins/machine-health/skills/audit/reference/windows/check-catalog.mdholds 25powershellchecks the audit runs.!plugins/knowledge/skills/video-digest/SKILL.md.plugins/debugging/skills/analyze-profile/reference/formats/cpuprofile.md:40-42is a query the skill runs with--param.plugins/docs-hygiene/skills/rename-references/context/patterns.md:43-45is the pattern the sweep uses.Out of scope by directory:
evals,tests,test,__tests__,fixtures,testdata,vendor. These hold eval inputs, test data, and upstream copies a sync replaces, such asplugins/playwright/skills/playwright/vendor/, which would otherwise fail CI on a sync PR. This scope followsscripts/js-complexity-share.py.Basis for per-file over one total:
/toolchain:ratchet"Per-file counters". A count that files own is held per file, so cutting a fence in one file cannot pay for a new one elsewhere.Known imprecision: some counted
jsfences are code a skill executes, such as the course-digest adapters, the playwright references and the harness-ops extraction. The tag cannot tell those apart, so they count. A declared-ecosystem surface or executed adapter that needs a new fence raises its file's ceiling in the PR that adds it, with the reason in the body.Today's count is 174 fences in 39 files at f552369. #16634 reported 179 in 51 files. The figures differ because of scope: #16634's count also took in
plugins/*/reference/**, while this one covers skills and agents only, drops the eval, test, fixture and vendor directories, and excludessqlandregex.Verification
python3 scripts/test_skill_stack_fences.py: written first and failed (7 of 9 cases, counter absent), then 9 of 9 pass. The review fix for container prefixes added three cases, which failed first (3 of 12) and now pass: 12 of 12.bash scripts/skill-stack-fences.test.sh: 12 pass. The count is unchanged after that fix: 174 fences in 39 files.ratchet.py checkon a file holding only the new row:OK skill-stack-fences: every file at or below its ceiling (fences=174 total). With apythonfence appended to an uncounted skill file it reportedABOVE ... fences=1 > ceiling 0and exited 1. With the file restored it exited 0. The fullratchets.jsonwas not run locally because the hook-census rows need strace, so CI runs it.scripts/check-affected.sh --no-fetch: 16 passed, 3 skipped over 5 changed files.scripts/run-ruff.sh checkandformat --check; shellcheck;check-fixture-git-isolation.sh --check;check-test-tmp-cleanup.sh --check;check-silent-skips.sh;check-purged-em-dashes.sh;check-vocabulary.py;check-adr-numbers.sh --check;check-discriminating-test-skips.sh;check-line-citations.py;check-lane-coverage.sh --check;check-cant-fail-scan.sh origin/main; and the commit-time comment lint.Related
docs/plugin-philosophy.md. This PR is the ADR 0101 amendment its Related section leaves to a follow-up. CI does not depend on docs(philosophy): ship no ecosystem-specific examples in skills #16634 merging first: the#hardcoded-consumer-specificsanchor the amendment links to is already on main (docs/plugin-philosophy.md:124), and the two PRs touch no file in common.Not in this PR
plugins/*/reference/**files. They are model-read under ADR 0101 and docs(philosophy): ship no ecosystem-specific examples in skills #16634 proposed counting them; this ratchet covers skill and agent bodies only.scripts/suite-seconds.txtrow for the new suite. The rows are measured from CI runs, and until one is added the lane planner sizes an unlisted suite at the mean.🤖 Generated with Claude Code