Repository navigation
feat(performance): rank slow checks per lane and ratchet their growth - #16648
Conversation
performance:target takes slow git hooks, local check scripts and CI jobs as a target type: one candidate per check and lane, a p50 and p90 baseline per lane, the moves to rank (scope to the change, a later lane, parallel runs that report every job's failure, the tool's cache or incremental mode, a tool swap), a dependency-collision check on any upgrade or swap, and stack specifics resolved at run time. A growing check count goes to toolchain:ratchet, which now names check counts as a counter source. performance:goal records 50,90 for a check-lane goal. A new eval case, ta-check-lanes-per-stage, covers the target path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Lane run report
Failed GitHub API calls
|
|
claude-security-review reviewed Security review: PR #16648 (head c5126d1)I found no security issues. I read every hunk of all 11 changed files in the full diff. What changed
What I checked
critical-findings: 0 |
|
claude-review reviewed Code review: PR #16648, rank slow checks per lane and ratchet their growthI read all 11 changed files in full in Closing issue: none. The PR body says this was the owner's decision, so there are no acceptance criteria to check. Embedded instructions: none found in the PR context. Rules:
Ratchet: "four places" became "five places", and the ratchet still refuses durations ( Goal: Verification: the PR's validation and eval runs are claimed by the author. I could not run them here. Findings (2, both advisory, inline):
Spend: the new eval case ( Verifier dropped 0 of 2 candidates. No verifier subagent ran; I checked both candidates myself against the head files and the eval samples. critical-findings: 2 Findings not posted inline (their lines are not in the diff):
Fix: anchor the stage to the second cell: I traced this pattern by hand against the existing samples, without running it: both Findings not posted inline (their path, line or side is malformed):
How it goes wrong: on a check candidate like Fix: pick one cell format for check candidates and say how it relates to the scaling-arm meaning. For example: "Fill Growing state with |
No related issue: owner decision to extend the existing performance and ratchet skills for check speed rather than add a skill.
Summary
Slow checks (git hooks, local check scripts, CI jobs) become a target type of the existing performance chain instead of a new skill.
performance:targetranks them per check and lane,performance:goalrecords a p50 and p90 baseline per lane, andtoolchain:ratchetholds a growing check count as a counter.Fix
plugins/performance/skills/target/SKILL.md: a "Slow checks" input row and a "Slow checks, hooks and CI" section. It inventories each check and its lane from the repository's own hook config, scripts and workflows, reads the repository's placement record and budgets first (this repository's worked case isdocs/contracts/check-placement/README.md, linked, no numbers copied), names the lane in the Candidate cell so the table's columns stay as they are, and ranks five moves: scope to the change, move to a later lane (never drop; CI stays the hard stop), parallel runs that report every job's failure, the tool's cache or incremental mode, and a tool swap. An upgrade or swap carries a dependency-collision check. Stack specifics resolve at run time through/discovery:researchand the repository's conventions, with no per-stack examples or tool links. A zero-input pass is a finding, and a growing count goes to/toolchain:ratchet. The description gains one trigger phrase, and## Nextnames the ratchet.plugins/performance/skills/goal/SKILL.md: a check-lane goal records50,90through the existing percentile list (the 1/(1-p) floor already covers p90). The p50/p95 default is unchanged.plugins/toolchain/skills/ratchet/SKILL.md:/performance:targeton slow checks is a fifth counter source (checks per hook stage or CI job, files a check scans, test suites a change selects). Durations stay refused.plugins/performance/evals/ta-check-lanes-per-stage/: a new eval case with a scaffold, three regex graders and pass/fail samples..changes/performance/…,.changes/toolchain/…: minor-bump fragments.docs/skill-cheat-sheet.md: regenerated for the new## Nextentry.Verification
scripts/validate-plugins.sh --only performanceand--only toolchain: both end "All plugin manifests and the catalog validated." The first run flagged the stale cheat sheet;node scripts/generate-cheatsheet.mjsfixed it.plugins/evals/skills/validate/scripts/validate-cases.py plugins/performance/evals: exit 0, no FAIL. The new case's samples pass offline, and its only WARNs come from other cases.scripts/check-vocabulary.pyover the three edited skill bodies: no findings.claude plugin evalon the new case (ta-check-lanes-per-stage, 1 run per arm, plugin copied outside git because the eval refuses a plugin inside a repository with many linked worktrees): with plugin 1.00 (3/3 graders), without 0.00, delta 1.00.claude plugin evalon the existingta-ranks-by-evidence-tier(1 run, no ablation) as a regression check: 1.00 (4/4).Related
docs/contracts/check-placement/README.mdthrough fix(scripts): fail checks that checked nothing and record the hook timings #16645; the target section links that contract and copies none of its numbers.Left out
plugins/toolchain/skills/ratchet/SKILL.mdlinks ESLint, PHPStan and Oxlint docs in sections 2 and "Replacing a lint tool", which the no-per-stack-tool-links decision would remove; this PR leaves them unchanged.🤖 Generated with Claude Code