fix(evaluating-skill-quality): add Dimension-5 exception for sequential-pipeline skills

[tvna]

Duplicate-PR-waiver: PR #1632 only Refs #1662 (it discovered the Dimension-5 gap this issue tracks and explicitly declined to fix it there -- its own body states "Filed as a new, separate issue (#1662) for a rubric-level fix... not attempted inside this unrelated PR", per the repository owner's own direction to pursue the rubric fix through its own measured process). This PR is that separate fix; no overlapping file changes with #1632.

Summary

Add one narrow, loophole-resistant exception bullet to rubric.md's
Dimension 5 (Progressive disclosure) for a skill that is already
cohesion-confirmed sequential/functional and whose every-use content
demonstrably exceeds BODY_MAX_LINES, so such a skill has at least one
satisfiable Dimension-5 passing configuration -- gated through
scorer-gated-skill-edits's own held-out measured process, per issue
#1662's own Constraints.

Facts

• scorer-gated-skill-edits's held-out gate ran to completion: one new selection fixture (sequential-pipeline-body-cap-exception-selection.yaml) moved selection mean 0.666667 -> 1.000000, KEEP. Full record: evals/evaluating-skill-quality/results/2026-09-02-issue-1662-sequential-pipeline-body-cap-exception/manifest.json.
• A Transfer check against the adjacent, pre-existing dispatch-self-guard-boundary-selection.yaml fixture (issue fix(evaluating-skill-quality): dimension 5 (Progressive disclosure) NOT-MATURE -- clarify boundary for the dispatch self-guard file #1346's own self-guard boundary) confirmed no regression (score 1.0) -- the new bullet did not loosen the pre-existing self-guard exemption.
• Step 8 (mandatory refactor + independent adversarial review) ran over the full accumulated diff, in two rounds. Round 1 (executing-a-branch-plan's own Step 8, commit d3c04b54): one refactor wording fix, plus 2 Medium + 2 Low adversarial findings, all confirmed and fixed -- (1) the run record's commit field pointed at a third, later commit instead of matching before_commit per this corpus's own convention; (2) condition 2 of the new exemption bullet lacked condition 1's own explicit anti-self-assertion guard -- fixed by adding the same guard and grounding it in dimension 2's own named tests; (3) undefined "padding cut" terminology, closed by the same fix; (4) ambiguous bare "condition 1"/"condition 2" labels colliding with the adjacent self-guard bullet's own numbering, disambiguated. Round 2 (drafting-a-pr-to-merge's own Step 8, an isolated evaluating-skill-quality specialist dispatch against the rubric.md diff, commit ef6d5f0e): found the new bullet's own bold title named only "sequential pipeline," undershooting condition (1)'s actual, intentionally broader disjunctive test ("single-outcome sequential or functional cohesion," reusing dimension 4's own clearing bucket verbatim). An independent counterfactual check against dimension 4's own decision rule (an independently-triggerable branch already routes a target to a procedural/temporal/logical split finding, never a functional/sequential clear) confirmed this was not an exploitable gap in condition (1) itself -- only a title/rationale wording-precision gap -- fixed by broadening the title and grounding it in that same dimension-4 rule. None of these four rounds' fixes changed the irreducible floor string the fixture's own assertion scores on, so the already-banked KEEP verdict was not re-gated -- disclosed in the run record's own known_gaps instead.
• gitapex_check_skill_shape.py: 70/70 (post-fix, both rounds). Full local pytest suite: passing (post-fix). origin/main drift check: clean, no conflict.
• Disclosed limitation: the confirmed eval runner (evals/scripts/gitapex_run_eval_suite.py) could not execute live in this session's environment -- every claude --bare --tools "" -p invocation fails with an empty-stderr auth error matching issue fix(waza-eval-gate): live eval runs fail with "model CLI exited 1" and empty stderr, repo-wide #1304's already-tracked, repo-wide infrastructure defect, independently reconfirmed by this same PR's own eval-gate CI run failing identically on every push. Scoring used an isolated Agent-tool subagent dispatch substitute instead (reasoning over pre-pasted skill text, no further tool access), fully disclosed in the run record's own known_gaps.

Assumptions

Speculation: one new selection fixture is sufficient to gate this edit
(issue #1346's own precedent for the same rubric section used exactly
one) -- confirmed sufficient in practice by the completed gate run above.

Risk / blast radius

Scoped to skills/evaluating-skill-quality/references/rubric.md's
Dimension 5 section, its held-out eval corpus
(evals/evaluating-skill-quality/), and that skill's own
metadata/gitapex.yaml decision log. No change to any other skill's
SKILL.md or references.

Rollback

Revert this PR's merge commit; no schema, dependency, or runtime change
to roll back beyond the rubric text and eval-corpus files themselves.

Verification

Criterion	Interpretation	Planned ops	Proof method	Residual risk	Result
A cohesion-confirmed, over-cap sequential-pipeline orchestrator has at least one satisfiable Dimension-5 passing configuration	Insert one new bullet into Dimension 5 immediately after the existing self-guard exemption bullet (rubric.md, after line 1663), gated on two conditions neither self-assertable by the reviewed target: (1) the cohesion check has already graded the target sequential/functional at its own Procedure step 2 -- reused, never re-derived from an "inherently sequential" claim in the target's own prose; (2) the target's every-use content demonstrably exceeds BODY_MAX_LINES, interlocked with dimension 2's own padding check so a body merely declining to inline does not qualify. Append matching clauses to the existing Fail and Pass bullets.	Edit skills/evaluating-skill-quality/references/rubric.md's Dimension 5 section only	scorer-gated-skill-edits's own measured, held-out-gated edit process (checkable scorer + held-out split) -- the same process issue #1346's own ACM used for the prior Dimension-5 fix; gitapex_check_skill_shape.py full run	The exception could be read as a loophole any long-procedure skill could claim; mitigated by gating on an independently-produced cohesion finding and a deterministic body-cap check rather than any self-reported claim	Done. Gate KEEP (0.666667->1.000000); gitapex_check_skill_shape.py 70/70; condition-2 self-assertion loophole and title/condition-1 scope mismatch both closed across Step 8's two adversarial-review rounds
The new exception does not silently reverse issue #1346's own self-guard boundary or its own qualifying condition	No change to the self-guard exemption bullet's own text; the two exemptions are additive, independently gated	Diff review confirming the self-guard bullet (rubric.md, the bullet immediately before the insertion point) is unchanged	Manual read-through comparing before/after rubric.md	If the two exemptions' conditions overlap in an unforeseen way, that overlap itself becomes a new Dimension-5 finding	Done. Self-guard bullet text unchanged (diff-confirmed); Transfer check against its own fixture scored 1.0, no regression
executing-a-branch-plan's own current 3-reference layout is re-graded once the rubric change lands	Re-run evaluating-skill-quality's Procedure (isolated dispatch) against the current executing-a-branch-plan content	N/A (verification step, no code change to that skill in this issue)	Isolated subagent dispatch re-review; verdict recorded in that skill's own metadata/gitapex.yaml decision log	The motivating analysis found a separate, real residual defect this rubric fix alone does not resolve (events-and-review-gate.md every-run/failure-only content mixing) -- tracked separately under PR #1632 / issue #1648, not this PR's own scope	Not done in this PR -- deferred as its own follow-up; this row is a verification step against a different skill (executing-a-branch-plan), not a code change this PR makes, and re-grading it is not required to close issue #1662's own Constraints

Checklist

• Tests pass locally
• Docs updated if behavior changed
• Issue number cited in every commit
• If this PR adds/modifies a skills/*/SKILL.md, a docs/superpowers/specs/*.md design doc, a security-relevant skill, or a deterministic checker script (skills/*/scripts/*.py, evals/scripts/*.py, .github/scripts/*.py), a ## Skill audit evidence section discloses the required verdicts/waivers (see .github/scripts/gitapex_gate_skill_audit_disclosure.py)
• If this PR adds a new Kept-edit-log entry to any evals/*/split.md, that entry discloses a Transfer check line (see .github/scripts/gitapex_gate_transfer_check_disclosure.py)
• If this PR adds or increases a skills/*/SKILL.md's Stop-boundary bullets or named dispatch branches, evals//tasks/*.yaml gained at least as many new fixtures (see .github/scripts/gitapex_gate_skill_branch_fixture_coverage.py)

Independent review verdict

• Verdict: CLEAN
• Verified commit: ef6d5f0

Outer layer (GitHub-native reviewer). No confirmed installation of Anthropic's "Claude Code Review" GitHub App on this repository; GitHub Copilot's copilot-pull-request-reviewer[bot] was requested via github:request_copilot_review (twice, once per head commit). Per the repository owner's explicit direction this round, waiting further on this layer was skipped rather than run to the usual 30-minute unreachable threshold. This layer did not run to completion; its absence is disclosed here, not silently treated as a clean pass.

Inner layer (reviewing-an-artifact, low effort). Ran against this PR's full diff (base d8f32cb9, head ef6d5f0e):

• Step 0 mixed-target handling: skills/evaluating-skill-quality/references/rubric.md (a references/ file of a SKILL.md-owning skill) was specialist-deferred to an isolated evaluating-skill-quality dispatch, per Step 0's own eight-way deferral list. The remainder (the new eval fixture, split.json/split.md, eval-status.md, metadata/gitapex.yaml, the new evals/.../results/2026-09-02-issue-1662-.../ run-record directory, and the regenerated docs/skill-eval-status.md) classified safe under Step 1 (log additions, a new test fixture, doc regeneration; no security-tier signal anywhere) -- Steps 2-5 skipped for that remainder, recorded as the skip it is.
• The deferred evaluating-skill-quality dispatch (isolated, no memory of authoring this change) found one confirmed finding: the new Dimension-5 exemption bullet's own bold title named only "sequential pipeline," undershooting its own condition (1), which is intentionally broader ("single-outcome sequential or functional cohesion"). Independently re-verified in this thread (not accepted at the dispatch's own say-so): read rubric.md:327-346 directly and confirmed dimension 4's own cohesion-check decision rule already routes any skill with an independently-triggerable branch to a procedural/temporal/logical split finding rather than a functional/sequential clear -- so condition (1) itself carries no exploitable gap, only the title undersold its own scope. Fixed in commit ef6d5f0e (title broadened, grounded in that same dimension-4 rule); gitapex_check_skill_shape.py re-run 70/70 and the targeted pytest subset (326 tests) re-run clean after the fix, before this verdict was recorded.

Zero confirmed findings remain outstanding against the current head, and zero unconfirmed-concern findings were raised. Per the four-outcomes rule: outer layer skipped per owner instruction (disclosed above), inner layer zero confirmed findings on the current head -> continue to step 9.

Merge gate: independent review

This PR is also subject to the independent-review-pending required
status check (see .github/workflows/independent-review-pending.yml /
.github/scripts/gitapex_gate_independent_review_pending.py). It stays
pending/failing until a ## Independent review verdict section naming
this PR's current head commit is recorded in this body --
drafting-a-pr-to-merge's own Step 8 records it once its independent
review completes. There is nothing for you to do here now: do not
pre-fill this section yourself, and do not remove this note.

Related Issue

Closes #1662

Execution log

• PlanApproved{run_id: 379c173a}
• TaskStarted{run_id: 379c173a, task_id: A}
• TaskCompleted{run_id: 379c173a, task_id: A, commit_sha: 13d7a8be0a825e49303e0372b2a508f3604425b2}
• Step 8 round 1 (executing-a-branch-plan's own refactor + independent adversarial review) complete: 2 Medium + 2 Low findings confirmed and fixed, commit d3c04b5419db91eb5e3e4520ad55ade423980f1c.
• Step 8 round 2 (drafting-a-pr-to-merge's own inner-layer review) complete: 1 confirmed finding fixed, commit ef6d5f0eb6070b9d18c9385fb98a1779b74489e6. origin/main drift check clean.

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: cd72eb3b-0910-479e-b0b3-26a46545e4c3

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.66%. Comparing base (d8f32cb) to head (ef6d5f0).
⚠️ Report is 11 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1672   +/-   ##
=======================================
  Coverage   99.66%   99.66%           
=======================================
  Files         149      149           
  Lines       24479    24479           
  Branches     2960     2960           
=======================================
  Hits        24397    24397           
  Misses         82       82           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tvna]

eval-gate check (CI) -- not this PR's failure, no fix exists yet.

error: model CLI exited 1: (empty stderr, ~2s after the live claude CLI invocation starts) is a known, repo-wide infrastructure defect tracked in #1304, not specific to this PR's fixture content. Confirmed by comparing this run (33614022384) against an unrelated PR's eval-gate run on a different branch/skill from the same day (33591072524, claude/drafting-a-skill-implementation-lkdypz): identical empty-stderr failure signature. #1304's own Facts note every eval-gate run with a live invocation has failed the same way, root cause unconfirmed (candidates: ANTHROPIC_API_KEY misconfig/expiry, a @anthropic-ai/claude-code@2.1.240 behavior change, or an API-side change) -- no fix PR exists yet.

eval-gate is not a required status check (.github/rulesets/main.json does not list it), so it does not block this PR's merge. Re-ran the failed job once (rerun_failed_jobs) per the drive-to-green re-run allowance; if it fails identically again, that confirms #1304's own root cause rather than a fixable defect in this PR.

This PR's actual scorer-gated-skill-edits gate measurement is being run separately via isolated Agent dispatch (this session's own substitute for the live claude CLI runner, since it hits the identical #1304 auth/CLI failure locally too) -- results and the Kept-edit-log entry follow once that completes.

---

Generated by Claude Code