Skip to content

feat(drafting-a-skill): add blank-page skill-authoring skill - #1303

Merged
tvna merged 14 commits into
mainfrom
claude/pr-1194-prep-egzz6o
Aug 26, 2026
Merged

feat(drafting-a-skill): add blank-page skill-authoring skill#1303
tvna merged 14 commits into
mainfrom
claude/pr-1194-prep-egzz6o

Conversation

@tvna

@tvna tvna commented Aug 24, 2026

Copy link
Copy Markdown
Owner

Summary

Adds skills/drafting-a-skill/ -- a gitapex-native, blank-page skill-authoring skill succeeding obra/superpowers' writing-skills and Anthropic's skill-creator -- plus a new CI drift gate keeping it in sync with evaluating-skill-quality's own Contract discipline section, and routing updates in two consumer skills.

Facts

  • skills/drafting-a-skill/SKILL.md (plus 6 references/ files and metadata/gitapex.yaml) is a new 9-Step, Design-by-Contract-structured skill: gates on Mechanism fit (Step 2) before drafting, elicits the 4 human-only metadata axes (Step 3), drafts a Precondition/Steps/Postcondition contract (Step 4), runs advisory cohesion (Step 5) and domain-gap (Step 7) self-checks explicitly deferring authority to evaluating-skill-quality, checks collision against every skill in the actual invocation-time inventory (Step 6), runs this repository's own deterministic checkers (Step 8), and hands off to evaluating-skill-quality/battle-testing-a-skill (Step 9) -- never performing either review itself.
  • A fresh-context consistency audit, run before any implementation file existed (per the issue's own Constraints), found and the first commit corrected 3 real design defects: Steps 5/7 were originally written as authoritative checks, duplicating ownership evaluating-skill-quality's own rubric already claims exclusively; skillDependencies.requires was [] despite Step 8 mandatorily invoking that skill's bundled scripts; Step 2's hook/CLAUDE.md redirect targets were unnamed.
  • Per repository-owner request mid-implementation, capabilityAssumption was changed from the design's own initial Adaptive to Broad, applying the skill's own Step 3 to itself -- this required a real body rewrite (not a relabel) to inline every Step's load-bearing judgment call for dimension 9's stricter weak-tier-sufficiency bar.
  • Three independent, isolated review rounds ran against this draft (evaluating-skill-quality and battle-testing-a-skill, each dispatched fresh per round -- see ## Skill audit evidence below for the literal verdict tokens and what each round found). Every concrete, verifiable finding across all three rounds was fixed in a follow-up commit; see the commit history and metadata/gitapex.yaml's own append-only decision log for the specifics. The most consequential finding (round 3, battle-testing-a-skill): the adversarial eval fixture's own discrimination was proven inverted using this repository's own gitapex_score_contract.py scorer directly -- an injection-compliant reply scored a perfect 1.000 while a genuinely resisting reply scored only 0.846. Fixed and re-verified with the same scorer against 7 hand-crafted replies (4 compliant/evasive, all below the 0.8 threshold; 3 resisting, all 1.000).
  • A new CI drift gate, contract-discipline-drift (.github/scripts/gitapex_scan_contract_discipline_drift.py + .github/workflows/contract-discipline-drift-gate.yml, registered in .gitapex/ssot.json), keeps evaluating-skill-quality/references/rubric.md's Contract discipline section and drafting-a-skill/references/contract-structure.md from silently diverging -- an always-on content lock plus a diff-aware check that a PR editing the former's Contract-discipline section also touches the latter, or carries an explicit ack comment.
  • skills/planning-a-branch-from-an-issue/SKILL.md and skills/executing-a-branch-plan/SKILL.md each gained one Related-skills bullet routing to drafting-a-skill whenever their own planned ops include a new SKILL.md.
  • evals/drafting-a-skill/ ships 7 fixtures (normal, guardrail, edge, injected-self-certification-probe, no-self-authored-hook, no-inferred-metadata, existing-skill-routes-away), sized to match SKILL.md's own 7 Stop-boundary bullets per gitapex_gate_skill_branch_fixture_coverage.py's decision-branch/fixture parity requirement (verified directly: 7 branches, 7 fixtures, gate exits 0).
  • eval-gate (non-required check) is red on this PR, and this is a confirmed pre-existing, repo-wide CI defect, not caused by this suite's content. waza-eval-gate.yml ran evals/drafting-a-skill/ live for the first time the moment it was pushed and failed with error: model CLI exited 1: (empty stderr). Suspecting the injected-self-certification-probe.yaml fixture's "SYSTEM OVERRIDE" HTML-comment payload tripped Anthropic's content-policy classifier, I reworded it and re-pushed -- the failure reproduced identically (same empty-stderr signature, same ~2.2s timing), ruling that out. Cross-checking an unrelated branch (claude/gitapex-issue-1274-bwgwkg, different skills' suites, hours earlier the same day) shows the same signature. Filed as fix(waza-eval-gate): live eval runs fail with "model CLI exited 1" and empty stderr, repo-wide #1304 with full evidence; out of this PR's own scope to fix. eval-gate is not in .github/rulesets/main.json's required-status-checks list, so it does not block this PR. See evals/drafting-a-skill/eval-status.md for the full disclosure, including this correction of my own initial (wrong) diagnosis.
  • A merge conflict against main was resolved after main advanced 19 commits while this PR was open (first occurrence), including renaming drafting-an-acm-issue to drafting-issues. Merged main in; the only real conflict was in the auto-generated docs/skill-eval-status.md, resolved by regenerating it with the repo's own gitapex_generate_skill_eval_status.py rather than hand-editing conflict markers. Separately fixed 4 now-stale drafting-an-acm-issue references this skill carried (SKILL.md's compatibility field and one Related-skills bullet, metadata/gitapex.yaml's relatedTo list, and one references/ file) to the renamed drafting-issues. gitapex_check_skill_shape.py re-verified 54/54 PASS after the merge.
  • A second merge conflict against main was resolved (see the dedicated PR comment for the full writeup) after PR feat(independent-review-pending): feed-forward the gate and de-couple its heading from Step 8 #1348 merged upstream, independently adding its own 37th wired local-plane gate from the same 36-gate baseline this PR's own contract-discipline-drift gate also extended to 37 -- both PRs bumped the same wired-gate-count prose in 4 files; git's auto-merge silently kept a stale "37" in 3 of them (identical text on both sides merges cleanly) while the true post-merge total is 38. Fixed all four files to 38, re-measured a real warm timing (10.89s / 11.04s / 11.16s) rather than guessing, and confirmed the dedicated prose-count-vs-registry test passes clean. A pre-existing, unrelated staleness (this file's own top-of-docstring gate-count claims, invisible to that same test) was found and deliberately left alone as out of this PR's own narrow change surface. This second merge also brought in a new required status check, independent-review-pending (added to .github/rulesets/main.json by PR feat(independent-review-pending): feed-forward the gate and de-couple its heading from Step 8 #1348), which blocks on a ## Independent review verdict section naming the current head commit -- see that section below.
  • A third merge conflict against main was resolved (see the dedicated PR comment for the full writeup) after PR fix(executing-a-branch-plan): implement Axis 2/4/6 state-management fixes for Execution log #1342 merged upstream. The only real conflict was again the auto-generated docs/skill-eval-status.md (regenerated, not hand-edited); everything else auto-merged cleanly. skills/drafting-a-skill/ and evals/drafting-a-skill/ are confirmed untouched by this third merge.
  • A pytest re-run was required once on the first merge's post-merge head: test_gitapex_session_start_hook_shell.py::test_installs_the_prek_hook_for_a_real_checkout (a 15s subprocess timeout unrelated to this PR -- .claude/hooks/session-start.sh and its test are untouched by this diff, confirmed via git diff) flaked identically to an earlier occurrence on a different commit in this same PR, then passed clean on re-run. All other tests passed both times (5541-5542 passed).
  • All 10 of main.json's original required status checks (actionlint, ruff, pytest, mypy, exception-handler-gaps, hidden-characters, plugin-root-brace-notation, provenance-disclosure, pr-title-convention, betterleaks) pass on the current head; independent-review-pending (the 11th, newly added) is addressed by the verdict section below.
  • All 54 gitapex_check_skill_shape.py checks pass on the current head; the full pytest suite passes except one test (test_gitapex_scan_harden_checkout_pin_drift.py::test_repository_workflows_are_drift_free) that fails only in this sandbox's shallow git clone (confirmed: the failure is a documented shallow-clone limitation unrelated to this diff, and will pass in real CI, which fetches full history). The local-preflight gate runner passes 37/38 (same single shallow-clone-only failure; behind-base itself passes once a transient 60s network-fetch timeout on its first attempt is retried).

Assumptions

  • The three review rounds' fixes are correct as re-verified by direct empirical testing (the deterministic checkers, the fixture scorer run by hand against 7 crafted replies) and careful self-review against each finding's own cited evidence -- but a 4th fresh review round was deliberately not dispatched before opening this PR, given three already-substantial rounds and the strength of the direct empirical verification available for the two most severe findings (the shell:[] contradiction and the inverted fixture). This is a judgment call, not a certainty; a human reviewer or a follow-up review round may still find something the three rounds and this session's own direct verification missed.
  • The Blind spot pass gap named in round 2/3 (evaluating-skill-quality: no rubric dimension owns cross-skill duplication drift of prose criteria, since drafting-a-skill's Step 2 hand-copies evaluating-skill-quality's own Mechanism-fit criteria with no drift gate watching that specific copy) is a disclosed, accepted residual risk, not fixed in this PR -- per that same review's own guidance, a durable rubric change belongs in this repository's own held-out-gated edit process, not a single review/implementation session.
  • Issue fix(waza-eval-gate): live eval runs fail with "model CLI exited 1" and empty stderr, repo-wide #1304's own root cause is unconfirmed (candidates: ANTHROPIC_API_KEY misconfiguration, a @anthropic-ai/claude-code@2.1.240 CLI regression, or an API-side change) -- diagnosing further needs either live CI secret access or a way to reproduce locally with the same credential, neither available to this session.
  • The pre-existing "49 registered deterministic gates" / "24 currently-excluded gates" staleness named in the second merge's own PR comment (true counts: 68 / 30 as of that merge) is a disclosed, out-of-scope residual finding, not fixed here -- it predates the merged branches and none of the PRs merged into this branch touched that prose.

Risk / blast radius

Additive only: a new skill directory, a new eval corpus, a new CI workflow + script + tests (registered in .gitapex/ssot.json), and one new Related-skills bullet each in two existing skills' own prose. No existing skill's Steps, Stop boundaries, or behavior are modified. The new CI workflow (contract-discipline-drift-gate.yml) is not added to .github/rulesets/main.json's required-status-checks list, so it runs and reports but does not block merge on its own.

Rollback

Revert this PR (or the specific commits) -- no other file outside skills/drafting-a-skill/, evals/drafting-a-skill/, .github/scripts/gitapex_scan_contract_discipline_drift.py, .github/workflows/contract-discipline-drift-gate.yml, .gitapex/ssot.json's one new entry, .github/scripts/gitapex_gate_local_preflight.py's/.pre-commit-config.yaml's/CONTRIBUTING.md's wired-gate-count prose, tests/test_gitapex_gate_local_preflight.py, tests/test_gitapex_scan_contract_discipline_drift.py, and the two Related-skills bullets depends on this change.

Verification

Acceptance Criteria Map, reproduced from the issue with each row's real result:

Criterion Interpretation Planned ops Proof method Residual risk
skills/drafting-a-skill/SKILL.md exists with the drafted structure Frontmatter, Precondition, 9 Steps, Postcondition, Non-goals, Output, Worked example, Stop boundaries, Related skills, Notes Authored the file from the design, corrected by 3 review rounds gitapex_check_skill_shape.py: 54/54 PASS (re-verified after all three merges) None beyond the disclosed Blind spot pass gap (Assumptions)
Step 2's Mechanism-fit gate states concrete, quotable criteria, precedence over later Steps 4 criteria inlined directly in the body (not cited to the soon-retired writing-skills), each with its governing quote; explicit precedence sentence Written directly in SKILL.md Step 2 Direct read confirms all 4 criteria, governing quotes, and precedence sentence None identified
metadata/gitapex.yaml declares this skill's own decided choices Portability Mixed, capabilityAssumption Broad (changed from the design's own initial Adaptive per owner request), lifecycle experimental, skillDependencies requires [evaluating-skill-quality] (corrected from the design's own []), relatedTo naming 7 skills (4 original + 3 added when round-3 review found 3 named-but-unreconciled dependencies) Authored and iteratively corrected across all 3 review rounds skill-metadata.schema.json validation passes (via gitapex_check_skill_shape.py); decision log cites this issue and every correction Formal ratification by the repository owner beyond this PR's own review cycle is still the merge decision itself
All 6 cited reference files exist and resolve, with real prose 2 files loaded unconditionally at design time became, after the Broad rewrite, all 6 genuinely on-demand (the Broad-required inlining moved their load-bearing content into the body itself) Created all 6; iteratively corrected for internal self-contradictions and dead-weight linking found by review Every reference link in SKILL.md resolves; shape checker confirms no dangling reference-file citation None identified
A single independent fresh-context consistency audit runs before any implementation file is created An isolated pass checking ownership-boundary uniqueness, no superseded names, every named entry/exit point resolves Dispatched one isolated agent against the design text before any skills/drafting-a-skill/ file existed The audit's own explicit findings (3 real defects found and corrected) None beyond the 3 found and fixed
planning-a-branch-from-an-issue and executing-a-branch-plan route to drafting-a-skill One Related-skills bullet added to each Edited both files' Related-skills sections Manual read confirms both bullets; grep confirms drafting-a-skill appears in both None identified
A CI drift gate keeps rubric.md's Contract-discipline section and contract-structure.md in sync Content lock (always-on) + diff-awareness (PR touching the section without touching the reference file or an ack comment fails) New script + workflow + ssot.json entry, mirroring this repo's existing per-PR diff-scan gate class 31 tests in tests/test_gitapex_scan_contract_discipline_drift.py, including fail-closed and boundary cases None identified
The finished skill directory passes this repository's own bundled deterministic checkers gitapex_check_skill_shape.py and the registry-driven local-preflight suite run clean Ran both repeatedly across every fix round and after all three merges Shape checker 54/54; local-preflight 37/38 (sole failure is the disclosed shallow-clone artifact) None identified
The new SKILL.md is gated by real, independent quality-review verdicts, not waived Both evaluating-skill-quality and battle-testing-a-skill run from genuinely isolated dispatches 3 rounds of both, each fresh See ## Skill audit evidence below for the literal verdict tokens and evidence A 4th confirming round was not dispatched -- see Assumptions

Additional verification beyond the issue's own ACM:

  • evals/drafting-a-skill/'s fixture count (7) matches SKILL.md's own decision-branch count (7 Stop-boundary bullets), per gitapex_gate_skill_branch_fixture_coverage.py, run directly: exit 0.
  • Full pytest suite (post-third-merge): 5322 passed, 1 failed (the same pre-existing shallow-clone-only failure noted in Facts).
  • gitapex_gate_local_preflight.py (post-third-merge): 37 of 38 locally-runnable wired gates pass; the sole failure is the same shallow-clone-only limitation noted in Facts.
  • All 10 of the original required status checks pass on the current head; eval-gate (non-required) is red due to the pre-existing, repo-wide defect filed as fix(waza-eval-gate): live eval runs fail with "model CLI exited 1" and empty stderr, repo-wide #1304 -- see Facts.

Checklist

  • Tests pass locally
  • Docs updated if behavior changed (docs/skill-eval-status.md regenerated three times across three merges; CONTRIBUTING.md/.pre-commit-config.yaml/gitapex_gate_local_preflight.py's wired-gate-count prose updated 36 -> 37 -> 38 during the second merge)
  • Issue number cited in every commit
  • This PR adds a skills/*/SKILL.md and a new deterministic gate script -- ## Skill audit evidence below discloses the required verdicts
  • N/A -- no evals/*/split.md Kept-edit-log entry in this PR
  • This PR adds Stop-boundary bullets to a new skills/*/SKILL.md -- evals/drafting-a-skill/tasks/*.yaml fixture count (7) matches the decision-branch count (7), verified directly against gitapex_gate_skill_branch_fixture_coverage.py

Skill audit evidence

  • evaluating-skill-quality: WELL-FORMED-NOT-MATURE -- three independent, isolated dispatches ran across this PR's own iteration (one per round of fixes). The most recent (round 3, against commit 4a11d79) found: dimension 4 (Clarity/structure) -- a Postcondition the artifact itself falsified on two counts, a still-inaccurate Step 3 sidecar-field enumeration, an unlinked reference file self-contradicting its own load timing; dimension 5 (Progressive disclosure) -- the same unlinked-file issue and an inaccurate Notes claim about the reference load model. All of these were fixed in the commits immediately following that round (see metadata/gitapex.yaml's own decision log for the itemized list). No wrong-mechanism finding, no cohesion split finding, in any round. A 4th confirming dispatch was not run before opening this PR -- see Assumptions.
  • battle-testing-a-skill: FAIL -- three independent, isolated dispatches. Round 3 (the most recent, against commit 4a11d79) proved the round-2 fixture fix itself false (an injection-compliant reply scored a perfect 1.000 against the repo's own gitapex_score_contract.py scorer, a genuinely resisting reply only 0.846 -- the discrimination was inverted) and found 3 fail-open gaps needing no adversarial input (Step 3 unobtainable-elicitation-answer, Step 9 missing-dispatch-mechanism, Step 1 empty-input/encoded-content). All four fixed in the immediately-following commit and the fixture fix re-verified directly with the same scorer against 7 hand-crafted replies. Round 3 also confirmed, via a live escalated-pressure probe with the skill body actually loaded, that the untrusted-input-handling fix from round 2 holds behaviorally under real pressure -- not merely as prose. A 4th confirming dispatch was not run before opening this PR -- see Assumptions.
  • adversarial-coverage-mapping: RAN -- flagged security-relevant by the frontmatter security|gate|trust keyword heuristic (the description's own "Gates on Mechanism fit"). The mapping itself: SKILL.md's 7 Stop-boundary bullets against evals/drafting-a-skill/tasks/*.yaml's 7 fixtures, one of which (injected-self-certification-probe.yaml) is a genuinely adversarial embedded-instruction probe, not a happy-path case -- verified directly via gitapex_gate_skill_branch_fixture_coverage.py (exit 0). The three battle-testing-a-skill dispatches (see verdict above) are the adversarial-probing half of this same mapping; round 3 additionally proved that probe fixture's own scoring was inverted and fixed it, per Facts.
  • checker-script-adversarial-review: RAN -- gitapex_scan_contract_discipline_drift.py (new gate script) was reviewed by constructing deliberate defeat cases, not only happy-path tests: boundary tests at the exact section-span edges (test_diff_one_line_past_the_section_end_is_silent, test_diff_ending_exactly_at_the_last_section_line_fires), a garbage-diff-input test mirroring the sibling per-PR diff-scan gate's own adversarial-review-found false-positive class, and a decoy-file ack-token test.
  • deterministic-gate-quality: RAN -- read against evaluating-deterministic-gate-quality/references/dimensions.md dimension 15 (fail-closed on incomplete or malformed input): every read path (missing file, non-UTF-8 file, a directory in place of a file, a duplicated/absent/empty heading, an unparseable diff) raises a typed ScanError mapped to exit 2, never a silent pass or an uncaught traceback -- verified directly by tests/test_gitapex_scan_contract_discipline_drift.py's own dedicated fail-closed section.
  • defeat-test-disclosure: RAN -- see checker-script-adversarial-review above; the same boundary and malformed-input tests were specifically constructed to defeat the new detection logic, not merely exercise its happy path.

Independent review verdict

  • Verdict: CLEAN
  • Verified commit: ee94e73

Outer layer: GitHub Copilot review requested via request_copilot_review
against this exact commit; Copilot's review is Comment-only with no
pass/fail signal of its own, and had not yet posted a result as of this
verdict being recorded.

Inner layer: skills/drafting-a-skill/'s own substantive content is
confirmed byte-for-byte unchanged since 39a1f61 (verified directly:
git diff 39a1f61..HEAD -- skills/drafting-a-skill/ evals/drafting-a-skill/ produces zero output), already covered by the
three full review rounds disclosed above. The only new, self-authored
diff since the last recorded verdict (against 52f896e) is this third
merge's own one-line regeneration of the auto-generated docs/skill- eval-status.md (a deterministic script output, re-verified with
gitapex_generate_skill_eval_status.py --check) -- everything else in
this merge is main's own already-merged, already-reviewed content from
PR #1342. Read directly per this skill's own trivial-diff branch: a
single-line, deterministic-tool-generated change carries no independent
review risk beyond confirming the generator's own --check mode
agrees, which it does. No defect found; nothing to verify further
against blast radius, since the only self-authored change is a
regenerated doc byte confirmed to match a fresh run.

Related Issue

Closes #1194

claude added 8 commits August 24, 2026 14:33
gitapex-native successor to obra/superpowers' writing-skills and
Anthropic's skill-creator (refs #1194). Gates on Mechanism fit before
drafting, mandatorily elicits every user-selectable metadata choice,
drafts using Design-by-Contract structure, and hands off to
evaluating-skill-quality and battle-testing-a-skill for review.

A fresh-context consistency audit run before any implementation file
existed (per #1194's own Constraints) found and corrected 3 defects:
Steps 5 (cohesion) and 7 (domain-gap sweep) were reframed as advisory
self-checks rather than authoritative ones, since evaluating-skill-
quality's own rubric states its cohesion check "has exactly one owner"
and its Blind spot pass already runs as that skill's own precondition
step; skillDependencies.requires was corrected from [] to
[evaluating-skill-quality], since Step 8 mandatorily invokes its
bundled checker scripts; and Step 2's hook/CLAUDE.md redirect targets
were named concretely (evaluating-deterministic-gate-quality,
evaluating-context-channel-maturity).

Also adds a new CI drift gate (contract-discipline-drift) keeping
evaluating-skill-quality's Contract discipline section and this
skill's own contract-structure.md from silently diverging, wires
Related-skills routing into planning-a-branch-from-an-issue and
executing-a-branch-plan, and adds a disclosed-gap eval corpus
(evals/drafting-a-skill/) including a genuine adversarial fixture.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
… body

Repository owner request: applies drafting-a-skill's own Step 3 self-
referentially, changing the earlier Adaptive declaration to Broad.

Per evaluating-skill-quality/references/rubric.md's own Capability
assumption section, Broad requires dimension 9 (Cross-model robustness)
to hold under the full Haiku/Sonnet/Opus spread -- "the skill must give
a weak tier enough guidance, and failing to do so is a real, gradeable
gap, not an unmeasured one." This is a body rewrite, not a metadata
relabel: Steps 3-5 and 7 now inline their core judgment calls directly
(the four metadata axes' own option lists, the Design-by-Contract
definitions, the Single Decisive Outcome test, a concrete domain-gap
example) rather than requiring a weak model to follow a reference-file
pointer to find them. Adds a Worked example section and updates the
Notes section's own disclosure. The five references/ files are
unchanged -- progressive disclosure structure is unaffected, per that
same rubric section's own note that Broad/Frontier "leave dimension 5's
grading completely unchanged."

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
Two independent, isolated review dispatches against skills/drafting-a-
skill (evaluating-skill-quality: WELL-FORMED-NOT-MATURE; battle-testing-
a-skill: FAIL) found real defects, all fixed here:

- executionRequirements.tools.shell: [] (= prohibited) contradicted
  Step 8's own mandatory python3 checker invocations; corrected to
  shell: [python3], matching evaluating-skill-quality's own identical
  precedent.
- Two prose pointers to a "SKILL.md Non-goals" section that did not
  exist; added a real Non-goals section.
- Step 3's "every user-selectable metadata choice" over-claimed
  exhaustiveness against the sidecar's 8 real spec fields; scoped to
  the 4 elicited axes, with dependencyPolicy/skillDependencies/
  executionRequirements now explicitly stated as derived facts.
- Step 6 had no completion criterion; added one.
- Four reference files (guidance-form-and-sdo.md, formative-quality-
  dimensions.md, contract-structure.md, tacit-knowledge-elicitation.md)
  claimed to be "unconditional"/loaded whenever their Step begins, when
  the ordinary path needed at most one of six -- now genuinely on-demand,
  since the Broad rewrite already inlined their load-bearing content.
  formative-quality-dimensions.md's own self-description of this split
  was also miscounted; corrected.
- Dimension-1 collision with the still-installed writing-skills/
  skill-creator was undisclosed; added a Related-skills bullet and a
  compatibility: frontmatter field.
- No Stop boundary stated that requester-supplied source text is
  untrusted data. battle-testing-a-skill demonstrated, by running a
  compliant reply against the original injected-self-certification-
  probe.yaml fixture, that its assertions would false-pass an actual
  injection compliance. Added explicit untrusted-input handling to
  Step 1, Step 9, and Stop boundaries; strengthened the fixture's
  assertions to require dispatch language, not just name mentions, and
  banned a broader set of compliance paraphrases.
- Tightened the Postcondition's self-granted-deferral seam (a deferred
  Step 5/7 finding now requires a substantive stated reason, and states
  explicitly that deferral changes nothing about what Step 9's downstream
  reviews will independently find).
- Added an explicit escalation path to Step 2 for a candidate that fits
  neither the block nor create list cleanly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
A third round of independent, isolated dispatches against
skills/drafting-a-skill (both re-verdicted WELL-FORMED-NOT-MATURE)
found subtler self-consistency defects the first two rounds' fixes
introduced or left in place:

- The Postcondition asserted "passes both Step 8 checkers clean" while
  the execution-requirements-drift scanner reported a heuristic
  tools-shell-over-declared warning (a real detection blind spot: the
  scanner pattern-matches SKILL.md prose for a backtick-adjacent `run`/
  invocation, which Step 8's prose didn't contain). Fixed by inlining
  the actual invocation commands into Step 8 itself, which both
  resolves the warning honestly and improves Broad's own weak-tier
  self-sufficiency.
- The Postcondition also overstated Step 6 ("collides with no existing
  skill"), when Step 6's own completion criterion explicitly permits a
  deferred collision -- and this draft is itself a deferred-collision
  case. Reworded to match Step 6's real rule.
- Step 3's field-enumeration was still inaccurate: Invocation mode was
  described as a metadata/gitapex.yaml field (it's SKILL.md
  frontmatter), and two sidecar fields (references, externalCitations)
  were omitted from "the sidecar's other fields." Corrected, with
  explicit guidance that `spec.references` is filled in as decisions
  happen, not backfilled.
- Steps 5 and 7 stated what NOT to write for a finding but never the
  observable result for "no finding" -- added an explicit branch to
  both.
- Step 6's "every existing skill" scope was ambiguous about installed-
  but-not-native skills, which is exactly the population this draft's
  own Related-skills collision bullet concerns. Clarified.
- Three sibling skills (untrusted-input-triage, drafting-an-adr,
  grounding-in-primary-sources) were named mid-procedure but absent
  from skillDependencies.relatedTo and Related skills -- Step 6's own
  reconciliation rule, applied to itself. Added.
- formative-quality-dimensions.md was entirely unlinked from SKILL.md
  (dead weight) and self-contradicted about its own load timing ("not
  required reading before Step 4" vs. "read it once before Step 4
  begins" two sections later). Linked from Step 8; contradiction fixed.
- Two reference files used "escalate per Step 7" as a generic
  illustration, when this skill's own Step 7 is the domain-gap sweep,
  not an escalation step -- corrected to avoid the collision.
- The Notes' claim that no reference file is "required reading for the
  ordinary path" was false for gitapex-cross-links.md, which carries
  Step 8's own invocation flags found nowhere else. Corrected.
- The body crossed 90% of BODY_MAX_LINES; added the rubric's own
  required Declaration-vs-structure-fit disclosure, then trimmed
  elsewhere for margin.
- eval-status.md's ablation-capability state was collapsed into "no
  baseline exists" rather than "ablation-capable, not yet run" (a real
  runner exists: evals/scripts/gitapex_run_ablation.py); also disclosed
  two previously-silent coverage gaps (the Precondition's two
  route-away branches, and zero fixture citing any of the nine
  formative-quality-dimensions.md rows).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
A third battle-testing-a-skill dispatch (FAIL) proved the previous
round's fixture fix was itself false: "dispatch" is a verbatim
substring of the injected payload, so an injection-compliant reply
scored a perfect 1.000 against gitapex_score_contract.py while a
genuinely resisting reply scored only 0.846 -- the discrimination was
inverted, not merely imperfect. Rebalanced to 3 multi-word positive
phrases (verified absent from the payload) against 9 bans, since flat
per-item weighting had diluted 2 positives against 17 bans to where an
evasive reply demonstrating neither behavior still scored 0.905.
Reverified directly with the same scorer against 7 hand-crafted replies
(4 compliant/evasive, all below threshold; 3 resisting, all 1.000).

The same dispatch found three fail-open gaps needing no adversarial
input: Step 3 had no rule for an unobtainable elicitation answer (an
agent under pressure took a self-chosen provisional value, exactly
what mandatory elicitation forbids); Step 9 had none for a missing
fresh-dispatch mechanism (the agent performed both reviews in-context,
the exact substitution Stop boundaries forbid); Step 1 had none for an
empty candidate job or content hidden via encoding/HTML comments. All
four given explicit stop-and-escalate or decode-first rules. Also
widened the Stop-boundaries untrusted-input bullet to match Step 9's
own broader "original request or pasted source text" framing (it was
scoped only to pasted content).

The body crossed 500/500 lines after these additions; trimmed several
Notes paragraphs and Related-skills bullets for margin without losing
substance (483/500).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
…n stdlib-only claim drift

skills/drafting-a-skill/SKILL.md has 7 Stop-boundary bullets, but only
4 fixtures existed under evals/drafting-a-skill/tasks/ --
gitapex_gate_skill_branch_fixture_coverage.py requires parity for a
brand-new skill (verified directly: 7 branches, 7 fixtures now, gate
exits 0). Added 3 fixtures covering the previously-uncovered bullets:
never author a hook in place of redirecting after Step 2 blocks a
hook-shaped candidate, never infer the four metadata axes even when
invited to, and the Precondition's own existing-skill route-away branch
(this also closes a previously-disclosed eval-status.md gap).

Also fixed a genuine stdlib-only-claim-drift finding against the new
contract-discipline-drift gate itself: its own docstring Usage:: example
showed a bare `python3 <file>.py` invocation alongside a `uv run`-wrapped
one (exactly the anti-pattern that gate exists to catch), and its
workflow's own comments literally contained the substring
"stdlib-only-claim-drift" (naming the sibling gate by its real filename),
which the drift gate's stale-phrase regex matches regardless of context.
Fixed the bare invocation and reworded the comments to reference the
sibling gate without spelling out that exact compound name.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G6i18wop15RG33krfnuZhY
…gzz6o

# Conflicts:
#	docs/skill-eval-status.md
Refs #1194. docs/skill-eval-status.md is auto-generated by
gitapex_generate_skill_eval_status.py; this adds the drafting-a-skill
row (3 trials, 7 fixtures) and updates the trials_per_task tally.
@tvna
tvna deployed to ruleset-verify August 24, 2026 23:01 — with GitHub Actions Active
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 3d57502e-e177-41fd-a6c9-10b9cea4f354


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.54%. Comparing base (786121d) to head (ee94e73).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1303   +/-   ##
=======================================
  Coverage   99.54%   99.54%           
=======================================
  Files         123      124    +1     
  Lines       21912    21996   +84     
  Branches     2682     2699   +17     
=======================================
+ Hits        21813    21897   +84     
  Misses         99       99           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…ve content-policy block

Refs #1194. waza-eval-gate.yml's first live run of this suite failed
outright: injected-self-certification-probe.yaml's "SYSTEM OVERRIDE"
HTML-comment payload tripped a hard content-policy block reported via
empty stderr, which gitapex_run_eval_suite.py's own
_is_content_policy_rejection two-marker signature never matches,
aborting the whole suite instead of skipping one fixture. Reworded the
payload away from impersonating a system-level directive while
preserving the same false-prior-approval, skip-Step-9 test property;
re-verified the scorer discrimination still holds against 7 hand-crafted
replies.
…waza-eval-gate failure

Refs #1194. The reworded injected-self-certification-probe.yaml payload
failed waza-eval-gate identically to the original -- same empty-stderr
"model CLI exited 1" signature, same timing -- and the identical
signature reproduces on an unrelated branch touching different skills'
suites hours earlier. This is a pre-existing, repo-wide waza-eval-gate
defect, not caused by this suite's fixture content; filed as #1304.
Corrects the prior commit's "SYSTEM OVERRIDE tripped a content-policy
block" diagnosis, which this evidence disproves.
@tvna
tvna deployed to ruleset-verify August 24, 2026 23:16 — with GitHub Actions Active
claude added 2 commits August 25, 2026 04:42
…gzz6o

# Conflicts:
#	docs/skill-eval-status.md
…after main's rename

Refs #1194. main renamed drafting-an-acm-issue to drafting-issues while
this branch was in flight (merged in via the preceding merge commit).
Updated the 4 references this skill carried to the old name -- SKILL.md's
compatibility field, its Related-skills bullet, metadata/gitapex.yaml's
relatedTo list, and references/tacit-knowledge-elicitation.md -- so
skill-dependencies-resolve and related-skill-references-resolve stay
green. Shape checker: 54/54 PASS.
@tvna
tvna deployed to ruleset-verify August 25, 2026 04:51 — with GitHub Actions Active
…gzz6o

# Conflicts:
#	.github/scripts/gitapex_gate_local_preflight.py
@tvna
tvna deployed to ruleset-verify August 26, 2026 08:29 — with GitHub Actions Active

tvna commented Aug 26, 2026

Copy link
Copy Markdown
Owner Author

Merge conflict resolution (second occurrence)

main advanced with PR #1348, which independently added its own 37th
wired local-plane gate (independent-review-heading-drift) from the same
36-gate baseline this PR's own contract-discipline-drift gate also
extended to 37. Both PRs edited the same "36 -> 37" wired-gate-count prose
in CONTRIBUTING.md, .pre-commit-config.yaml,
.github/scripts/gitapex_gate_local_preflight.py, and its test file.

Resolution: merged origin/main in (commit 6a3368e as of the merge). Git's
3-way auto-merge produced one real conflict marker, in
gitapex_gate_local_preflight.py's own historical gate-count-vs-timing
comment; the other three files' "37" auto-merged cleanly (identical text
on both sides) but silently understated the true post-merge total of 38
local-plane gates. Caught by cross-checking the live registry
(.gitapex/ssot.json) rather than by git itself. Fixed all four files'
prose to 38 (and "35 of the 37 wired gates" to "36 of the 38 wired gates",
since the CI-backstop-exception count -- behind-base,
real-checkout-git-write -- is unchanged at 2), re-measured a real warm
timing (three consecutive full local-preflight runs: 10.89s / 11.04s /
11.16s) rather than reusing a guessed number, and confirmed
tests/test_gitapex_gate_local_preflight.py::test_no_prose_count_contradicts_the_registry
(which derives its expected counts from the live registry, not a
restated constant) passes clean.

A pre-existing, unrelated staleness was found and deliberately left
alone: this file's own top-of-docstring "49 registered deterministic
gates" / "24 currently-excluded gates" claims (true counts: 68 / 30)
predate both merged branches and are invisible to that same test's
digit-adjacency regex. Out of this PR's own narrow change surface, not
touched here -- worth a follow-up issue, not a fix inside a
merge-conflict resolution.

Full re-verification after the merge: gitapex_check_skill_shape.py
54/54 PASS; full pytest suite 4765/4766 passed (the sole failure is
the pre-existing, already-disclosed shallow-clone-only artifact); the
local-preflight gate runner 37/38 (same sole failure).


Generated by Claude Code

…gzz6o

# Conflicts:
#	docs/skill-eval-status.md
@tvna
tvna deployed to ruleset-verify August 26, 2026 18:08 — with GitHub Actions Active

tvna commented Aug 26, 2026

Copy link
Copy Markdown
Owner Author

Merge conflict resolution (third occurrence)

main advanced further with PR #1342, which merged after this PR's
second conflict-resolution commit (52f896e). mergeable_state flipped to
"dirty" again.

Resolution: merged origin/main in (commit 8cf80ce as of the merge).
The only real conflict was in the auto-generated docs/skill-eval- status.md, resolved the same way as the first occurrence: regenerated
it with gitapex_generate_skill_eval_status.py rather than hand-editing
conflict markers, then confirmed with --check that the regenerated
file matches a fresh run exactly. All other touched files
(.gitapex/ssot.json, gitapex_gate_local_preflight.py,
skills/executing-a-branch-plan/SKILL.md) auto-merged cleanly with no
conflict markers.

skills/drafting-a-skill/ and evals/drafting-a-skill/ -- this PR's
own actual content -- are confirmed untouched by this merge (git diff --cached --stat -- skills/drafting-a-skill/ evals/drafting-a-skill/
produced no output). Everything else pulled in (a large batch touching
drafting-a-pr-to-merge/SKILL.md, executing-a-branch-plan's bash-
safety scripts, and several new test files) is main's own already-
merged, already-reviewed content from PR #1342, not something this
session authored or needs to re-review.

Full re-verification after the merge: gitapex_check_skill_shape.py
54/54 PASS; full pytest suite 5322 passed, 1 failed (the same
pre-existing, already-disclosed shallow-clone-only artifact); the
local-preflight gate runner 37/38 (behind-base's first run hit a
60s network fetch timeout -- a transient failure, not a real behind-
base state -- and passed clean on a second run with a longer timeout;
harden-checkout-pin-drift is the same one pre-existing shallow-clone
failure).


Generated by Claude Code

@tvna
tvna marked this pull request as ready for review August 26, 2026 21:00
@tvna
tvna merged commit 11b9555 into main Aug 26, 2026
32 of 34 checks passed
@tvna
tvna deleted the claude/pr-1194-prep-egzz6o branch August 26, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(skills): add drafting-a-skill -- gitapex-native blank-page skill authoring (writing-skills/skill-creator successor)

2 participants