feat(gates): add unguarded-shell-pipe-in-docs gate - #1713
Merged
Conversation
Issue #1531 (refs #1567): a documented "git log ... | python3 gitapex_check_task_commit_provenance.py" invocation piped two commands directly together, so an upstream git log failure silently reported a clean result instead of blocking a merge. Add a documentation-lint gate (sibling to no-raw-gh-cli-in-docs) that flags a cmd1 | cmd2 shaped shell pipe example in a SKILL.md, a skills/*/references/*.md file, or a checker/gate script's own module docstring, when no pipefail disclosure is nearby. The new gate's own authoring found 8 pre-existing instances of this exact defect class already in the repository (7 checker-script Usage recipes, 1 worked-example transcript); each is fixed with a brief pipefail-risk caveat or an explicit exception marker in this same change, so the gate ships already green rather than red from day one. Registers the gate in .gitapex/ssot.json (ci and local planes), wires a CI workflow, and updates the wired-gate-count prose (44 to 45) across CONTRIBUTING.md, .pre-commit-config.yaml, and the local-preflight runner's own docstring and tests. Also records a metadata sidecar correction for the one-line count drift the pipefail-caveat edit caused in gitapex-worked-examples.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
Issue #1531. Add direct-call tests for describe(), markdown_violations_ in_text, violations_in_markdown_file, violations_in_python_file, _tracked_files, _read_text, and GateUnguardedShellPipeInDocsArgs's _root_must_exist accepting path -- each was already exercised indirectly through find_violations()/main(), but function-body-test- coverage-gate wants each mentioned by name in a test the same diff adds. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Team Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1713 +/- ##
========================================
Coverage 99.66% 99.67%
========================================
Files 152 153 +1
Lines 24792 24963 +171
Branches 2981 3010 +29
========================================
+ Hits 24710 24881 +171
Misses 82 82 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
Issue #1531. Independent review (5 adversarial passes: correctness, blast-radius, reuse, convention, security) found and this fixes: - The regex could not see a pipe split across a shell line-continuation (a "cmd \\" line followed by "| consumer" on the next line) -- this repository's own common Usage:: wrapping convention for a long producer command. Live-confirmed 13 real, previously-undetected instances of the exact #1531 defect class already in the repository (git diff/git -c ... diff piped into a checker with no pipefail disclosure); each is now fixed with the same pipefail caveat as the first 8. Detection now joins a backslash-continued line with its successor before matching. - _MARKDOWN_PATHSPECS lacked the :(glob) pathspec magic _PYTHON_PATHSPECS already carried, so a bare `*` could cross a `/` and match a doubly-nested SKILL.md/references file outside the intended one-level scope -- live-confirmed via git ls-files, fixed, and pinned with two regression tests. - A stale :func: cross-reference and test-count claim in the properties test's own docstring. - Softened an overclaiming "closes both false-positive classes" line (a Markdown table cell literally equal to a consumer token still matches) and disclosed the whole-block/whole-docstring exemption scope and the backtick exclusion's line-wide (not span-aware) shape as known gaps, each pinned by a regression test. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018Aou8TJ15YpJfjaoQC6biK
tvna
marked this pull request as draft
September 3, 2026 05:28
tvna
marked this pull request as ready for review
September 3, 2026 05:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a documentation-lint gate (issue #1531, refs #1567) that flags an
unguarded
cmd1 | cmd2-shaped shell pipe example, with no nearbypipefaildisclosure, in a
skills/*/SKILL.md, askills/*/references/*.mdfile, or achecker/gate script's own module docstring -- and fixes the 8 pre-existing
instances of that exact shape the gate's own authoring found in this
repository, so it ships already green.
Live re-verification (against current
origin/main, post-#1675 + post-#1701)found #1488 and #1626 already fully resolved with no residual gap; see the
closing comment on #1567 for the reproduction. This PR is scoped to #1531
only.
Facts
git log ... | python3 gitapex_check_task_commit_provenance.pyinvocation piped two commands directly together, so an upstream
git logfailure (a stale/unresolvable BASE ref, a rebase, a shallow worktree)
silently reported a clean
PASS: no commits in rangeinstead of blockinga merge.
gitapex_check_task_commit_provenance.py's own module docstring alreadydocuments the fix (a two-step, never-piped invocation) and warns against
the pipe form -- but nothing in this repository checked that no other
documented recipe reintroduces the same shape.
further live instances of the same undisclosed-pipe shape: 7 checker/gate
script
Usage::recipes (.github/scripts/gitapex_detect_changed_gate_ scripts.py,gitapex_gate_acm_issue_disclosure.py,gitapex_gate_independent_review_pending.py,gitapex_gate_pr_title_ convention.py,gitapex_gate_provenance_disclosure.py,skills/executing-a-branch-plan/scripts/gitapex_check_canonical_ governance_paths.py,gitapex_check_file_ownership_conflicts.py) and 1worked-example timing transcript
(
skills/evaluating-deterministic-gate-quality/references/gitapex-worked- examples.md). Each is fixed in this same change (a one-line pipefailcaveat for the 7 scripts; an explicit exception-marker comment for the
transcript, since altering its own copy-pasteable content would corrupt
the recorded measurement).
#1488/#1626re-verification (documented on gate-proposal-umbrella: local-hook fail-open remediation #1567 directly): reproducedboth original defects live against
hooks/check-pr-skill-audit- disclosure.shon currentorigin/main(commit9db591d, confirmed viagit logto include both feat(gate-preconditions): establish environment/repo-state preconditions before gates run #1675 and fix(hooks): resolve PATH-dependent bare python3 false-deny (#1697, #1581) #1701 before testing). A syntheticcreate_pull_requestpayload modifying a.github/scripts/gitapex_gate_ *.pyfile with no## Skill audit evidencesection was denied(exit 2); a payload disclosing only 1 of 3 applicable lines was still
denied, correctly naming the 2 remaining missing lines; a payload
disclosing all 3 passed (exit 0). No residual gap for either issue.
reuse, convention, security) found a real, high-confidence gap: the
regex could not see a pipe split across a shell line-continuation, and
13 further real, previously-undetected instances of this same defect
class already existed in the repository. All 13 are fixed in this same
change; see the Independent review verdict section below for the full
disposition of every finding.
Assumptions
no-raw-gh-cli-in-docs-gate.yml"alreadyscans skills/*/SKILL.md and script docstrings for a different unsafe-
shell-command shape" -- checked directly against that gate's own source
(
.github/scripts/gitapex_gate_no_raw_gh_cli_in_docs.py) and this isnot accurate: that gate's own scope is
docs/**/*.mdonly, viagit ls-files -- docs, with no SKILL.md/references/script-docstringcoverage at all. This does not change what the new gate should do
(issue gate-proposal: retro #1530 repair 1: Undocumented fail-open via unguarded shell pipe in a merge-blocking gate's own invocation #1531's own proposed scope stands on its own), so it is noted here
as a correction to the issue's own premise rather than acted on further.
pipefailmentioned inside thesame fenced block (Markdown) or the same module docstring (Python), or an
explicit
gitapex-allow-unguarded-shell-pipeHTML-comment exceptionmarker (with a reason) directly above the fence/flagged line -- a
deterministic, precedented interpretation (mirrors the sibling gate's own
exception-marker convention) rather than a looser "anywhere in the
surrounding prose" reading, which would need markdown-heading-based
section detection with no precedent in this repository. Disclosed as a
scope choice in the new gate's own module docstring (which spells out the
exact marker syntax).
Risk / blast radius
New CI workflow (non-required by default; promoting it to a required status
check is a separate GitHub admin action) plus a new local-preflight-wired
gate. Docstring-only edits to 20 existing checker/gate scripts (no logic
changed) and one exception-marker line in a worked-example doc. No runtime
behavior changes anywhere in this diff.
Rollback
Revert this PR's merge commit. The new workflow file, gate script, and
.gitapex/ssot.jsonregistry entry are removed together; the docstringcaveats and the exception-marker line revert with it (
git revert, nomanual follow-up needed).
Verification
Acceptance Criteria Map:
hooks/check-pr-skill-audit-disclosure.shon currentorigin/main: syntheticcreate_pull_requestpayload, no-disclosure body.github/scripts/gitapex_gate_unguarded_shell_pipe_in_docs.py+ CI workflow +.gitapex/ssot.jsonregistration + 65 tests (example + Hypothesis property), plus 21 total pre-existing instances fixed in the same change (8 found at authoring time, 13 more found by independent review)\bboundary; dropping the escaped-backslash guard) confirm property tests catch each regressionCommands run and their output:
uv run --frozen python3 .github/scripts/gitapex_gate_unguarded_shell_pipe_in_docs.py->OK: 92 Markdown file(s) and 167 Python file(s) carry no unguarded shell pipe examples.uv run --frozen pytest tests/test_gitapex_gate_unguarded_shell_pipe_in_docs.py tests/test_gitapex_gate_unguarded_shell_pipe_in_docs_properties.py --cov=... --cov-report=term-missing -q->65 passed, 100% coverageuv run --frozen pytest -q(full suite) ->8071 passeduv run --frozen python3 .github/scripts/gitapex_gate_local_preflight.py->all 45 wired gate(s) passedgit push(pre-push hook re-runs the same local preflight) -> passed, twice more after the review-driven fix roundsChecklist
evals/*/split.mdKept-edit-log entry addedskills/*/SKILL.mdStop-boundary/dispatch-branch changeSkill audit evidence
This diff touches no
skills/*/SKILL.md, so the base battle-testing-a-skill/ evaluating-skill-quality audits do not apply
(
gitapex_compute_skill_audit_flags.py --check-diffconfirmsskill-md-changed: false). It does touch multiple deterministicchecker/gate scripts (the new gate itself,
.gitapex/ssot.json, and manyexisting gate scripts with docstring-only edits):
Evidence for the new gate (
gitapex_gate_unguarded_shell_pipe_in_docs.py):false-positive calibration against the real corpus (Python type hints,
Markdown tables, backtick-quoted illustrative prose, a
curl | bashsecurity discussion) before finalizing the consumer-token vocabulary and
the backtick-exclusion rule; fail-closed defeat cases constructed and
confirmed (empty repo, non-existent root, non-UTF-8 file, invalid Python
syntax -- all exit 2, never a silent pass, per dimension 15); live mutation
tests confirmed property tests catch each regression before it shipped; a
full independent review round (5 adversarial personas) found and this PR
fixed a real detection gap (a pipe split across a shell line-continuation,
with 13 real live instances) plus several smaller correctness/convention
issues -- full disposition in the Independent review verdict section below.
The modified existing gate scripts received docstring-only edits (no logic
changed), so their own prior fail-closed properties are unaffected.
Independent review verdict
Outer layer: GitHub Copilot review requested via
request_copilot_reviewat 2026-09-03T04:58Z. No response posted as ofthis recording; per the repository owner's explicit direction this turn,
the wait for it is skipped rather than held for the full 30-minute window
this skill's own Step 8 otherwise specifies -- disclosed here as
owner-skipped, not as "unreachable."
Inner layer:
reviewing-an-artifactrun at low effort via 5 paralleladversarial dispatches (correctness, blast-radius, reuse-and-simplification,
convention, security) against this PR's full diff. Findings and their
disposition, each independently verified (FABRICATED check + live
confirmation) before acting:
pipe split across a shell line-continuation -- live-confirmed against 13
real, previously-undetected instances of issue gate-proposal: retro #1530 repair 1: Undocumented fail-open via unguarded shell pipe in a merge-blocking gate's own invocation #1531's own defect class
already in this repository; all 13 fixed with the same pipefail-caveat
convention as the original 8, plus a new
_effective_linejoin, 5 newexample tests, 2 new property tests, and a live mutation-test confirming
detection.
_MARKDOWN_PATHSPECSlacked the:(glob)pathspec magic_PYTHON_PATHSPECSalready carried, so a bare*could cross a/and match a doubly-nested SKILL.md/references fileoutside the intended scope -- live-confirmed via
git ls-files, fixed,pinned with 2 regression tests.
:func:cross-reference and a hardcoded,already-drifting test-count claim in the properties test's own
docstring; a typo; an ssot.json
triggerfield omitting the_properties.pyfile; an imprecise attribution sentence.false-positive classes" sentence (a Markdown table cell literally equal
to a consumer token, e.g.
| Parser | jq |, still matches) -- softened,disclosed as a known gap, pinned by a regression test.
test: the whole-fenced-block/whole-module-docstring exemption scope
(a
pipefailmention or allow marker clears every match in the sameunit, not only the one it was written to justify) and the backtick
exclusion's line-wide, not span-aware, check (an unrelated backtick term
earlier on a line can exempt a real, unquoted recipe later on that same
line). Closing either needs a position-aware span check this gate does
not implement; judged disproportionate to add for a documentation-lint
advisory check whose own stated residual risk already accepts this class
of imprecision, and a human reviewer reading a marker's stated reason
against the block's actual content remains the backstop, the same trust
this gate's own sibling already places in its analogous marker.
find_violations's fail-closedcheck from "raise when neither corpus has any files" to "raise when
either corpus alone is empty" -- live-confirmed this breaks 30 of this
file's own tests (each legitimately populates only one category to
isolate what it tests), so reverted;
test_repository_scan_reaches_a_ real_tracked_set's own real-corpus floor assertion is judged theproportionate backstop for that concern instead.
diff): a
gitapex_gate_local_preflight.py's own "49 registeredgates"/"24 excluded" claim not reconciling against the current registry,
and
skills/executing-a-branch-plan/scripts/test_gitapex_check_task_ full_verification.py's own stale "42 real wired gates" claim (alreadystale before this diff bumped the true count to 45) -- both predate this
diff and are unrelated to its own change surface.
safety, path-traversal, the workflow file's own permission scoping and
action pinning, the fence-pairing/marker-indexing/off-by-one arithmetic,
the docstring-only-ness of every existing-script edit, and the
bookkeeping consistency of every "44 -> 45 wired gates" claim this diff
did touch.
Zero outstanding
confirmedfindings remain after the fixes above; allunconfirmed-concern/disclosed-gap items are recorded above rather thansilently dropped. Full audit trail (every candidate raised, and why each
was fixed, disclosed, reverted, or ruled out of scope) is in this PR's own
commit history (the review-driven fix commits following the initial
implementation).
Related Issue
Closes #1567
Refs #1531, #1488, #1626