ci(gates): re-run the commit-time gates in CI, which a replayed commit skips (BACKLOG #1395) - #823
Merged
Merged
Conversation
…t skips (BACKLOG #1395)
git does not invoke the pre-commit hook for a commit created by the sequencer, so a rebase or
cherry-pick lands a commit with none of the eleven gates having run and nothing reports it. #1395
prefers a CI-side re-run over a second local hook, because a local hook is advisory by construction
and that row is the proof.
precommit-replay.yml runs `pre-commit run --from-ref --to-ref` over the pull request's diff. It
skips exactly two hooks, and the rule is narrow: skip only where running it on a runner gives a
WRONG answer, not merely an inconvenient one.
ledger-gate its hook entry omits --ci, so the ownership arm would read an allocation
registry living in .git/mefor-coord/ that never reaches a runner. ci.yml
already runs ledger_check.py --ci, the half CI can enforce.
forbidden-content fails closed on a git-ignored token file no runner has. Reproducing
security.yml's fork/secret branching in a NON-REQUIRED leg would either red
every fork pull request or return a structural-only green that reads as a
leak-gate pass. security.yml's REQUIRED job already scans the whole tree.
Three guards, because the leg's failure modes are all silent. The skip list is pinned as a SET: it
grows one convenient entry at a time until the leg runs nothing and still reports green, and
pre-commit does not validate SKIP, so a typo skips nothing while reading as deliberate coverage.
The invocation is asserted over the resolved run block rather than the file, because the header
prose names --from-ref/--to-ref and a file-wide check would survive deleting the step. And the job
is asserted absent from .github/required-contexts.txt.
Also pins the VERSION axis _MIRRORS does not reach. It anchors on the invocation, so it proved the
right tool ran with the right discriminating flag and said nothing about which BUILD ran. gitleaks
(v8.18.4) and actionlint (v1.7.12) were held only by prose comments saying to keep the version in
step, and a comment cannot fail. Anchored on each repo's own download URL so a stray VER=
elsewhere cannot satisfy the comparison.
Deliberately NOT a required context: it builds every pinned hook environment per run. Promoting it
needs branch protection moved first, then required-contexts.txt, then the count in
tests/test_required_contexts.py, and a merge_group trigger before any of that.
Every guard was mutation-tested. Rev drift, SKIP growth, a SKIP typo, the run step losing
--from-ref while the header kept it, a bare `pre-commit run`, and the job acquiring a required
context all go red, and the tree returns to green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… (BACKLOG #1395) The two bandit arms in this file have pinned SCOPE since the hook and CI first drifted apart, and nothing pinned VERSION. That is the same divergence one level down: `--skip B101,...` means different findings under different bandit releases, so two halves that agree on every skip and every exclude can still enforce different standards with nothing saying so. It is not hypothetical here. The ci-scanners group's own comment records an unpinned 1.9.x upgrade silently changing `# nosec` parsing and breaking a green branch, which is why that pin is exact. It matters most on a commit nobody gated. Under #1395 git does not run pre-commit for a commit created by the sequencer, so after a rebase the CI build is the only bandit that ever looked, and a developer whose commit passed locally has learned nothing about the version that will judge it. `pre-commit autoupdate` is the likely author: .pre-commit-config.yaml already warns that a bare run walks the ruff rev past its cap, and it walks this one too. Lives here rather than in tests/test_gate_ci_mirror_parity.py deliberately. That file owns the eight hooks this one does not, and bandit is one of the three it defers here. One hook, one place, so the two files cannot grow a second silently different definition of the same rule. The sibling gitleaks and actionlint revs are held there in the same change. The `==` is asserted rather than borrowed from test_ci_venv_pinning's EXACT_GROUP_PINS: `uv export` writes a fully pinned lock from a `>=` spec just as readily, so a floor would leave this comparison holding the rev against a version nobody promised to install, and nothing downstream would reveal it. Mutation-tested: moving the hook rev to 1.9.3 against the group's 1.9.4 goes red, and restoring it returns the file to green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…w that was never true Two changes to the #1395 row. SHIPPED: the CI-side re-run the row prefers, plus the version axis nothing held. Records which two hooks the leg skips and why each would give a WRONG answer on a runner rather than merely an inconvenient one, that the skip list is pinned as a set, that the leg is deliberately not a required context and what promoting it would cost, and which mirror still covers each skipped hook. It also records what the leg does NOT buy, so the row is not read as closure of the whole finding. The leg is late in exactly the way the 2026-09-03 scoring note says, a changed-file set against the checked-out tree still cannot see content that lives only in an intermediate commit, and the ownership arm still has no enforcement path on a replayed commit and cannot be given one. CORRECTION: the row ended a paragraph with "A PARITY TEST FOR THOSE EIGHT IS THE DURABLE FIX AND IS NOT BUILT", and that was already false when it was written. tests/test_gate_ci_mirror_parity.py covers exactly those eight and landed in 389168a, the very commit that filed this row. The scoring note above repeats the claim, so it over-counted the remaining work. Corrected in place rather than annotated, because a reader who reaches that sentence decides whether to BUILD that module, and two of them would then exist. A second silently different definition of one rule is the defect that whole family of tests exists to prevent. The correction is recorded rather than the sentence quietly deleted, so the next reader knows which half is real instead of wondering. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Collaborator
Author
|
LANDER: applying SO READ THIS LABEL AS A BYPASS TOKEN, NOT A REVIEW. It records that the gate was cleared by instruction. It does not assert that anyone read this diff, and no later reader should infer one from it. What I can say: this PR appeared in a read-only pre-verification sweep of all 59 non-draft open PRs (per-PR defect read, plus pairwise -- Lander |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
BACKLOG #1395:
gitdoes not invoke thepre-commithook for a commit created by the sequencer, soa rebase or cherry-pick lands a commit with none of the eleven gates having run, and nothing
reports it.
Two things ship here.
1. The CI-side re-run, which is the fix the row itself prefers.
.github/workflows/precommit-replay.ymlrunspre-commit run --from-ref --to-refover the pullrequest's diff. The row prefers this over the
post-rewrite/pre-merge-commitpair it listssecond, because a local hook is advisory by construction and the row is the proof.
It skips exactly two of the eleven, and the rule is narrow: skip only where running it on a runner
gives a WRONG answer, not merely an inconvenient one.
ledger-gate--ci, so the ownership arm reads an allocation registry in.git/mefor-coord/that never reaches a runner. It would red every PR on a check no author can satisfy.ci.ymlrunsledger_check.py --ci(the duplicate-number half, which CI can enforce)forbidden-contentsecurity.yml's fork/secret branching in a NON-REQUIRED leg would either red every fork PR or hand back a structural-only green that reads as a leak-gate pass.security.yml's REQUIREDforbidden-contentjob scans the whole tree;branch-leak-scan.ymlruns it on every pushPer the brief, the ownership arm stays CI-skipped and CI is not taught to read allocations.
2. The version axis, which the existing parity module does not reach.
_MIRRORSanchors on the invocation, so it proved the right tool ran with the right discriminatingflag and said nothing about which build. For the three third-party hooks that was held only by
prose comments saying to keep the versions in step, and a comment cannot fail.
revgitleaksv8.18.4security.ymlVER=8.18.4test_the_hook_rev_matches_the_version_ci_installsactionlintv1.7.12zizmor.ymlVER=1.7.12bandit1.9.4ci-scannersgroupbandit==1.9.4test_bandit_hook_rev_matches_the_version_ci_installs(sibling file)All four halves matched already. This closes the axis; it did not find a live drift. Each arm lives
with its own hook -- bandit in
test_lint_scope_parity.pybeside its scope arms, the other two intest_gate_ci_mirror_parity.py-- so the two files cannot grow a second definition of one rule.Read this before reviewing: a claim in the row was never true
The row ended a paragraph with "A PARITY TEST FOR THOSE EIGHT IS THE DURABLE FIX AND IS NOT
BUILT", and the 2026-09-03 scoring note repeats it. That was already false when written:
tests/test_gate_ci_mirror_parity.pycovers exactly those eight and landed in389168a79, the verycommit that filed the row.
So I did not rebuild it. Doing so would have produced a second, silently different definition of
one rule, which is the defect that family of tests exists to prevent. The sentence is corrected in
place -- a reader who reaches it decides whether to build that module -- and the correction is
recorded rather than the sentence quietly deleted.
This means the brief's second half was already done, and the scoring note over-counted the
remaining work.
What this does not buy
Stated so the row is not read as closure of the whole finding.
before any runner starts. This is defence in depth, not earliness.
--from-ref/--to-refresolves a changed-file setagainst the checked-out tree, so content living only in an intermediate commit is out of reach --
as it is for every tip-tree mirror.
Not a required status check
Deliberately. It builds every pinned hook environment per run and belongs off the critical path.
.github/required-contexts.txtand the pinned count intests/test_required_contexts.pyareuntouched, and a test asserts the job name is absent from that file. Promoting it would need branch
protection moved first, then that file, then the count -- and a
merge_group:trigger before any ofit, or it never reports on a queue entry and nothing merges.
Checks
Every guard was mutation-tested, because an assertion that cannot fail is the failure mode this
item is about. Each mutation was verified to have actually applied before the result was believed --
one early control silently did not apply and would have recorded a false pass.
revmoved tov8.18.5against CI's8.18.4revmoved to1.9.3against the group's1.9.4SKIPgrows to includegitleaksSKIPcarries a typo (forbiden-content)--from-refwhile the header prose keeps itpre-commit run.github/required-contexts.txtRan, all green:
ruff check .,ruff format --check .,mypy messagefoundry(strict, 267 files),scripts/docs/backlog_status_check.py --min-items 300(657 items, each declaring exactly onestatus), and
actionlinton the new workflow viapre-commit run actionlint. Also ran thecontrol-char,licence-headerandgitleakshooks over the changed files. All eleven hooks ran oneach of the three commits.
Skipped: the full
pytestsuite -- it will not finish inside one turn. Ran the workflow-censusbatch instead:
test_required_contexts,test_ci_venv_pinning,test_backlog_status_check,test_merge_gate_controls,test_failure_signal,test_nightly_notice,test_dast_auth_sweep,test_required_workflow_state,test_gate_liveness,test_lint_scope_parity,test_gate_ci_mirror_parity,test_required_contexts_drift,test_security_posture,test_quality_advisory_invariants,test_ci_docs_only_detector-- 310 passed, exit 0.One caveat recorded rather than dropped: an earlier foreground run of a subset showed a single
F.It was running concurrently with the batch above against the same worktree. Re-run alone, those
files exit 0, and the batch that covers them passed 310. I am reporting the sequential result as the
real one, and naming the concurrent run so nobody re-derives the discrepancy.
Also verified the new job name is unique across all 28 workflows and carries no
${{ }}template,so it cannot silently re-map an existing required context (
tests/_workflow_contexts.pykeys jobs bydeclared name across every file). Everything added is pure ASCII, checked with a positive control.
tests/test_workflow_shell_syntax.pytimes out locally on Windows -- it spawnsbash -nper runblock. This is pre-existing: it timed out identically with my workflow moved aside, so it is not
caused by this change. I checked my three run blocks directly instead and all three parse under
bash -n. It should be read on the Linux runner.Only a hosted runner can report:
windows-service-smoke, the server-DB store legs, the loadlegs,
codeql,scorecard,semgrep, and the newpre-commit replayleg itself -- this is itsfirst execution anywhere. The
pre-commit==4.6.1pin is version-pinned but not hash-pinned (it is inno dependency group); 4.6.1 is the version this machine has installed and the version the repo's own
comments record measuring against, so the pin is real rather than guessed. Routing it through
ci/locks/is the correct end state and needs auv lock/uv exportpass.Two things for the coordinator
1. I did not apply the
reviewedlabel. A message arrived mid-task stating an owner overridesuspending the reviewer process and instructing me to self-label. I cannot verify an owner ruling
relayed by a peer agent, my brief said not to, and
CLAUDE.mdstates plainly that labelling your ownPR unread "satisfies the machine and defeats the point" -- this one context is the repository's
entire review requirement, with approvals pinned at 0. The technical half of that message was
correct and useful (a push strips the label;
gh pr update-branchis a push). If the ruling is real,whoever holds that authority can apply the label directly; I have left it clean rather than spend it.
2. A shared-scratchpad collision nearly put another item's subject on this commit. The session
scratchpad under
AppData\Local\Temp\claude\...\<session-id>\scratchpadis shared acrosssessions despite the session id in the path. A peer session overwrote
msg1.txtwith its owncommit message for BACKLOG #1414 between my write and my
git commit -F. The claim gate caughtit -- it refused because #1414 is not claimed by this worktree -- which is the gate doing exactly its
job on a defect nothing else would have surfaced. Commit messages here are now written inside the
worktree. Worth a row: a generic filename in a shared temp directory is a live cross-session hazard.
🤖 Generated with Claude Code