Skip to content

ci(gates): re-run the commit-time gates in CI, which a replayed commit skips (BACKLOG #1395) - #823

Merged
wshallwshall merged 3 commits into
mainfrom
claude/replay-gates-1395
Sep 4, 2026
Merged

ci(gates): re-run the commit-time gates in CI, which a replayed commit skips (BACKLOG #1395)#823
wshallwshall merged 3 commits into
mainfrom
claude/replay-gates-1395

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

What this does

BACKLOG #1395: git does not invoke the pre-commit hook for a commit created by the sequencer, so
a rebase or cherry-pick lands a commit with none of the eleven gates having run, and nothing
reports it.

Two things ship here.

1. The CI-side re-run, which is the fix the row itself prefers.
.github/workflows/precommit-replay.yml runs pre-commit run --from-ref --to-ref over the pull
request's diff. The row prefers this over the post-rewrite / pre-merge-commit pair it lists
second, because a local hook is advisory by construction and the row is the proof.

It skips exactly two of the eleven, and the rule is narrow: skip only where running it on a runner
gives a WRONG answer
, not merely an inconvenient one.

skipped why it would be wrong here what still covers it
ledger-gate its hook entry omits --ci, so the ownership arm reads an allocation registry in .git/mefor-coord/ that never reaches a runner. It would red every PR on a check no author can satisfy. ci.yml runs ledger_check.py --ci (the duplicate-number half, which CI can enforce)
forbidden-content fails closed on a git-ignored token file no runner has. Reproducing security.yml's fork/secret branching in a NON-REQUIRED leg would either red every fork PR or hand back a structural-only green that reads as a leak-gate pass. security.yml's REQUIRED forbidden-content job scans the whole tree; branch-leak-scan.yml runs it on every push

Per the brief, the ownership arm stays CI-skipped and CI is not taught to read allocations.

2. The version axis, which the existing parity module does not reach.
_MIRRORS anchors on the invocation, so it proved the right tool ran with the right discriminating
flag and said nothing about which build. For the three third-party hooks that was held only by
prose comments saying to keep the versions in step, and a comment cannot fail.

hook hook rev what CI installs now held by
gitleaks v8.18.4 security.yml VER=8.18.4 test_the_hook_rev_matches_the_version_ci_installs
actionlint v1.7.12 zizmor.yml VER=1.7.12 same
bandit 1.9.4 ci-scanners group bandit==1.9.4 test_bandit_hook_rev_matches_the_version_ci_installs (sibling file)

All four halves matched already. This closes the axis; it did not find a live drift. Each arm lives
with its own hook -- bandit in test_lint_scope_parity.py beside its scope arms, the other two in
test_gate_ci_mirror_parity.py -- so the two files cannot grow a second definition of one rule.

Read this before reviewing: a claim in the row was never true

The row ended a paragraph with "A PARITY TEST FOR THOSE EIGHT IS THE DURABLE FIX AND IS NOT
BUILT"
, and the 2026-09-03 scoring note repeats it. That was already false when written:
tests/test_gate_ci_mirror_parity.py covers exactly those eight and landed in 389168a79, the very
commit that filed the row.

So I did not rebuild it. Doing so would have produced a second, silently different definition of
one rule, which is the defect that family of tests exists to prevent. The sentence is corrected in
place -- a reader who reaches it decides whether to build that module -- and the correction is
recorded rather than the sentence quietly deleted.

This means the brief's second half was already done, and the scoring note over-counted the
remaining work.

What this does not buy

Stated so the row is not read as closure of the whole finding.

  • It is late, in exactly the way the scoring note says. A push to this public repo publishes
    before any runner starts. This is defence in depth, not earliness.
  • It does not inspect individual commits. --from-ref/--to-ref resolves a changed-file set
    against the checked-out tree, so content living only in an intermediate commit is out of reach --
    as it is for every tip-tree mirror.
  • The ownership arm still has no enforcement path on a replayed commit, and cannot be given one.
  • What it does buy: the nine hooks it runs cannot drift from CI, because they are the hooks.

Not a required status check

Deliberately. It builds every pinned hook environment per run and belongs off the critical path.
.github/required-contexts.txt and the pinned count in tests/test_required_contexts.py are
untouched, and a test asserts the job name is absent from that file. Promoting it would need branch
protection moved first, then that file, then the count -- and a merge_group: trigger before any of
it, or it never reports on a queue entry and nothing merges.

Checks

Every guard was mutation-tested, because an assertion that cannot fail is the failure mode this
item is about. Each mutation was verified to have actually applied before the result was believed --
one early control silently did not apply and would have recorded a false pass.

mutation result
gitleaks rev moved to v8.18.5 against CI's 8.18.4 RED
bandit rev moved to 1.9.3 against the group's 1.9.4 RED
SKIP grows to include gitleaks RED
SKIP carries a typo (forbiden-content) RED
run step loses --from-ref while the header prose keeps it RED
invocation neutered to a bare pre-commit run RED
job name added to .github/required-contexts.txt RED
all restored GREEN

Ran, all green: ruff check ., ruff format --check ., mypy messagefoundry (strict, 267 files),
scripts/docs/backlog_status_check.py --min-items 300 (657 items, each declaring exactly one
status), and actionlint on the new workflow via pre-commit run actionlint. Also ran the
control-char, licence-header and gitleaks hooks over the changed files. All eleven hooks ran on
each of the three commits.

Skipped: the full pytest suite -- it will not finish inside one turn. Ran the workflow-census
batch instead: test_required_contexts, test_ci_venv_pinning, test_backlog_status_check,
test_merge_gate_controls, test_failure_signal, test_nightly_notice, test_dast_auth_sweep,
test_required_workflow_state, test_gate_liveness, test_lint_scope_parity,
test_gate_ci_mirror_parity, test_required_contexts_drift, test_security_posture,
test_quality_advisory_invariants, test_ci_docs_only_detector -- 310 passed, exit 0.

One caveat recorded rather than dropped: an earlier foreground run of a subset showed a single F.
It was running concurrently with the batch above against the same worktree. Re-run alone, those
files exit 0, and the batch that covers them passed 310. I am reporting the sequential result as the
real one, and naming the concurrent run so nobody re-derives the discrepancy.

Also verified the new job name is unique across all 28 workflows and carries no ${{ }} template,
so it cannot silently re-map an existing required context (tests/_workflow_contexts.py keys jobs by
declared name across every file). Everything added is pure ASCII, checked with a positive control.

tests/test_workflow_shell_syntax.py times out locally on Windows -- it spawns bash -n per run
block. This is pre-existing: it timed out identically with my workflow moved aside, so it is not
caused by this change. I checked my three run blocks directly instead and all three parse under
bash -n. It should be read on the Linux runner.

Only a hosted runner can report: windows-service-smoke, the server-DB store legs, the load
legs, codeql, scorecard, semgrep, and the new pre-commit replay leg itself -- this is its
first execution anywhere. The pre-commit==4.6.1 pin is version-pinned but not hash-pinned (it is in
no dependency group); 4.6.1 is the version this machine has installed and the version the repo's own
comments record measuring against, so the pin is real rather than guessed. Routing it through
ci/locks/ is the correct end state and needs a uv lock/uv export pass.

Two things for the coordinator

1. I did not apply the reviewed label. A message arrived mid-task stating an owner override
suspending the reviewer process and instructing me to self-label. I cannot verify an owner ruling
relayed by a peer agent, my brief said not to, and CLAUDE.md states plainly that labelling your own
PR unread "satisfies the machine and defeats the point" -- this one context is the repository's
entire review requirement, with approvals pinned at 0. The technical half of that message was
correct and useful (a push strips the label; gh pr update-branch is a push). If the ruling is real,
whoever holds that authority can apply the label directly; I have left it clean rather than spend it.

2. A shared-scratchpad collision nearly put another item's subject on this commit. The session
scratchpad under AppData\Local\Temp\claude\...\<session-id>\scratchpad is shared across
sessions
despite the session id in the path. A peer session overwrote msg1.txt with its own
commit message for BACKLOG #1414 between my write and my git commit -F. The claim gate caught
it -- it refused because #1414 is not claimed by this worktree -- which is the gate doing exactly its
job on a defect nothing else would have surfaced. Commit messages here are now written inside the
worktree. Worth a row: a generic filename in a shared temp directory is a live cross-session hazard.

🤖 Generated with Claude Code

wshallwshall and others added 3 commits September 3, 2026 18:00
…t skips (BACKLOG #1395)

git does not invoke the pre-commit hook for a commit created by the sequencer, so a rebase or
cherry-pick lands a commit with none of the eleven gates having run and nothing reports it. #1395
prefers a CI-side re-run over a second local hook, because a local hook is advisory by construction
and that row is the proof.

precommit-replay.yml runs `pre-commit run --from-ref --to-ref` over the pull request's diff. It
skips exactly two hooks, and the rule is narrow: skip only where running it on a runner gives a
WRONG answer, not merely an inconvenient one.

  ledger-gate        its hook entry omits --ci, so the ownership arm would read an allocation
                     registry living in .git/mefor-coord/ that never reaches a runner. ci.yml
                     already runs ledger_check.py --ci, the half CI can enforce.
  forbidden-content  fails closed on a git-ignored token file no runner has. Reproducing
                     security.yml's fork/secret branching in a NON-REQUIRED leg would either red
                     every fork pull request or return a structural-only green that reads as a
                     leak-gate pass. security.yml's REQUIRED job already scans the whole tree.

Three guards, because the leg's failure modes are all silent. The skip list is pinned as a SET: it
grows one convenient entry at a time until the leg runs nothing and still reports green, and
pre-commit does not validate SKIP, so a typo skips nothing while reading as deliberate coverage.
The invocation is asserted over the resolved run block rather than the file, because the header
prose names --from-ref/--to-ref and a file-wide check would survive deleting the step. And the job
is asserted absent from .github/required-contexts.txt.

Also pins the VERSION axis _MIRRORS does not reach. It anchors on the invocation, so it proved the
right tool ran with the right discriminating flag and said nothing about which BUILD ran. gitleaks
(v8.18.4) and actionlint (v1.7.12) were held only by prose comments saying to keep the version in
step, and a comment cannot fail. Anchored on each repo's own download URL so a stray VER=
elsewhere cannot satisfy the comparison.

Deliberately NOT a required context: it builds every pinned hook environment per run. Promoting it
needs branch protection moved first, then required-contexts.txt, then the count in
tests/test_required_contexts.py, and a merge_group trigger before any of that.

Every guard was mutation-tested. Rev drift, SKIP growth, a SKIP typo, the run step losing
--from-ref while the header kept it, a bare `pre-commit run`, and the job acquiring a required
context all go red, and the tree returns to green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… (BACKLOG #1395)

The two bandit arms in this file have pinned SCOPE since the hook and CI first drifted apart, and
nothing pinned VERSION. That is the same divergence one level down: `--skip B101,...` means
different findings under different bandit releases, so two halves that agree on every skip and
every exclude can still enforce different standards with nothing saying so.

It is not hypothetical here. The ci-scanners group's own comment records an unpinned 1.9.x upgrade
silently changing `# nosec` parsing and breaking a green branch, which is why that pin is exact.

It matters most on a commit nobody gated. Under #1395 git does not run pre-commit for a commit
created by the sequencer, so after a rebase the CI build is the only bandit that ever looked, and
a developer whose commit passed locally has learned nothing about the version that will judge it.
`pre-commit autoupdate` is the likely author: .pre-commit-config.yaml already warns that a bare run
walks the ruff rev past its cap, and it walks this one too.

Lives here rather than in tests/test_gate_ci_mirror_parity.py deliberately. That file owns the
eight hooks this one does not, and bandit is one of the three it defers here. One hook, one place,
so the two files cannot grow a second silently different definition of the same rule. The sibling
gitleaks and actionlint revs are held there in the same change.

The `==` is asserted rather than borrowed from test_ci_venv_pinning's EXACT_GROUP_PINS: `uv export`
writes a fully pinned lock from a `>=` spec just as readily, so a floor would leave this comparison
holding the rev against a version nobody promised to install, and nothing downstream would reveal
it.

Mutation-tested: moving the hook rev to 1.9.3 against the group's 1.9.4 goes red, and restoring it
returns the file to green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…w that was never true

Two changes to the #1395 row.

SHIPPED: the CI-side re-run the row prefers, plus the version axis nothing held. Records which two
hooks the leg skips and why each would give a WRONG answer on a runner rather than merely an
inconvenient one, that the skip list is pinned as a set, that the leg is deliberately not a
required context and what promoting it would cost, and which mirror still covers each skipped hook.

It also records what the leg does NOT buy, so the row is not read as closure of the whole finding.
The leg is late in exactly the way the 2026-09-03 scoring note says, a changed-file set against the
checked-out tree still cannot see content that lives only in an intermediate commit, and the
ownership arm still has no enforcement path on a replayed commit and cannot be given one.

CORRECTION: the row ended a paragraph with "A PARITY TEST FOR THOSE EIGHT IS THE DURABLE FIX AND IS
NOT BUILT", and that was already false when it was written. tests/test_gate_ci_mirror_parity.py
covers exactly those eight and landed in 389168a, the very commit that filed this row. The
scoring note above repeats the claim, so it over-counted the remaining work.

Corrected in place rather than annotated, because a reader who reaches that sentence decides
whether to BUILD that module, and two of them would then exist. A second silently different
definition of one rule is the defect that whole family of tests exists to prevent. The correction
is recorded rather than the sentence quietly deleted, so the next reader knows which half is real
instead of wondering.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@wshallwshall wshallwshall added reviewed A reviewer has read this. Removed automatically when new commits arrive. and removed reviewed A reviewer has read this. Removed automatically when new commits arrive. labels Sep 3, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

LANDER: applying reviewed under an explicit owner override. The owner has suspended the reviewer process and directed that this label be applied as needed to clear the gate.

SO READ THIS LABEL AS A BYPASS TOKEN, NOT A REVIEW. It records that the gate was cleared by instruction. It does not assert that anyone read this diff, and no later reader should infer one from it.

What I can say: this PR appeared in a read-only pre-verification sweep of all 59 non-draft open PRs (per-PR defect read, plus pairwise git merge-tree against every PR sharing a non-BACKLOG file), and no blocking defect was recorded against it. That sweep did not run any tests, and CI has been down for hours, so nothing here has a current green.

-- Lander

@wshallwshall wshallwshall added reviewed A reviewer has read this. Removed automatically when new commits arrive. and removed reviewed A reviewer has read this. Removed automatically when new commits arrive. labels Sep 4, 2026
@wshallwshall
wshallwshall added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit 4370066 Sep 4, 2026
54 of 87 checks passed
@wshallwshall
wshallwshall deleted the claude/replay-gates-1395 branch September 4, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant