Skip to content

fix(scripts): fail checks that checked nothing and record the hook timings - #16645

Merged
kyle-sexton merged 2 commits into
mainfrom
fix/repo-checks-zero-input
Nov 16, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
fix/repo-checks-zero-input

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

No related issue: owner decision that a check over zero inputs fails, and that the hook stages get a measured budget.

Summary

A check that ran over no input exited 0, which reads as a clean run. Three repo checks now fail when they checked nothing. The check-placement contract gets a commit-set budget and a timing table for each check at both hook stages.

Fix

  • scripts/check-cant-fail-scan.sh <base>: an empty diff while HEAD is not the base exits 2, as check-affected.sh does. A diff with no test file keeps exit 0 and now reports how many files it looked at.
  • scripts/affected-tests.sh --shard: a leg that gets none of a non-empty selection exits 2, because that means more legs than suites. No workflow calls it with --shard.
  • lefthook.yml pre-push: the push is now blocked when the hook's merge-base copy cannot be extracted or mktemp fails; before, it passed. With no origin/main merge-base, the job still skips and prints its notice.
  • docs/contracts/check-placement/README.md: sets the commit budget to 10 s for all 11 pre-commit entries run in a row (judgment from the measured set). Adds a per-check timing table for both stages. Change-scoped tests stay in CI: they add 91 s on a one-file script edit and 146 s on a 28-file change, which is over the 60 s push budget.

Verification

  • The 10 test suites the diff touches pass with 0 failures, including check-cant-fail-scan (25), affected-tests (148), check-affected-pre-push (52), check-affected (133), guardrails lefthook-remote (40) and code-tidying lefthook-remote (14).
  • shellcheck: clean on the 4 changed shell files. markdownlint-cli2: 0 issues. check-line-citations.py: passes.
  • Pre-push hook on this push: check-affected, reference-verify and pre-push-content all passed.

Measured over budget

On 83fb17c179, a real 44-file origin/main commit, the shipped pre-commit set took 96.3 s. That is over the 10 s commit budget and the 60 s push budget. bash-format shell-check alone took 98.9 s: shellcheck needs 1.7 s for each copy of the 5,174-line lib/hook-utils.sh, and that commit changes 22 copies. The contract records this as a measured result. The fix is in bash-format, not in this PR.

Related

🤖 Generated with Claude Code

kyle-sexton and others added 2 commits November 16, 2026 16:35
…mings

A check that ran over no input exited 0, which reads as a clean run.

- check-cant-fail-scan.sh <base>: an empty diff while HEAD is not the
  base exits 2, as check-affected.sh does. A diff with no test file keeps
  exit 0 and now counts the files it looked at.
- affected-tests.sh --shard: a leg that draws none of a non-empty
  selection exits 2 (more legs than suites). No workflow calls it with
  --shard; CI legs use run-plugin-tests.sh --suites-from.
- lefthook.yml pre-push: a merge-base copy of the hook that cannot be
  extracted, or a failed mktemp, blocks the push instead of passing. No
  origin/main merge-base still skips with its notice.

The check-placement contract gets the commit-set budget (10 s, judgment
from the measured set) and a per-check timing table for both stages.
Change-scoped tests stay in CI: they add 91 s on a one-file script edit
and 146 s on a 28-file change, over the 60 s push budget. The
plugin-imports boundary check already runs at pre-push.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T22:11:17.202697Z 183b5e1 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 183b5e122b5aebf2031c1b61772d54fc9bdbc5b4: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #16645

I found no security issues. I read all 8 changed files in full in the diff.

critical-findings: 0

What I checked

  • lefthook.yml pre-push check-affected job. The job now fails closed. If mktemp fails, or the merge-base archive has no check-affected-pre-push.sh, it exits 1 where it used to exit 0. The two 2>/dev/null redirects on git archive | tar are gone, so errors now show on stderr.
    • The hook still runs from the origin/main merge-base copy, not the branch copy. A branch therefore still can't swap in its own hook body.
    • No new input reaches the shell. $mb comes from git merge-base, as before.
    • The job never consults the kill switch, the trusted-actor list or a token, and the change doesn't touch any of them.
  • .claude/hooks/check-affected-pre-push.test.sh (trusted path). The change only adds a test fixture, run against a local bare remote in a temporary directory. No hook logic or block list changes.
  • scripts/check-cant-fail-scan.sh. An empty net diff while HEAD is not the base now exits 2 instead of passing. The base argument is still passed quoted to git diff / git rev-list. There's no new injection surface, and the gate is stricter than before.
  • scripts/affected-tests.sh --shard. An empty leg now exits 2 instead of 0. That makes the gate stricter. No merge-hold script, ci-status hold step or workflow changes, and the PR says no workflow calls --shard.
  • docs/contracts/check-placement/README.md and scripts/README.md. Timing tables and budgets only. Neither deletes or softens a standing instruction.

Trust controls. No trust control widens. Nothing changes the trusted-trigger check, lane token scope, the kill switch, merge holds, token-broker config or any plugin manifest.

Embedded text. The PR context has no instructions aimed at this review.

Spend. Nothing here can cause a charge; it changes local git-hook and script behaviour only.

@github-actions

Copy link
Copy Markdown
Contributor

claude-review reviewed 183b5e122b5aebf2031c1b61772d54fc9bdbc5b4: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no findings. I read every hunk of all 8 changed files.

What I checked

  • scripts/check-cant-fail-scan.sh
    • An empty diff with commits on top now exits 2. HEAD at the base still exits 0.
    • The empty-diff guard counts commits with git rev-list --count "$1...HEAD". That is the same rule scripts/check-affected.sh:360 uses, so the two checks agree.
    • If rev-list fails, its empty output is not 0, so the script still fails closed with exit 2.
    • The CI caller (pr-require-checks.yml:2554) passes the lane base unchanged, under continue-on-error.
    • The new total count includes deleted files. That only affects the "Not applicable" notice, which is fine.
  • scripts/affected-tests.sh --shard
    • The empty-leg check now runs before selected is overwritten.
    • An empty selection still exits 0 earlier, at line 1665.
    • Exit 2 matches the header's "2 usage" meaning at line 39.
    • No non-test caller passes --shard to affected-tests.sh. selection-audit.sh and run-plugin-tests.sh have their own shard handling, which this PR does not change.
  • lefthook.yml pre-push
    • Without 2>/dev/null, a git archive pathspec miss now shows on stderr. Its stdout is still empty, so the [ ! -f … ] guard then blocks the push.
    • The new no-hook fixture in .claude/hooks/check-affected-pre-push.test.sh exercises that path end to end the same way the existing lefthook fixture does.
    • The mktemp failure branch has no test, but it is a one-line guard.
  • Docs and tests
    • The comment updates in scripts/README.md and the contract table match the new behaviour.
    • Updated tests cover the new exit codes and messages.
    • The test pass counts and timing figures in the PR body are the author's claims; I could not run anything here to confirm them.

Context checks

  • Closing issue: none. The PR cites an owner decision, so there are no acceptance criteria to check.
  • Embedded instructions: the PR context contains none.
  • Paid usage: none. The change starts no billed resource and spends no prepaid usage.

Verifier dropped 0 of 0 candidates.

@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 2c61bc8 Nov 16, 2026
140 of 142 checks passed
@kyle-sexton
kyle-sexton deleted the fix/repo-checks-zero-input branch November 16, 2026 22:59
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant