fix(gate): run-tests.sh could not check its own most important precondition

[ZacxDev]

1. REQUIRED_TOOLS could not express its own most important precondition

It is a list of binaries checked with command -v. pytest is not a binary this runner calls — it is a module (python -m pytest) — so the one guard whose entire job is "the thing that runs the tests is present" was structurally unable to check pytest. It also asserted python3 while the runner actually invokes python.

Measured on the dev host with every REQUIRED_TOOLS binary present but no pytest importable — all 17 targets printed:

run-tests: ERROR — could not parse pytest's summary for <dir>.

and the run ended TOTAL collected=0 … RESULT: FAIL, exit 1.

Correction to the briefed severity

The gate did not go green, and the briefed "reports per-target PASS with collected=0" does not exist on this revision — GUARD 4 (unparseable summary) and GUARD 3 (the collected floor) both fire. The defect is diagnostic: seventeen copies of a message blaming pytest's output format for a missing dependency, pointing at the wrong subsystem entirely. That is the #276 shape — a real finding that reads like an environment fault.

Now a single named FATAL, exit 2, before any suite starts.

2. declare -a + set -u — a guard's own error path was broken

declare -a RESULTS / declare -a SKIP_LINES leave the arrays declared but unset. Under set -u the first ${#arr[@]} on a still-empty array aborts the command with unbound variable (measured, bash 5.3.15). With zero skips this printed a raw

scripts/run-tests.sh: line 479: SKIP_LINES: unbound variable

where GUARD 2's skip list belonged, and the unpinned-skip loop below it never executed. There is no set -e, so the script continued and the skip-total accounting still fired — the damage was confined to the diagnostic path, at exactly the moment someone is reading why the gate is red. Fixed with NAME=().

Not in the brief; found while reproducing hole 1.

3. "printed RESULT: FAIL and exited 0" — did NOT reproduce

Measured: exit 1. The structure forbids it — RESULT: FAIL is printed only when fail != 0, and the very next statement is exit "$fail", whose only non-zero value is 1. The most likely origin is reading the status through a pipeline (| tail), which yields the last command's status rather than the runner's — the trap the brief itself warns about.

Pinned end-to-end anyway by test_a_failing_run_never_exits_zero, which forces a red run and asserts the two can never disagree — and pytest.fail()s if it could not force one, so it cannot pass vacuously.

Known-bad states — proven LOUD by before/after, not by reading code

state	before	after
no pytest module	17 misleading "could not parse pytest's summary", exit 1	one named FATAL, exit 2, no suite started
target dir exists but is empty	per-directory collected 0 tests floor → RESULT: FAIL	unchanged (proven reachable)
typo'd target entry	GUARD 5 aborts naming the entry + "does not exist"	unchanged (proven reachable)

The last two were already guarded before this PR — nothing had ever proven those paths could fire. Labelled REACHABILITY proofs, not regression coverage.

Measured — both tiers

tier	command	result
authoritative	nix build .#checks.x86_64-linux.pytests	5800 collected / 5799 passed / 1 skipped / 0 failed, RESULT: PASS, exit 0
dev host	scripts/tests/test_run_tests_preconditions.py + test_runtime_shebangs.py under nix-shell	15 passed

Delta accounted exactly: baseline 5792 (origin/main 13bc8bd, sandbox) + 8 new tests = 5800. Skips unchanged at 1.

The sandbox caught a defect the dev host could not

First sandbox run was RED (5800 collected / 1 failed). The repo-wide runtime-shebang scanner added by #306 rejected this file: the shim helper wrote its own #!{shutil.which("bash")}. My dev-host run was green because I ran this one file, and the scanner lives in another — a per-file run structurally cannot see a repo-wide guard. Fixed by routing the shim through scripts/testlib/mockbin.write_exec (ec63d11), which is exactly the helper #306 created for this.

Recording it because it is the two-tier lesson landing on the PR that is about two-tier honesty, and because the green I would otherwise have reported was a claim about a subset.

Red/green matrix

scripts/tests/test_run_tests_preconditions.py, base ref origin/main (13bc8bd), measured by swapping origin/main's runner into place:

revision	result
origin/main runner	3 failed / 5 passed
this branch	0 failed / 8 passed

The 3 reds are exactly the three regression tests (test_python_is_a_required_tool, test_missing_pytest_module_is_named, test_empty_arrays_are_initialised_assigned). The 5 greens at base are the reachability proofs and controls, which should pass at base because those guards pre-existed — labelled as such in the module docstring rather than counted as coverage for this change.

Control pairs

• test_the_runner_copy_helper_actually_works — positive control on the mutation helper. Every known-bad-state test rewrites HERMETIC_TARGETS in a copy; if that regex stopped matching, the copies would silently be the unmodified runner and each test would exercise the real 17-target list instead. It asserts the rewrite both applies and takes effect (all 1 hermetic target(s) resolve).
• test_set_u_empty_array_mechanism_holds — positive control on the bash behaviour the source pin describes. Asserts both halves: declare -a errors, =() succeeds. Without it the pin could guard a non-issue on a bash where neither form errors.
• test_a_failing_run_never_exits_zero fails loudly if it cannot force a red run, rather than passing on an absence.

Also found, deliberately NOT fixed

scripts/claude-hooks/tests/test_bash_guard.py prints its own verdict line as RESULT: all good, which collides with the runner's own RESULT: PASS/RESULT: FAIL. Any grep -E "RESULT:" over a gate log returns two different subsystems' verdicts interleaved, and the hook test's line appears before the runner's. It cost real confusion while measuring this PR. Left alone because renaming a hook test's output is orthogonal to this change and belongs with whoever owns those scripts — flagging rather than silently widening the diff.

Audit of the remaining exit paths

Asked the wider question — what else can this runner do that produces reassuring output without having tested anything? Findings beyond the above:

• run_pytest is called in the parent shell (not a subshell), so fail=1 genuinely persists; no lost-status hazard there.
• The HOOK_TESTS loop's || continue was already converted to a loud failure by a prior PR; verified still loud.
• _count_of returns 0 for an absent word, so a summary naming only failed still yields collected >= 1 and cannot silently pass the per-directory floor.
• --check-targets deliberately zeroes missing_tools and now also bypasses the pytest precondition — correct (it runs no tests), and pinned by test_check_targets_is_cheap_and_runs_no_tests from the earlier PR.

Files touched

• scripts/run-tests.sh
• scripts/tests/test_run_tests_preconditions.py (new)

No overlap with nix/home.nix, scripts/bar-status-poll, or scripts/collector/**.

🤖 Generated with Claude Code

[ZacxDev]

Gated on the MERGED tree, not just this branch

Per RULES.md — a PR green on its own branch proves nothing about the tree its merge creates. Built integration/309-310 off origin/main (13bc8bd), merged both #309 and #310, and ran the full authoritative gate there:

check	result
nix build .#checks.x86_64-linux.pytests	5810 collected / 5809 passed / 1 skipped / 0 failed — RESULT: PASS
nix build .#checks.x86_64-linux.nodetests	997 tests / 997 pass / 0 fail (3 suites, 29 files) — RESULT: PASS

Combined exit 0.

Delta accounted exactly: 5792 (baseline) + 10 (#309's test_run_node_tests_suites.py) + 8 (#310's test_run_tests_preconditions.py) = 5810. Node: 468 (baseline) + 508 + 21 = 997. No unexplained gap on either.

git merge-tree --write-tree between the two branches exits 0 (branched on the exit code, not a marker grep — merge-tree prints only a tree OID on success and emits no conflict markers either way). The branches also share no files: #309 touches scripts/run-node-tests.sh, flake.nix, scripts/tests/test_run_node_tests_suites.py; #310 touches scripts/run-tests.sh, scripts/tests/test_run_tests_preconditions.py. The only semantic coupling is that both add a file under scripts/tests/, which is a HERMETIC_TARGETS entry — and that is what the 5810 total confirms.