fix(gate): add nix to the pytests check — 10 handle tests could not run in the sandbox

[ZacxDev]

nix flake check is green end to end with this. Third and last of the three regressions #276 shipped.

The bug

scripts/tests/test_opencode_config.py's nix_eval() shells out to nix-instantiate --eval to pin the generated handle values from nix/agent-handles.nix, and calls pytest.fail() — not skip — when the binary is absent. That is deliberate and correct:

"a skip here is how a wrong kubeconfig path ships"

nix was not in the pytests check's nativeBuildInputs, so all 10 test_handles_resolve_to_the_exact_expected_paths[...] cases failed in the nix sandbox and passed on every dev host.

It was invisible until #289 — it hid behind the gate being red for an unrelated reason. Its sibling is the exact complement: session_insight's test_patterns_cover_bash_guard fails only where ~/.claude/hooks/bash-guard.py is deployed, and skips in the sandbox. So #276's three regressions each concealed the others; that lesson is in #287.

Measured, not predicted

I argued in review that agent-handles.nix is { home }: {…} with no nixpkgs import and no fetch, so --eval is a pure evaluation needing no daemon, network or store realisation. That was a prediction. This is the check's own output:

	collected	passed	failed
scripts/tests before	959	949	10
scripts/tests after	959	959	0

TOTAL collected=4353  passed=4351  skipped=2  failed=0   (floor: 2850)
---- skips (pinned: 2) ----   → observed 2, reconciled
RESULT: PASS

nodetests:  files=14  tests=468  pass=468  fail=0  skipped=0   (floor: 450)
nix flake check → all checks passed

Identical collected count with failures to zero — the 10 pass, they did not merely stop running. That was the distinction asked for, and a count is the only thing that shows it: a target that is accepted but not executed looks the same from outside.

The diagnosis was confirmed by a positive control first, on a dev host: running that file with PATH stripped to python alone reproduced exactly 10 failed, 455 passed, naming the same 10 tests.

Cost, stated as deliberately as the nodejs paragraph above it

• +30 store paths, 22 of them the nix closure — nix-{util,store,expr,main,flake,fetchers,cmd} including -dev outputs, plus boost and libarchive.
• A nixpkgs bump that moves nix now invalidates this check's build cache — the same trade already accepted for nodejs.

Bought: 10 tests that structurally cannot run without it, pinning that the handles every agent shell exports ($DEVRC, $HOMELAB, $KC_*, …) resolve to the exact expected paths.

Rejected alternative — DEVHOST_TARGETS. It keeps the closure small, but weakens the pin to "runs only where someone remembers to run it", which is precisely the failure mode this whole area keeps hitting. Preserving fail-not-skip was the point.

Also

nix-instantiate added to REQUIRED_TOOLS, so a missing binary becomes one named precondition failure rather than 10 unexplained assertion failures deep in the run — that guard's entire purpose. It binds the pre-push tier too, which is safe here: both hosts are NixOS.

Scope

Branched off 88eb6d0, not stacked on anything — it touches flake.nix where #289 touched run-tests.sh. Deliberate: stacking buys nothing for orthogonal changes and carries the gh pr merge --delete-branch hazard that auto-closes a child PR GitHub then refuses to reopen.

Process note

The wrapper around this run reported exit 0 for the pipeline while the meaningful status lived in the log — $? read the trailing echo. Every number above is read from the build log, not from an exit status. Same class as the rc=$? case documented in #287.

🤖 Generated with Claude Code

[ZacxDev]

Also green on the merged tree with #291

Beyond this PR's own green (collected=4353 passed=4351 skipped=2 failed=0, scripts/tests 959/959), I built an integration branch off main merging this PR + #291 and ran the gate there:

PASS  scripts/tests                                  (collected=959 passed=959 skipped=0)
PASS  scripts/session-analysis/session_insight/tests (collected=57  passed=57  skipped=0)
TOTAL collected=4353  passed=4352  skipped=1  failed=0   (floor: 2850)
RESULT: PASS   |   nix flake check -> all checks passed

Both PRs touch scripts/run-tests.sh — this one adds nix-instantiate to REQUIRED_TOOLS, #291 removes a conditional EXPECTED_SKIPS entry. Different regions, ort auto-merged, and I read the merged result to confirm it is semantically coherent rather than just textually clean.

So the two are safe to land in either order as a pair; on current main, #291 alone would leave the sandbox red for this PR's reason, so this one should land first.