From ba531db8fbbe0dec5f4939a9eca4bc725811d853 Mon Sep 17 00:00:00 2001 From: thekevinbot <248217931+thekevinbot@users.noreply.github.com> Date: Fri, 4 Sep 2026 16:52:20 -0400 Subject: [PATCH] docs: state the observed preflight step count and how to recount it The three places quoting 40 pairs predate #973 and #1019. The dry-run matrix now emits 50 '==>' steps (42 gate invocations + 8 python drift guards) plus 3 SKIPped packaging pairs, so each site now names what it counts and points at 'just preflight --dry-run | grep -c "^==>"'. skip-changelog: docs-only, no package source touched --- AGENTS.md | 2 +- agents/reference/testing-gates.md | 4 +++- justfile | 13 ++++++++----- 3 files changed, 12 insertions(+), 7 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 543db993b..6a4d8c054 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -121,7 +121,7 @@ A feature is not done until integration tests pass and cover the new functionali The colocation rule is a blocking CI gate ([`testing-conventions`](https://github.com/thekevinscott/testing-conventions), wired in the per-domain CI workflows + `testing-conventions.toml`): every source file needs a colocated unit test (Rust: inline `#[cfg(test)]`), a PR-only co-change check flags modified sources whose test didn't change, and `unit lint` enforces **isolation** -- a unit test must mock every collaborator (Python `patch(...)`, TS `vi.mock`, Rust trait double or relocate the effectful test), never weaken the test. The gates also cover the two binding crates and the repo-tooling `internals/checks` / `internals/distcheck` uv packages. **There is exactly one exemption**: `no-first-party-patch` on the two `packages/python/tests/integration/hermetic/` suites, waived because patching the first-party `_RustDirSQL` core boundary *is* what makes that tier hermetic -- there is no third-party target the rule could redirect it to, so the rule does not describe that seam. Everywhere else the count stays zero and an exemption is never the escape hatch: barrels get a colocated surface test, logic gets extracted, dead shells get deleted. Full wiring, per-package gate map, and history: `agents/reference/testing-gates.md`. -**Run every gate CI declares before pushing** with `just preflight` (#781), which *derives* the (source, gate) pairs from the CI workflows instead of restating them -- 40 pairs across 8 scan roots, versus the 6 across 3 the four hand-written recipes it replaced covered. It reads every `.github/workflows/*.yml` and folds together each job that calls the reusable workflow, so a lane added in a new per-domain file is a pair it runs. It also encodes the invocations that differ from the naive one (python suites via the package venv, typescript via `npx`, per-gate option support), each of which used to pass locally while failing in CI. `--dry-run` prints the matrix; `--gate ` and `--conventions ` narrow it (both repeatable); `packaging` reports SKIP since it needs a built artifact (`just test-packaging`). +**Run every gate CI declares before pushing** with `just preflight` (#781), which *derives* the (source, gate) pairs from the CI workflows instead of restating them -- 50 steps across 8 scan roots today (42 gate invocations plus 8 python drift guards; 3 further `packaging` pairs report SKIP), versus the 6 across 3 the four hand-written recipes it replaced covered. Treat that 50 as a snapshot and read the live number off the tool: `just preflight --dry-run | grep -c '^==>'`. It reads every `.github/workflows/*.yml` and folds together each job that calls the reusable workflow, so a lane added in a new per-domain file is a pair it runs. It also encodes the invocations that differ from the naive one (python suites via the package venv, typescript via `npx`, per-gate option support), each of which used to pass locally while failing in CI. `--dry-run` prints the matrix; `--gate ` and `--conventions ` narrow it (both repeatable); `packaging` reports SKIP since it needs a built artifact (`just test-packaging`). ### Mutation (testing-conventions) diff --git a/agents/reference/testing-gates.md b/agents/reference/testing-gates.md index 36d3c699b..725cc3c76 100644 --- a/agents/reference/testing-gates.md +++ b/agents/reference/testing-gates.md @@ -27,7 +27,9 @@ the CI workflows rather than restating them, reading every `.github/workflows/*. and folding together each job that calls the reusable workflow -- so a caller added in a new per-domain file is a pair it runs. It replaced four hand-written recipes that between them named 6 pairs across 3 of the 8 scan roots -- so a locally-green -run said nothing about the other 34 pairs, which is how the #777 failures reached CI. It also encodes the invocations +run said nothing about the rest of the matrix, which is how the #777 failures +reached CI. For today's size, count it rather than trusting a number written +down here: `just preflight --dry-run | grep -c '^==>'`. It also encodes the invocations that differ from the naive one, each of which was a silent false pass before: python `mutation` / `unit-coverage` run through the package's own venv (#706), typescript through `npx` (only the npm CLI appends `--ts-mutation-adapter`), rust diff --git a/justfile b/justfile index 41c28ec03..e5ff37e68 100644 --- a/justfile +++ b/justfile @@ -75,11 +75,14 @@ e2e-attest-ts: # Every testing-conventions gate CI declares, derived from the (source, gates) # pairs of every workflow in .github/workflows that calls the reusable workflow -# -- 40 pairs across 8 scan roots in 6 workflows today (#781, #973). Supersedes -# the hand-restated `test-conventions` / `unit-coverage` / `mutation` / -# `e2e-verify` recipes, which named 6 pairs across 3 roots and drifted from the -# workflow. `--conventions` narrows it to named workflows (repeatable), `--gate` -# to named gates; `--dry-run` prints the matrix without running it. Needs the +# -- 50 steps across 8 scan roots in 6 workflows today (#781, #973): 42 gate +# invocations plus the 8 python drift guards below, with 3 further `packaging` +# pairs listed as SKIP. That count drifts; read it off the tool instead -- +# `just preflight --dry-run | grep -c '^==>'`. Supersedes the hand-restated +# `test-conventions` / `unit-coverage` / `mutation` / `e2e-verify` recipes, +# which named 6 pairs across 3 roots and drifted from the workflow. +# `--conventions` narrows it to named workflows (repeatable), `--gate` to named +# gates; `--dry-run` prints the matrix without running it. Needs the # native build first for the suite-executing gates (maturin / napi); run # `just build` if missing. #