Skip to content

fix(checks): derive preflight's gate matrix from every workflow that calls testing-conventions - #983

Merged
thekevinscott merged 4 commits into
mainfrom
claude/tackle-973-4e1epu
Aug 20, 2026
Merged

thekevinscott merged 4 commits into
mainfrom
claude/tackle-973-4e1epu

Conversation

@thekevinbot

@thekevinbot thekevinbot commented Aug 20, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #973

What

just preflight died immediately on main with a bare FileNotFoundError: dirsql-checks preflight still defaulted --conventions to .github/workflows/conventions.yml, which #861 deleted when the #834 epic split it into six per-domain workflows.

Pointing the default at one surviving file would have traded the crash for something worse — a green preflight covering one of eight lanes — so the matrix is now discovered: --conventions is repeatable and defaults to nothing, and with nothing named, every workflow in .github/workflows/ that calls the reusable workflow is read and their callers concatenated (issue option 1). That is 40 pairs across 8 scan roots in 6 workflows today, where the old default resolved to 0.

Discovery rather than a list of six names for the same reason #781 derived the pairs from the workflow in the first place: a name is a restatement the next rename invalidates. The option-3 guard rides along — every unresolvable case names the fix instead of raising.

Changes

  • preflight/matrix.py — sources() (named workflows, else discovery), discovered(), named(), read_text(), NoGateMatrix; parse_gate_matrix tolerates documents with no jobs, since discovery hands it every file in the directory.
  • preflight/cli.py — --conventions repeatable with no default; announces which workflows the matrix came from; an unresolvable matrix prints preflight: followed by the fix and exits 1.
  • preflight/gate.py — run(workflows: Sequence[str], ...) builds roots from every workflow rather than one document.
  • .github/workflows/internals-checks-ci.yml — new internals-checks-integration job. Nothing ran this package's integration tier in CI, which is precisely why the tier that reads real workflow files could not have caught the rename; the issue asks for a test that would catch the next one.
  • Docs: the preflight recipe comment in justfile and internals/checks/README.md.

The docs sweep for AGENTS.md / agents/reference/* is left to #932 item 1, per the issue's Notes. Heads-up for whoever lands second: #985 edits the same two lines of internals/checks/README.md and the same justfile comment.

Red/green

  1. 1ded91c — RED tests pushed alone. CI went red for the right reason: Internals checks CI → internals/checks integration tests failed on it_derives_a_matrix_with_no_arguments with assert 1 == 0 (FileNotFoundError from the command); every other job in that run passed. Run 32406989719
  2. d750069 — implementation + colocated unit tests, flipping it green.
  3. 4ffd060 — kills the one mutant the gate surfaced (* → / on the sources seams).

New tests: tests/integration/preflight_test.py (the command with its real defaults over the real .github/workflows/ — covers every caller job, spans every file holding one, and names a missing target instead of raising) and tests/e2e/preflight_test.py (same two properties through the packaged console script from the repo root).

Manual verification

$ uv run --project internals/checks dirsql-checks preflight --dry-run | head -3
preflight: gate matrix from .github/workflows/dirsql-python-ci.yml, .github/workflows/dirsql-rust-ci.yml, .github/workflows/dirsql-typescript-ci.yml, .github/workflows/internals-checks-ci.yml, .github/workflows/internals-distcheck-ci.yml, .github/workflows/plugin-dirsql-embedding-ci.yml
==> python-sdk [python] uv-sync: uv sync --project packages/python
==> python-sdk [python] declared-deps: uv run --project internals/checks dirsql-checks declared-deps packages/python/dirsql

$ ... preflight --conventions .github/workflows/conventions.yml --dry-run   # the deleted file
preflight: --conventions .github/workflows/conventions.yml: no such workflow. Name one that calls .github/workflows/testing-conventions.yml, or drop the flag to derive the matrix from every caller in .github/workflows.   (exit 1)

$ ... preflight --conventions .github/workflows/docs-ci.yml --dry-run       # a real workflow, no callers
preflight: --conventions .github/workflows/docs-ci.yml: no job in it calls .github/workflows/testing-conventions.yml, so it declares no gates. ...   (exit 1)

$ ... preflight --conventions .github/workflows/internals-checks-ci.yml --gate unit-lint   # a real gate, really run
==> internals-checks [python] unit-lint: uvx testing-conventions unit lint --language python --config internals/checks/testing-conventions.toml internals/checks/src
preflight: 0 failing pair(s), 0 skipped   (exit 0)

Gates run locally for this lane, all green: colocated-test, unit-lint, integration-lint, unit-coverage (100% floor + changed lines), unit mutation (32 mutants, none surviving), plus the uv sync / declared-deps drift guards.

E2E Verification

  • Ran e2e suites locally for every affected SDK
  • Python SDK e2e: N/A — no packages/** change
  • TypeScript SDK e2e: N/A — no packages/** change
  • Rust core e2e (if applicable): N/A — no packages/** change
  • packages/python/e2e-attestation.json refreshed if packages/python changed — N/A
  • packages/ts/e2e-attestation.json refreshed if packages/ts changed — N/A
  • Command(s) run: cd internals/checks && uv run python -m pytest src/checks tests/integration tests/e2e -q (380 passed), then uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e -x -q'
  • Result summary: all three tiers green; internals/checks/e2e-attestations/claude-tackle-973-4e1epu.json refreshed as the last commit touching the package.

Changelog / Migrations

  • Changelog fragment — not required: the change is confined to internals/checks and .github/workflows, and the gate scopes to packages/ and plugins/.
  • Migration fragment — not required: repo tooling only, no public API.

PARITY.md untouched: no SDK surface changes.

`just preflight` dies with a bare FileNotFoundError: the command still defaults
`--conventions` to `.github/workflows/conventions.yml`, which #861 deleted when
the #834 epic split it into six per-domain workflows. Pointing the default at
one survivor would trade the crash for a green run covering one of eight lanes,
so the matrix has to come from every caller in `.github/workflows/`.

The colocated unit tests feed `parse_gate_matrix` inline YAML, so none of them
can see that the default target stopped existing. These read the repo's own
workflows instead:

  * integration: the command with its real defaults must cover every job that
    calls the reusable workflow, spanning every file that holds one, and must
    name a missing `--conventions` target rather than raise.
  * e2e: the packaged console script from the repo root, same two properties.

Nothing ran internals/checks' integration tier in CI, which is why the tier that
reads real workflow files could not have caught the rename; internals-checks-ci
now runs it.

Refs #973
claude added 3 commits August 20, 2026 19:18
…orkflow

`--conventions` becomes repeatable and defaults to nothing: with no workflow
named, every file in `.github/workflows/` that calls the reusable workflow is
discovered, and the matrix is the concatenation of their callers -- 40 pairs
across 8 scan roots in 6 workflows today, where the old default named a file
#861 deleted and resolved to 0.

Discovery rather than a list of six names for the same reason #781 derived the
pairs from the workflow in the first place: a name is a restatement that the
next rename invalidates, and the failure it produces is silent (a green run
covering one lane) or fatal (a bare FileNotFoundError from `just preflight`).

Both unresolvable cases now name the fix, per the actionable-error rule: a named
workflow that cannot be read, or that declares no caller, and a workflows
directory that is missing or holds no caller at all.

Fixes #973
`sources` takes its `directory` / `listdir` / `read` seams after a `*`, and
nothing failed when the mutation gate turned that into `/`: a caller could then
pass a directory as the second positional argument and have it land on
`listdir`. The sibling `run` already carries the same assertion.
@thekevinscott
thekevinscott merged commit 32677d9 into main Aug 20, 2026
9 checks passed
@thekevinscott
thekevinscott deleted the claude/tackle-973-4e1epu branch August 20, 2026 19:38
thekevinbot pushed a commit that referenced this pull request Aug 20, 2026
The recipe comments still routed readers to python-test.yml, ts-test.yml
and conventions.yml for the distcheck jobs, the python e2e-verify gate and
the packaging gate. Same four retired files as the rest of this branch,
and a contributor reading `just --list` hits these before any reference doc.

Also names `--conventions`, which #983 added while this branch was open, in
AGENTS.md's preflight line -- that line enumerates the other two options, so
omitting it reads as though it does not exist.

skip-changelog: docs and recipe comments only, no package source

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KcNqe44kvgP5mBRr9LjQcE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dirsql-checks preflight reads a workflow the #834 split deleted, so just preflight cannot run

3 participants