ci: run only the suites a PR's paths can reach, and 4 xdist workers - #1392
Merged
Merged
Conversation
`-n auto` counts physical cores when psutil is installed. The 4-vCPU ubuntu-24.04 runner reports 2, so the job started 2 xdist workers for ~11k tests (`created: 2/2 workers` in every run's log). `-n logical` starts 4. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The PR gate ran every suite on every pull request, so a docs-only change paid the full backend suite (7 minutes on the runner) for a result that could not have changed. A `changes` job in ci.yml now reads the PR's file list from the REST API and pipes it through scripts/ci/classify-changes.sh, which maps each path to the tests.yml suites that read it. Docs, root markdown and the TUI run nothing; frontend/ runs the two SPA jobs; infrastructure/ runs jest plus a new contracts-only backend job (supply_chain + architecture, the tests that read the CDK tree, about 30 seconds) instead of the full suite. The mapping is fail-open: an unrecognised path, an empty list or a workflow_dispatch runs everything, and .github/ and scripts/ always do. Filtering is on the jobs, not on the trigger's `paths:`, so a skipped suite reports as skipped rather than leaving a required check "Expected" if develop ever gains one. test_ci_path_filter.py pins the rules and their fail-open cases, and asserts that every run_* input tests.yml offers is wired on the PR gate so a new suite cannot silently run nowhere on a pull request. Also replaces the placeholder Testing doc page, which still said pytest was not run in CI. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The PR gate ran every suite on every pull request. A two-file docs PR paid the same 7–8 minutes as a backend change, and the backend pytest job alone was 7:02 while every other job finished in under 2 minutes. Two changes:
Fix 1 — run only the suites a PR's paths can reach. A
changesjob inci.ymlreads the PR's file list from the REST API (no third-party action, no fetch-depth games, renames fed under both names) and pipes it throughscripts/ci/classify-changes.sh, which maps paths to therun_*inputstests.ymlalready had:docs/,docs-site/,tui/, root markdown,LICENSEbackend/backend/scripts/frontend/--coverageinfrastructure/,RELEASE_NOTES.mdsupply_chain+architecture, the tests that read the CDK tree, ~30 s).claude/.github/,scripts/, anything unrecognised, empty list,workflow_dispatchFail-open by construction: only the explicit first row runs nothing. Filtering is on the jobs, not the trigger's
paths:, so a skipped suite reports as skipped rather than stranding a required check as "Expected" ifdevelopever gains one (it has none today, which is worth knowing on its own).Fix 2 —
-n logical. With psutil installed,-n autocounts physical cores; the 4-vCPU runner reports 2, so every run started 2 xdist workers (created: 2/2 workersin the logs).logicalstarts 4.Guard
backend/tests/supply_chain/test_ci_path_filter.pypins the classifier's rules and fail-open cases, asserts everyrun_*inputtests.ymloffers is wired inci.yml(a new suite cannot silently run nowhere on a PR), and asserts the backend step stays on-n logical.Also
docs/kaizen/review-queue.md.Verification
backend/tests/supply_chain+tests/architecture: 213 passed (32.9 s, which is the contracts job's expected runtime). Roottests/supply_chain: 48 passed..github/, so its own CI run should show thechangesjob classifying to everything; the first docs-only PR after merge is the real proof of the skip path.🤖 Generated with Claude Code