Add an opt-in integration branch so small fixes run the full suite once per batch

[MaxGhenis]

Draft pilot. Inert until someone creates an integration branch; merging this alone changes no PR's CI. Adopting it is a decision for Max and the US maintainers (this repo's CLAUDE.md asks for authorization on runner-budget changes); it is queued for Max.

What

An opt-in flow for small, independent fixes:

• PRs into integration run the cheap tier only: PR validation (ReleaseLock), Package and compatibility, and Quick Feedback. The four heavy groups (Baseline, HouseholdAPIPartners, Contrib, PythonTests) get if: github.base_ref == 'main'.
• PRs into main are unchanged: the same 26 runner jobs, the same draft behaviour. A code-health test proves the base-main job set and budget are identical to today's.
• promote-integration.yaml (every 4 hours at :17 UTC, and on dispatch) runs .github/promote_integration.py:
  • does nothing if integration is absent or a promote/* PR into main is already open (an open promotion is never restarted by new fixes); opens no promotion when integration has no commits main lacks, though it may still sync main into integration;
  • first merges main back into integration (picking up the last promotion's merge and the release bot's version bump) and pushes; on a conflict it stops with a warning;
  • otherwise freezes integration's head on promote/<UTC timestamp> and opens a non-draft PR into main with the GitHub App token, so the PR gets CI. The body lists member PRs and commits and says to merge with a merge commit.
• CONTRIBUTING.md: which PRs target integration, what runs where, the merge-commit rule, and the red-promotion procedure (bisect the snapshot's first-parent history with the failing shard's command, revert the culprit on integration, close the failed promotion, dispatch a new one).

Ready PR, per push	Runner jobs before	After
base main	26	26
base integration	0 (no workflow)	3

Why

The PolicyEngine org is on GitHub Team: 60 concurrent standard-runner jobs across every repo (Actions limits). On Oct 6-7 the org averaged 58.5 of 60 busy all day, and this repo's Pull request workflow was 80% of the org's runner-minutes (386 full runs over 153 PRs in 2.7 days, a median of 541 runner-minutes each). On Oct 8 at 21:35Z ten full runs started within 31 seconds (260 jobs).

Rough arithmetic at Oct 6-8 volume: about 57 PRs a day × 2.5 pushes × about 14 runner-minutes of cheap tier, plus up to 6 promotions a day at about 1,090 runner-minutes each (PR run plus push-to-main run), is about 8,500 runner-minutes a day, against about 72,800 measured. That is an estimate from those volumes, not a measurement; a red promotion adds a bisection and a fresh promotion.

What the hub keeps (US + core hub, 2026-10-09)

• The full suite runs on every promotion, so interactions between fixes that touch the same program area are caught before main (the class of break Subtract Mississippi's HSA deduction once, from the account holder #9865 + Combine a negative Mississippi joint column with the other column #9885 caused on 10/7).
• The hub's per-PR microsimulation impact gate is independent of CI and still runs before a fix merges into integration.
• "Merged to integration" and "released" become separate states in the hub's registry.

Invariants and tests

Invariant	Test
Base main: runner jobs and budget identical to today	test_main_runner_checks_and_matrix_members_are_unchanged, test_pr_runner_budget_keeps_the_heavy_suites_separate
Base integration: exactly the three cheap jobs can run	test_pr_jobs_and_runner_budget_by_base_and_draft
Draft PRs run nothing for either base	same parametrized test; test_every_job_is_skipped_for_drafts
Never a second promotion while one is open; an open snapshot stays frozen	test_open_promotion_freezes_every_remote_ref and the pagination / concurrency-shape tests
No promotion when integration is absent or not ahead of main (a behind branch is still synced)	the absent / equal tests and test_promoted_integration_receives_version_bump_without_new_pr

test_promote_integration.py exercises the script against tiny local git repositories with a mocked gh: snapshot-name collisions, merge conflicts, and a concurrent push rejected during synchronization.

Ran: the five affected code-health files (test_pr_runner_budget.py, test_pr_workflow_skips_drafts.py, test_make_test_matches_ci.py, test_rest_python_groups.py, test_promote_integration.py) with --confcutdir=policyengine_us/tests/code_health: 41 passed. actionlint on the new workflow: clean. On pr.yaml: the same single pre-existing SC2126 style note on base and head. ruff format and ruff check clean.

Before activating (unverified here)

• The GitHub App installation behind APP_ID needs contents, pull requests and workflows write on this repo for the synchronization push and the promotion PR. Not checked.
• No live run yet. The first activation should be a workflow_dispatch with one small fix on integration, watching that the promotion PR's CI starts and runs all 26 jobs.
• Open questions for maintainers: who approves promotions (main requires one approval) and enforces merge-commit merges; whether 4 hours (or daily) is the right cadence.

Review round 1 (independent Codex lane): approve as an inert pilot, no blocking findings (~/reviews/pe-actions-concurrency-2026-10-08/review-10018-out.md). Its notes (a squash-merged promotion would replay member commits; pin job dependencies in tests; bisect caveat) are being addressed in a second commit before this leaves draft.

Built by a Codex lane from a written brief (~/reviews/pe-actions-concurrency-2026-10-08/build-integration-prompt.md), read in full by the opening session, and independently reviewed before leaving draft.

🤖 Generated with Claude Code

[MaxGhenis]

Closing on Max's decision (d1106, 2026-10-09): not adopting the integration-branch pilot for now. The branch ci-capacity/integration-branch (head d7d964648c) is kept.

Why:

• The runner queue has cleared: 47 jobs running and 0 queued org-wide at 16:06Z on 10/9, down from 57 running and 167 queued on 10/8. Cancel-in-progress and timeouts landed in a dozen repos, and the US hub now pushes fix rounds as drafts in small batches.
• Capacity is being raised directly: GitHub Support ticket #4840030 asks to lift the org's 60-job limit, and if it isn't granted by 10/14 the org moves to Enterprise Cloud (500 concurrent standard-runner jobs).
• With room to spare, per-PR full CI gives each fix its own signal. The pilot would add up to four hours before fixes reach main and PyPI, a bisection whenever a promotion fails, and a merge-commit rule enforced by a guard that is not yet finished: the round-3 review (~/reviews/pe-actions-concurrency-2026-10-08/review-10018-round3-out.md) found that updated-order pagination can skip the newest promotion if a PR's updated_at changes mid-scan.

Revive if: after the cap goes up, US PR jobs still wait more than 30 minutes at the 90th percentile over a week. Before reopening, replace the updated-order scan with a stable enumeration and rerun the review.