Skip to content

ci: start runners only for work a pull request, queue group or push needs - #352

Merged
botandbang merged 1 commit into
mainfrom
bnb/ci-runner-trim
Oct 31, 2026
Merged

botandbang merged 1 commit into
mainfrom
bnb/ci-runner-trim

Conversation

@botandbang

@botandbang botandbang commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

What and why

An audit of #349 and #350's CI found runners that start with nothing to do. In the 72 hours from Oct 28 there were 7,324 jobs and 19,905 job-minutes, and the 20-job cap was full for 254 minutes. This stacks on #350 and edits its ci.yml.

ci.yml

  • The scope step now works out every output in one script, from the filter results and the event. The outputs it adds are locallibrary_docs, locallibrary_cache, locallibrary_deny, locallibrary_dist, web_verify, web_e2e, web_docker, repo_checks_policy, repo_checks_ruleset, codeql and codeql_languages.
  • A push to main runs only what pull requests use from it. That's the website's and localtoolbox's coverage and sizes, which main-artifact reads, and only when code they measure changed. It's also the Rust caches (lint, build, msrv and the mix report's) and the Docker cache when what they're keyed on changed, CodeQL, and the ruleset check when the ruleset file changed. Its commit is the merge queue group's, which already passed. This is the biggest single saving: main's pushes were 21.8% of job-minutes.
  • cancel-in-progress is off for pushes to main: 50 main jobs had run for over 5 minutes before a newer merge cancelled them. GitHub keeps only one run waiting per group.
  • "CI passed" moves its conditions into the job's if (always() && (...)), so it takes a runner only to fail. A required job that its if skips counts as passing. Its draft message now says "Re-run all jobs", since re-running only failed jobs reuses the draft's result.
  • Narrower filters:
    • locallibrary takes only locallibrary-ci.yml from the workflows. Of the lexicons it takes only the ones the Rust code embeds or its test reads, and of the tokens only colors.json, colors.schema.json and mix.json. It leaves out bombadil/, icon.svg and deny.toml.
    • Crate READMEs alone run only the doctests and cargo doc.
    • cargo deny runs when Cargo.lock, a manifest or deny.toml changes.
    • web_e2e and web_docker are new. Docker keeps the tool docs, since docker-smoke.sh checks the install page. e2e keeps the tools' READMEs, since the home page's cards are built from them.
    • web_integration leaves out the lint configs and the e2e, Docker, Lighthouse and knip files.
    • workflow_lint covers only what actionlint and zizmor read.
    • repo-checks skips a change that only edits Markdown and adds or deletes nothing.
    • all (a change to ci.yml) no longer forces macOS and Windows in the queue.
  • Filters that were too narrow:
    • mix_report now covers every crate the CLI builds from, the CLI, the lockfile and the toolchain. scripts/mix-fixture.ts builds locallibrary-cli from the workspace.
    • .config/nextest.toml and tools/hegel-coverage.sh join locallibrary.
    • patches/** joins every pnpm tool's filter.
    • The local actions (pr-comment, size-report, main-artifact) join the filters of the tools that use them.
  • The changes job runs actions/labeler for same-repo pull requests, where pr-labels.yml started a runner of its own on every push.
  • In a merge queue group, the changes job cancels this workflow's runs for groups whose branch GitHub deleted. A rebuilt group gets a new branch name, so concurrency never cancelled the old run, and its macOS and Windows jobs held those runners.
  • CodeQL is now a call (codeql.yml runs on workflow_call). It runs on every pull request into main, merge queue group and push to main, but not on drafts. The workflows' analysis always runs, so every commit has a result for the code scanning rule. TypeScript and Rust run only when their code changed: 49 of 130 pull request runs ran Rust's 13 to 19 minute analysis without a .rs change.

Called workflows

  • locallibrary-ci.yml: features and docs are now steps in lint. The headless clippy runs before ALSA's headers are installed, so it still checks what the features job checked. lint also runs dist plan and the "only locallibrary-cli" check when dist is set. test and hegel run only with linux, build and msrv also for cache, and deny with deny. The coverage report skips when coverage didn't run.
  • web-ci.yml has verify, e2e and docker inputs. Its report skips when there's neither coverage nor sizes to post.
  • repo-checks.yml has policy and ruleset inputs. The weekly run checks only the ruleset and the lexicons' breaking, which is what changes outside the repo.

Other workflows

  • Removed locallibrary-bench.yml. main has no benchmarks, so its 468 runs in 14 days found none. bnb/perf brings it back, and the docs say where to find it in the history.
  • locallibrary-release.yml has no pull_request trigger; the plan runs in lint.
  • Scorecard runs weekly, on demand and on branch protection changes, not on every push.
  • Dependency review runs only when a lockfile, manifest or workflow changes, not on reopened, and not on drafts.
  • pr-labels.yml labels only forks' pull requests, and not on reopened.
  • Dependabot opens one pull request a month for all four ecosystems (multi-ecosystem-groups). Security updates keep their own groups.
  • The weekly suites (locallibrary, localtoolbox, web) get a changes job that skips them when nothing they test changed since their last scheduled or manual run that passed. last-passed takes an events input for this, and its default behaviour is unchanged. Advisories still run every week. The report jobs skip on a week whose suites skipped, which only follows a week that passed.
  • locallibrary-cache.yml skips its macOS and Windows run on a night when nothing rust-cache keys on changed and the caches are still there.

Policies

  • The passed job's conditions are checked on its if now, and so is the draft condition.
  • cancel-in-progress may be ${{ github.event_name != 'push' }}.
  • The ruleset's CodeQL rule accepts codeql.yml called from a workflow that runs on merge_group.
  • A new rule fails when a file the Rust code embeds from outside its crate (include_str!, include_bytes!) isn't in the locallibrary filter. tools/policy.sh lists them into embedded-files.json.
  • Removed the bench and release workflows from ungated.

Docs: .github/CONTRIBUTING.md, .github/NEW_PROJECT.md, apps/locallibrary/CONTRIBUTING.md and apps/locallibrary/TODO.md say what runs when. CONTRIBUTING says to merge main instead of closing and reopening to re-run CI, which was 11.4% of job-minutes.

No ruleset change: "CI passed" keeps its name.

How it was tested

  • tools/policy.sh: 82/82 Rego tests at full coverage, 50/50 conftest checks. Each new or changed rule failed when broken on purpose (7 mutations, each failing 1 or 2 tests).
  • actionlint (with shellcheck) and zizmor --offline are clean.
  • The scope script, run locally with bash 5, gives the expected outputs for a Rust-only pull request, a Markdown-only one, a draft, a stacked pull request changing ci.yml, merge groups with and without Rust, pushes with and without a lockfile change, the all-os label and a 3000-file pull request.
  • Only shows on GitHub:
    • whether a skipped "CI passed" satisfies the queue (GitHub documents that it does);
    • whether code scanning accepts a commit with only the actions analysis, which this pull request's own run checks once it targets main, since it changes no TypeScript or Rust;
    • the orphaned-group cancel;
    • the weekly gates' first skip;
    • the Dependabot group's first pull request.

Checklist

  • Tests cover the change (the policies; the workflows themselves only run on GitHub)
  • For the website: Storybook stories cover any new UI states (no UI)
  • For locallibrary: a changelog line (nothing users see)
  • The READMEs, docs/ and --help still say what the code does

…eeds

From the runner audit of #349 and #350's CI: narrower paths filters, job-level gates, a push to
main that runs only what pull requests use from it, CodeQL called from ci.yml per language, and
side workflows that no longer start runners with nothing to do.
@botandbang
botandbang requested a review from bnb as a code owner October 31, 2026 17:29
@github-actions github-actions Bot added documentation Improvements or additions to documentation github_actions Pull requests that update GitHub Actions code locallibrary locallibrary and the loclib-* crates tooling Repo config, lint and build tooling, and GitHub settings labels Oct 31, 2026
bnb
bnb previously approved these changes Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Coverage Report for apps/web

Status Category Percentage Covered / Total
🟢 Lines 95.42% (🎯 93%)
🟰 ±0 pp
7911 / 8290
🟢 Statements 94.09% (🎯 91%)
⬆️ +0.01 pp
12048 / 12804
🟢 Functions 94.24% (🎯 92%)
⬆️ +0.03 pp
3126 / 3317
🟢 Branches 84.71% (🎯 80%)
🟰 ±0 pp
5619 / 6633
File CoverageNo changed files found.
Generated in workflow #11 for commit 0b4a00f by the Vitest Coverage Report Action

Size

Compared with main. KiB are 1,024 bytes.

This pull request main Change
Client JavaScript 680.3 KiB 680.3 KiB 0 B (0%)
Client JavaScript, brotli 225.8 KiB 225.8 KiB 0 B (0%)
Client CSS 89.6 KiB 89.6 KiB 0 B (0%)
Client CSS, brotli 12.9 KiB 12.9 KiB 0 B (0%)

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Coverage Report for apps/localtoolbox

Status Category Percentage Covered / Total
🟢 Lines 100% (🎯 100%)
🟰 ±0 pp
4303 / 4303
🟢 Statements 99.64% (🎯 95%)
🟰 ±0 pp
5862 / 5883
🟢 Functions 99.65% (🎯 95%)
🟰 ±0 pp
1715 / 1721
🟢 Branches 97.72% (🎯 50%)
🟰 ±0 pp
2791 / 2856
File CoverageNo changed files found.
Generated in workflow #11 for commit 0b4a00f by the Vitest Coverage Report Action

Size

Compared with main. KiB are 1,024 bytes.

This pull request main Change
Packaged (localtoolbox.zip) 366.1 KiB 366.1 KiB +8 B (+0%)
Unpacked (dist/) 842.8 KiB 842.8 KiB 0 B (0%)

@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Rust coverage

From run 11. Both are held to the thresholds in .cargo/config.toml.

Tests Lines Functions Regions
All 97.4% (84411/86668) 96.54% (13505/13989) 95.67% (185844/194265)
Hegel property tests alone 95.4% (82680/86668) 94.13% (13168/13989) 93.47% (181582/194265)

Base automatically changed from bnb/ci-merge-queue to main October 31, 2026 18:12
@botandbang
botandbang dismissed bnb’s stale review October 31, 2026 18:12

The base branch was changed.

@botandbang

Copy link
Copy Markdown
Contributor Author

Closing and reopening so CI and CodeQL run against main now that #350 merged and this retargeted. Branch unchanged.

@botandbang botandbang closed this Oct 31, 2026
@botandbang botandbang reopened this Oct 31, 2026
@botandbang
botandbang added this pull request to the merge queue Oct 31, 2026
Merged via the queue into main with commit 3f398a7 Oct 31, 2026
66 of 68 checks passed
@botandbang
botandbang deleted the bnb/ci-runner-trim branch October 31, 2026 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation github_actions Pull requests that update GitHub Actions code locallibrary locallibrary and the loclib-* crates tooling Repo config, lint and build tooling, and GitHub settings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants