Skip to content

CI: 3-minute pull-request check, slow suites on schedules - #271

Merged
Programcode merged 3 commits into
mainfrom
ci/max-3-min
Oct 29, 2026
Merged

Programcode merged 3 commits into
mainfrom
ci/max-3-min

Conversation

@Programcode

@Programcode Programcode commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

The pull-request check should finish in about 3 minutes, and CI starts roughly a third as many runs a month. Everything slow, paid or full-stack still runs, but on a schedule or on demand.

CI could not run while Actions billing is blocked, so every number below is local. The CI estimates come from timing each step pinned to 2 CPUs (taskset -c 0,1), the size of a private-repo runner. That proxy is close: on 2 CPUs the old smoke set took 147s and the old unit run 95s, against 169s and 155s measured on CI.

Pull-request check

  • Quality: one serial job, ~7.3 min → 5 parallel jobs, slowest ~2.5 min
  • Browser smoke: ~4 min, 47 tests → ~2.3 min, 26 tests (76s of tests on 2 CPUs)
  • Database on schema PRs: ~17 min → ~2 min, pgTAP on Postgres alone
  • Agentic, Live provider, Model matrix: on most PRs → off PRs

What changed:

  • Quality: lint, typecheck+build, unit, edge and browser-smoke run in parallel. node_modules is restored from a lockfile-keyed cache (.github/actions/node-deps), so npm ci runs only on a miss. The edge job installs no npm packages; I ran the edge tests in a clean export with no node_modules to confirm (498 passed). docs.yml is gone and Quality runs on every PR: only 8 of 329 Docs runs in the last 30 days were docs-only pushes.
  • Unit tests use Vitest's vmThreads pool. Each file still gets a fresh jsdom context, but workers are reused: 95s to 43s on 2 CPUs. src/test/setup.ts adds crypto.randomUUID and the web streams that a bare VM context lacks. All 756 tests pass.
  • Smoke set keeps the startup shell, Login, every public and signed-in page in English, and the first-run tour. Each test pays a full dev-server page load (about 6s on 2 CPUs), which is why the set has to stay small. Crossfire, Sparring, free tools, the layout regressions and admin announcements lost the tag. They still run every night on desktop and mobile in browser-nightly.yml, which runs every test.
  • Database: PRs that change migrations, supabase/tests/, the seed, config.toml, the db scripts or the gate run db-gate.sh --db-only. That now starts Postgres alone (supabase db start), and the same 545 pgTAP tests and concurrency scripts pass: 67s to 35s locally, and one image pull on CI instead of a dozen. The full gate (full-stack journeys on desktop and mobile) runs daily and on dispatch. It no longer runs on pushes to main.

Runs per month (last 30 days, from the Actions API)

Before: 3,742. After: about 1,240, and fewer once agents stop pushing the same branch several times.

  • Quality 607 + Docs 329 → ~615 (one workflow)
  • Protected staging E2E 614 (594 skipped) → ~150: workflow_run now has branches: [main]
  • Production canary 789 (672 skipped) → ~185: it runs on push to main, waits for Vercel's debate-navigator deployment of that commit, then checks the site. I ran the poll against the real API for the last two main commits. deployment_status can't be filtered at the trigger, which is where the skipped runs came from.
  • Database 367 → ~150 (schema PRs plus nightly)
  • Agentic 402 → ~30, Live 288 → ~31, Model matrix 195 → ~4 (schedules only)
  • Staging preview note 125 → ~40 (opened/reopened only; it posts once anyway)

Billed minutes drop too, with two exceptions. The canary now bills its wait for Vercel, about 2 to 3 minutes per push to main, where the old skipped runs billed nothing. Docs-only PRs (8 a month) now pay for a full Quality run.

Moved to nightly, weekly or manual

  • Full-stack database gate: daily at 05:23 UTC (was weekly)
  • Full browser suite, desktop and mobile: daily (unchanged), now also the service-worker offline test
  • Live provider E2E: daily (unchanged)
  • Agentic suite: daily, plus the weekly bug bash (unchanged)
  • Model matrix: weekly (unchanged)

Every one also runs via gh workflow run <file> --ref <branch>. AGENTS.md tells agents to do that when a change is squarely in that suite's area. The live keys now reach branch code only through a manual dispatch; docs/standards/security.md records the narrower exposure.

Local npm run verify

231s to 98s on the VPS. scripts/verify.mjs runs two lanes, logs each step to .verify/npm-verify/, prints one line per step, and prints the tail of any failed step's log. A third lane was no faster and peaked about 6 GB above idle; two lanes peak at about 3.8 GB. test:sw moved to verify:full and the nightly workflow.

Running build and smoke side by side surfaced a real bug. scripts/prerender.mjs starts its own Vite server, which re-optimized dependencies in the shared node_modules/.vite. A dev server running at the same time then failed to load lazy routes ("Unable to load this page" on Login, 5 failures). Prerender now uses its own cacheDir. With the fix, smoke passed while the build ran.

Agent guidance

AGENTS.md: during the work, run only the focused check; run npm run verify once at the final head; push once, batching review fixes; and dispatch a scheduled suite only when the change is in its area. docs/operations.md, docs/testing.md and the README describe the new layout.

Verified

  • npm run verify at the final head: all 9 steps passed, 97.4s, memory peak +3.8 GB.
  • ./.superset/db-gate.sh --db-only: 545 pgTAP tests and all db scripts passed, 35s; stack stopped afterwards.
  • actionlint on every workflow: clean. npm run lint:repo (pinned actions, persist-credentials): clean.
  • db-gate.test.ts and anthropic-only-guard.test.ts still pass.

Not verified: any of this on GitHub Actions. The 3-minute figure is an estimate until billing is restored. The one unknown is how long CI takes to pull the Postgres image for the schema job. The first push to main after merge is the real test of the canary's wait step: it should find the debate-navigator deployment within about 2 minutes. If Vercel ever skips a build for a main commit, that run waits 15 minutes and then fails instead of skipping.

Not done, worth a look: the 2026-10-01 plan cut unit tests to 15 files, and there are 100 again. With vmThreads they cost about 70s on CI, so I left them alone.

🤖 Generated with Claude Code

… schedules

Quality now runs five parallel jobs with cached node_modules. Unit tests use
Vitest's vmThreads pool (95s to 43s on 2 CPUs), and the @smoke set keeps boot,
sign-in, every page render and the first-run tour; tool journeys stay nightly.

The full-stack database gate, live providers, the agentic suite and the model
matrix no longer run on pull requests. Schema PRs run pgTAP on Postgres alone.
Staging E2E, the production canary and the preview note stop starting runs
that only skip, and docs.yml folds into Quality.

npm run verify runs in two lanes (231s to 98s locally). Prerender gets its own
Vite cache so a build no longer breaks a running dev server's lazy routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@Programcode
Programcode marked this pull request as ready for review October 29, 2026 19:11
Copilot AI balanced review requested due to automatic review settings October 29, 2026 19:11
@Programcode
Programcode merged commit 73b3c11 into main Oct 29, 2026
2 of 8 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants