[avallete]

TL;DR

Two Podman stack integration tests failed when GitHub Releases was slow or returned 5xx.

• Database.integration.test.ts › restarts initialized data on a new release revision of its build and refuses another build: failed with SchemaError: Expected a valid JSON string. The test fetched the published manifest from GitHub and decoded the body without checking the status. It now rejects non-2xx responses, which names the URL and status. It also falls back to the S3 mirror the stack itself uses and checks the body against the published manifest pin.
• Functions.integration.test.ts › service catalog › destroy survives a symlink planted where its runtime directory used to be: timed out at 30s. A cold edge-runtime download kept getting HTTP 504 from GitHub before it fell back to S3. The download now runs in a beforeAll with its own 120s guard, so the test only measures prepare and removeData against a warm cache.

ref

spotted in:

• https://github.com/supabase/cli/actions/runs/42045749730/job/127308085819
• https://github.com/supabase/cli/actions/runs/42056253334/job/127345322962

---

Generated by Claude Code

[avallete]

/ai-review

---

Generated by Claude Code

[github-actions]

CLI benchmark (Lightweight Commands): no changes found

Commit b43d61b against develop at 3394a54 · 10 runs each · report only, never blocks merging

Command	Workload	Before	After	Change
supabase --version	version	296 ms	317 ms	no real change (+7%, within normal variation)
supabase --help	help	315 ms	325 ms	no real change
supabase init	init	294 ms	306 ms	no real change
supabase migration new bench	migration-new	307 ms	310 ms	no real change
supabase gen types --lang typescript --project-id replayfixtureprojref	gen-types	343 ms	318 ms	no real change (-7%, within normal variation)
supabase projects list	projects-list	323 ms	334 ms	no real change
supabase functions list --project-ref replayfixtureprojref	functions-list	333 ms	335 ms	no real change

API calls: unchanged for all 7 commands.

Binary size: 134.7 MB, unchanged.

Details: timing ranges, API calls per command, slowest phases

Timing ranges

Command	Workload	Before range	After range	Change	Verdict
supabase --version	version	253-347 ms	285-358 ms	+22 ms (+7%)	within noise
supabase --help	help	286-371 ms	265-364 ms	+10 ms (+3%)	within noise
supabase init	init	204-324 ms	195-352 ms	+12 ms (+4%)	within noise
supabase migration new bench	migration-new	220-393 ms	219-398 ms	+3 ms (+1%)	within noise
supabase gen types --lang typescript --project-id replayfixtureprojref	gen-types	316-428 ms	242-391 ms	-25 ms (-7%)	within noise
supabase projects list	projects-list	212-360 ms	230-364 ms	+11 ms (+3%)	within noise
supabase functions list --project-ref replayfixtureprojref	functions-list	287-401 ms	256-412 ms	+2 ms (+1%)	within noise

API calls per command

Command	Workload	Before	After	Endpoint changes
supabase --version	version	0	0
supabase --help	help	0	0
supabase init	init	0	0
supabase migration new bench	migration-new	0	0
supabase gen types --lang typescript --project-id replayfixtureprojref	gen-types	2	2
supabase projects list	projects-list	1	1
supabase functions list --project-ref replayfixtureprojref	functions-list	2	2

Slowest phases

No command moved beyond noise, so there are no phase deltas.

Workflow run

Comment /bench to run the lightweight benchmark (this comment) again on the latest commit, or /bench full to also run the integration-heavy benchmark, which posts in its own comment.

[github-actions]

Superseded by a newer AI review

🤖 AI Review

Confirmed two test-reliability issues, merging the duplicate Functions warm-up reports. The new hook never executes its Effect, and the Database fixture still depends on GitHub for uncached archives.

Findings

Severity	Location	Category	Sources	Claim
🟡 MINOR	packages/stack/src/services/Functions.integration.test.ts:160	test-reliability	claude+codex	The beforeAll hook returns a lazy Effect without executing it, so the artifact warm-up never runs. A cold download still occurs inside the symlink test's 30-second timeout.
🟡 MINOR	packages/stack/src/services/Database.integration.test.ts:402	test-reliability	claude	The manifest fallback leaves archive acquisition dependent on GitHub. An uncached synthetic revision can still fail when GitHub remains unavailable, even if the published archive is available on S3.

Findings outside the diff

• 🟡 MINOR packages/stack/src/services/Database.integration.test.ts:402 — The manifest fallback leaves archive acquisition dependent on GitHub. An uncached synthetic revision can still fail when GitHub remains unavailable, even if the published archive is available on S3.

Stats

Claude findings: 2 · Codex findings: 1 · Confirmed: 2 · Refuted: 0 · Uncertain: 0

---

Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

[github-actions]

🤖 AI Review

Both independent reviews were available. After merging the duplicate Functions finding, two minor test-reliability issues are confirmed: the warm-up Effect is never executed, and the Database fallback leaves archive downloads dependent on GitHub. Integration tests were not run; dependencies are absent.

Findings

Severity	Location	Category	Sources	Claim
🟡 MINOR	packages/stack/src/services/Functions.integration.test.ts:160	test-reliability	claude+codex	The new beforeAll hook returns a lazy Effect without executing it, so its 120-second timeout does not cover the artifact download. A cold download still occurs inside the test's original 30-second timeout.
🟡 MINOR	packages/stack/src/services/Database.integration.test.ts:385	test-reliability	claude	The new S3 fallback covers only manifest fetching. On a cold cache, the synthetic next-revision archive still redirects exclusively to GitHub, so GitHub download failures can still fail the test.

Stats

Claude findings: 2 · Codex findings: 1 · Confirmed: 2 · Refuted: 0 · Uncertain: 0

---

Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

[avallete]

On the outside-diff finding (Database.integration.test.ts:402, the archive still redirects to GitHub): that's right, but it's a separate exposure from the reported flake. The failure was the manifest body decoded without a status check, and that's what this PR fixes. Pointing the fixture's redirect at S3 instead would only swap one upstream host for another. It would also need the fixture to know the S3 layout for every path the stack requests from the releases URL. If the archive leg shows up in the flaky watch, it gets its own fix. No change here.

---

Generated by Claude Code

[github-actions]

Flaky test check: ✅ no flaky tests

Commit d3f52e27832a · focused ×10 (5 executions each), 9743 tests · integration ×10, 6843 tests · unit ×10, 9492 tests · workflow run

_{Each execution is a separate Vitest process; Where lists failing runs, as run.iteration for suites executed more than once per run. The full data is in the flaky-check-report artifact.}