Repository navigation
ci: enable automatic AI review for PR authors with write access - #6419
Merged
Merged
Conversation
Uncomment the pull_request (opened/ready_for_review) trigger and gate the automatic path in resolve.ts to PR authors whose effective repository permission is write/admin, mirroring the manual /ai-review path's WRITE_PERMISSIONS gate. External contributors' PRs are skipped and keep going through the manual /ai-review maintainer path.
jgoux
approved these changes
Sep 1, 2026
Contributor
Supabase CLI previewnpx --yes https://pkg.pr.new/supabase/cli/supabase@d9916cad114733e21282c639ed363c82a09e4253Preview package for commit |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Superseded by a newer AI review
🤖 AI Review
Both independent reviews completed. Claude reported five findings and Codex reported none. Code inspection confirms four findings (two minor issues and two nits) and refutes the permission-error finding because the shared helper explicitly treats non-404 failures as fatal by design.
Findings
| Severity | Location | Category | Sources | Claim |
|---|---|---|---|---|
| 🟡 MINOR | .github/workflows/ai-review.yml:25 |
ci-concurrency |
claude | Automatic pull-request events share the manual review concurrency group, allowing a ready-for-review event to cancel an in-flight maintainer-requested review even when the automatic run subsequently skips. |
| 🟡 MINOR | .github/ai-review/README.md:171 |
documentation |
claude | The documentation overstates that the write-capable job runs exclusively trusted code: a pull_request run uses the PR's workflow definition, so pinning the job's checkout to develop does not itself prevent workflow-level changes from altering that job. |
| ⚪ NIT | .github/scripts/ai-review/resolve.ts:194 |
documentation |
claude | The comment incorrectly says a same-repository head branch implies the PR author could push; a PR author can open a pull request between branches they did not create or push, making the permission lookup authoritative rather than merely defense in depth. |
| ⚪ NIT | .github/ai-review/README.md:36 |
formatting |
claude | The edited resolver description introduces the only prose line in the file exceeding the surrounding 80-column wrapping convention. |
Refuted findings (kept for transparency, not posted as review comments)
.github/scripts/ai-review/resolve.ts:200(error-handling): Non-404 failures while resolving the PR author's permission fail the resolve job instead of being treated as unauthorized skips.
Refuted: Although non-404 errors do propagate, the shared helper explicitly defines and documents this as intentional behavior so transient failures remain visible. In this code, “unresolvable permission” means the helper'sundefinedresult, not an operational API failure.
Stats
Claude findings: 5 · Codex findings: 0 · Confirmed: 4 · Refuted: 1 · Uncertain: 0
Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run
This review runs once per PR. A maintainer can request another with a /ai-review comment.
… findings The supersede pass listed reviews AFTER posting, so the just-posted review (a marker-bearing bot review) was swept into its own supersede pass and every fresh review immediately collapsed as 'Superseded by a newer AI review'. Snapshot the prior runs before the POST instead, and make the test fake mirror real GitHub by including posted reviews in later listings. Also addresses the AI review findings on this PR: pull_request runs get their own per-PR concurrency group so a ready_for_review event can't cancel an in-flight maintainer-requested review; the write-access lookup is documented as the authoritative author check (a same-repo branch doesn't prove the author pushed it); the post-review trusted-code claim now acknowledges that pull_request events execute the PR ref's workflow file and why that's safe; README line rewrapped to the 80-column convention.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ends the AI review pipeline's shadow mode: the
pull_requesttrigger (opened/ready_for_review) inai-review.ymlis now live, so every qualifying PR gets its one-shot Claude + Codex review automatically.The automatic path is gated to internal contributors. On top of the existing draft/bot/fork skips,
resolve.tsnow resolves the PR author's effective repository permission and requireswrite/admin(the sameWRITE_PERMISSIONSgate the manual/ai-reviewpath already uses) before running. This lookup is the authoritative author check — a same-repo head branch only proves the branch exists in the repo, not that the author pushed it — and an unresolvable permission counts as unauthorized. External contributors' PRs are never reviewed automatically; a maintainer comments/ai-reviewto request one, exactly as before.The gate runs before the dedup listing (one permission lookup instead of two paginated list calls), and
PrDetailsnow carries the author's login so the resolver can do the lookup. The README's rollout section is rewritten to document the live trigger and the note that the Codex GitHub App's automatic reviews must stay disabled so PRs aren't double-reviewed.Two fixes surfaced by this PR's own live run:
postConsolidatedReviewlisted reviews after posting, so the just-posted review (a marker-bearing bot review) was swept into its own supersede pass and every new review immediately collapsed as "Superseded by a newer AI review". The prior runs are now snapshotted before the POST, and the test fake mirrors real GitHub by including posted reviews in later listings.pull_requestevents now use a per-PRautogroup instead of sharing the manualreviewgroup, so aready_for_reviewevent (whose run may resolve to a skip) can never cancel an in-flight maintainer-requested review.🤖 Generated with Claude Code