Repository navigation
feat: enable Spark evals in CI - #489
mattrossman wants to merge 6 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
|
Note the sample scenario passed when we added it in #482. Running it in CI now hits a known Spark bug (Spark mixes up which columns belong to each table) which means reading all columns fails with |
c19502d to
a6843ba
Compare
|
@codex review |
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6843ba10d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review again 🙏 |
|
Codex Review: Didn't find any major issues. 🚀 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@mattrossman https://supabase.slack.com/archives/C0C7XC4AV34/p1794516911579389?thread_ts=1794359901.275219&cid=C0C7XC4AV34 Me and Nikita were discussing this regression here. I already have the fix for it |
|
waiting for his approval to deploy |
|
Thanks @pedrocarlo. FWIW that issue isn't a blocker for this PR, just wanted to highlight why the results looked that way. |
Enables Spark evals in CI by running WARP inside Vercel Sandbox. Adds redaction of Spark URLs and credentials from published results and excludes Spark workspace and session archives from public artifacts.
The repository secrets
CLOUDFLARE_WARP_CLIENT_IDandCLOUDFLARE_WARP_CLIENT_SECRETare configured using a service token provided from here.Both Spark and non-Spark experiments completed in CI. The Spark trace shows a migration, row update, and query working. Grading hit a known issue.
Once merged, this will allow us to run Spark evals by dispatching the workflow e.g.
Closes AI-1413