Skip to content

Pre-ship security pass: 25-item audit and fixes - #148

Merged
Programcode merged 28 commits into
mainfrom
security/preship-pass
Oct 17, 2026
Merged

Programcode merged 28 commits into
mainfrom
security/preship-pass

Conversation

@Programcode

@Programcode Programcode commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

A 25-item pre-ship security pass, plus fixes for every finding with a clear fix. Three Opus auditors each covered one slice in their own worktree: the database, the Edge Functions, and the frontend with hosting. Each auditor ran its own Docker-free local Supabase. Nothing touched staging or production. The full report is in plans/2026-10-16-security-preship-pass.md, including what still needs Adam.

Do not merge yet. Adam reviews first. Merging ships the frontend, and the migration and Edge Functions need their own authorized deploys. See the deploy order below.

Highs fixed

  • Promo codes could be brute-forced through PostgREST: 40 guesses took 1.3 s. A user is now locked out after 5 failures in 15 minutes, and unknown, expired and exhausted codes all get the same answer.
  • Redeeming a promo could lower existing access. A lifetime Pro user who redeemed a trial code dropped to trial. Grants now only raise access.
  • A re-delivered or out-of-order Stripe subscription.updated event restored Pro after cancellation. The webhook now claims each event before running handlers and reads subscription state from Stripe. A claim left by a worker that died is taken over after 10 minutes.
  • Model Markdown rendered remote images, and the CSP allowed images from any HTTPS host. A prompt-injected source could leak a student's case through an image URL without a click. Images now render as links, and the CSP no longer allows arbitrary image hosts.

Mediums and lows fixed

  • Database:
    • the anon key could read operator system prompts and global rate limits;
    • client roles held TRUNCATE, which RLS doesn't cover;
    • three admin reporting RPCs were executable by anon;
    • new objects granted clients everything by default;
    • account deletion left the demo code's email and IPs.
  • Edge Functions:
    • unknown AI features used up quota and made a paid Jev call;
    • raw provider, Stripe, Google and Postgres errors reached clients;
    • the demo-code limiter failed open when there was no client IP;
    • rejected import URLs used up quota;
    • export-gdoc accepted LAN origins;
    • prompt text and promo codes went into the logs;
    • the supabase-js import had no exact version.
  • Client:
    • the CSP listed hosts the client never calls, plus *.supabase.co;
    • tool drafts survived sign-out;
    • auth event logs ran in production;
    • share codes went to Sentry;
    • every high in npm audit --omit=dev was a build-only dependency (now cleared).
  • The build now fails if dist/ contains source maps, a server-secret pattern, or a JWT whose role isn't anon.

Tooling

scripts/agent-stack.sh gives each agent its own Docker-free local Supabase. It uses the CLI's experimental stack engine on the native runtime, which needs Supabase CLI 2.121 or newer. Each stack starts in about 20 seconds, needs no lock, and runs pgTAP, the concurrency scripts and functions serve. The auditors used it to run RLS probes as two real users and to call every function with eight kinds of bad token. docs/development.md explains how to use it. Declarative Schemas 2.0 worked as an export (122 files, clean strict coverage). This PR keeps supabase/migrations/ as the source of truth; whether to switch is in the report.

Verification

All results are local, because GitHub Actions can't run on this account right now: every job stops after 2 seconds with zero steps. A local pass is not a CI pass.

  • npm run verify passed at the merge with main (633317ce):
    • docs, lint and typecheck;
    • 285 unit tests;
    • 30 collaboration tests;
    • 356 Edge tests;
    • the build with the new guards;
    • the service-worker smoke;
    • 30 Chromium smoke tests.
  • Docker database gate (./.superset/db-gate.sh) passed at 0dae75ee: 441 pgTAP tests (23 files), the three concurrency scripts, and all 71 full-stack desktop and mobile journeys, including billing.
  • Database suite on a private stack: 441 pgTAP tests and the three concurrency scripts. Without the migration, the new security pgTAP files fail 13 of their 16 assertions.
  • Stripe webhook driven against the private stack with signed test events: a first delivery completes, a duplicate is skipped, a delivery that finds a claim in flight gets a 409, and an abandoned claim is taken over. The billing journey now also proves that a late active event can't revive a canceled plan.
  • CSP on the staging preview, in Chromium, signed in as the seeded Pro user:
    • all 15 routes load with no console errors;
    • Ask returned a real answer from the staging Edge Function through the narrowed connect-src;
    • the only violation is script-src blocking eval, and production already shows the same violation from its current bundle.
  • Codex review (gpt-6.1-sol, xhigh), three rounds:
    • Round 1 found an in-flight claim acknowledged as a duplicate, Edge-versus-Postgres clocks, and the default function grant. Fixed in af01c879; the global function revoke was dismissed because it strips extension functions.
    • Round 2 found a released claim reported as a duplicate. Fixed in 4323a681.
    • Round 3 had no findings at 4323a681. The only commit after it, 0dae75ee, changes the report and nothing else.

Previews (the staging one uses VERCEL_STAGING_BYPASS or your Vercel login):

The staging preview runs this branch's frontend against staging's current backend. The migration and Edge Functions only take effect there after a staging sync.

Also in this PR:

  • The e2e fake Stripe serves GET /v1/subscriptions/:id, and stripe-webhook honours STRIPE_API_URL the way account already did.
  • main gained Stream every AI answer live, token by token #149 (live streaming) after review. It is merged in; its new streaming error paths return the same fixed messages, and stream.test.ts now asserts the provider's text stays out. Codex round 4 at ebb3f404 reviewed the merge and the policy change: no findings.
  • Production and staging carry rate_limits policies that no migration created, and both allow global rows. The migration now drops every policy on rate_limits and global_prompts by catalog. It was rehearsed in one transaction on read-only dumps of both projects and applies cleanly.

Not verified: Stripe Checkout itself, hosted Auth settings, hosted preflight, staging or production backends with this migration, the race in claim_webhook_event under real concurrency (reviewed, not reproduced).

Deploy order (needs authorization)

  1. Apply 20261016120000_security_preship_hardening.sql to staging, then production. stripe-webhook needs its webhook_events.completed_at column.
  2. Deploy every Edge Function, one at a time, with stripe-webhook after the migration. _shared/cors.ts and _shared/auth.ts changed, and every function imports them; 12 functions also changed directly.
  3. Merge, which ships the CSP and client changes through Vercel.

Needs Adam

Hosted settings to check:

  • Auth redirect URLs, OTP expiry, CAPTCHA and password sign-in;
  • Google OAuth redirect URIs;
  • no Storage buckets;
  • hosted CORS preflight;
  • what X-Forwarded-For contains.

Decisions:

  • revoke the Jina key that survives in two local git tags;
  • demo accounts allowed 999 Pro requests a day;
  • quota refunds for failed provider calls;
  • webhooks wiping promo grace and downgrading on the first failed payment;
  • the signing-key fallback to the service-role key;
  • the anon keepalive writes;
  • the dead legacy tables;
  • loading Jam on demand;
  • an egress proxy for DNS rebinding;
  • HSTS preload.

The report lists each one with its evidence.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Added instructions for running isolated local database stacks and related development tasks.
    • Added protections against repeated promo-code guesses and improved reliability when processing payment events.
  • Bug Fixes
    • Signing out now clears saved drafts and form data.
    • Invalid source URLs are rejected before they count toward import limits.
    • Markdown images display as links instead of loading images.
  • Security
    • Tightened browser security restrictions and access to sensitive data.
    • Error responses reveal less internal detail, and shared-link codes are removed from error reports.

Programcode and others added 20 commits October 16, 2026 20:01
Uses the CLI's experimental stack engine with the native runtime so
several agents can each run migrations, pgTAP and RLS queries at once
without the shared db-gate lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Lock a user out of promo redemption after 5 failures in 15 minutes and
  answer unknown, expired and exhausted codes alike (next-15 #2).
- Promo grants never lower a tier, end date or daily limit.
- Revoke TRUNCATE/TRIGGER/REFERENCES/MAINTAIN from client roles, make
  global_prompts and global rate limits server-only, make the admin
  reporting RPCs service-role only, and close default privileges.
- Account deletion removes the user's demo code and scrubs the IPs and
  fingerprints recorded against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stack does not find functions behind a symlink, and its workers read
supabase/functions/.env rather than --env-file, so copy both into the
overlay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A prompt-injected source could make the model emit an image whose URL
carries the student's work; the browser fetched it without a click.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop provider, Jina, Google API and font hosts the client no longer
calls; replace *.supabase.co with the two project hosts; stop allowing
images from any HTTPS host; add form-action; make X-Frame-Options agree
with frame-ancestors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Fail the build if dist/ holds source maps, server-secret patterns or a
  non-anon JWT.
- Log auth events only in development builds.
- Redact share codes from Sentry events and transactions.
- Move tailwindcss-animate to devDependencies and update dompurify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tate

Co-Authored-By: Codex <noreply@openai.com>
…nput

Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
…uota

Co-Authored-By: Codex <noreply@openai.com>
Co-Authored-By: Codex <noreply@openai.com>
Claiming an event before its handlers means a worker that dies mid-run
would leave a claim that blocks every Stripe retry. Mark claims complete,
and let a retry take over a claim left incomplete for 10 minutes.
payment_succeeded now copies the subscription status from Stripe instead
of writing active, so a late delivery cannot revive a canceled plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
esm.sh/@supabase/supabase-js@2 resolved whatever 2.x esm.sh served on
deploy day, inside functions that hold the service-role key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 20:59
@vercel

vercel Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
debate-navigator Ready Ready Preview Oct 17, 2026 7:18pm UTC
debatelab-staging Ready Ready Preview Oct 17, 2026 7:18pm UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository YAML (base), Organization UI (inherited)
  • Review profile: CHILL
  • Plan: Free
  • Run ID: 53d2b111-07b5-42df-aead-5f4187db1a9b
📥 Commits

Reviewing files that changed from the base of the PR and between 7b234b8 and ebb3f40.

⛔ Files ignored due to path filters: 1
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing: 17
  • docs/operations.md
  • e2e/full-stack/billing.spec.ts
  • e2e/full-stack/fake-openrouter.mjs
  • e2e/full-stack/stack-core.ts
  • plans/2026-10-16-security-preship-pass.md
  • src/lib/supabase/AuthProvider.tsx
  • supabase/README.md
  • supabase/functions/ai/index.ts
  • supabase/functions/ai/provider.ts
  • supabase/functions/ai/stream.test.ts
  • supabase/functions/ai/stream.ts
  • supabase/functions/stripe-webhook/index.ts
  • supabase/functions/stripe-webhook/webhook-event-lifecycle.ts
  • supabase/functions/stripe-webhook/webhook-event-lifecycle_test.ts
  • supabase/migrations/20261016120000_security_preship_hardening.sql
  • supabase/tests/client_grants_test.sql
  • supabase/tests/webhook_claims_test.sql
🚧 Files skipped from review as they are similar to previous changes: 1
  • plans/2026-10-16-security-preship-pass.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The PR adds a private per-worktree Supabase stack and applies security changes across database permissions, Edge Functions, Stripe webhooks, client behavior, and deployment configuration. It also adds related tests and an audit report.

Changes: Per-worktree Supabase stack

Layer / File(s) Summary
Stack setup and commands
scripts/agent-stack.sh, docs/development.md
The script prepares a named local Supabase stack and provides commands to start, test, serve, stop, and destroy it. The guide documents the CLI requirement, dynamic ports, overlay behavior, and supported workflow.

Security hardening

Layer / File(s) Summary
Database grants, promo redemption, and account deletion
supabase/migrations/*, supabase/tests/*, supabase/README.md
The migration updates promo redemption, client grants, account-deletion cleanup, and webhook completion state. SQL tests cover redemption, grants, rate-limit access, and deletion cleanup.
Edge request validation and rate limits
supabase/functions/ai/*, supabase/functions/demo-code-login/*, supabase/functions/import-source/*, supabase/functions/redeem-promo/*
Handlers validate request features, URLs, and promo codes before later processing steps. Demo-code rate limits count qualifying failures for identified and unidentified requests. Tests check rejection and quota behavior.
Edge errors, logs, and dependencies
supabase/functions/_shared/*, supabase/functions/admin/*, supabase/functions/ai*/*, supabase/functions/create-*/*, supabase/functions/export-gdoc/*, supabase/functions/redeem-promo/*, supabase/functions/share/*, supabase/functions/voice/*
Selected Edge Function errors now use generic client messages and bounded logs. Research and request logs omit full private text, CORS responses include Vary: Origin, and Supabase client imports are pinned to version 2.114.0.
Stripe webhook event lifecycle
supabase/functions/stripe-webhook/*, supabase/migrations/*, supabase/README.md, e2e/full-stack/*
Webhook processing claims events before handling, can reclaim stale incomplete claims, and releases claims after handler failures. Subscription events use current Stripe subscription state.
Browser, deployment, and build protections
package.json, scripts/check-build-boundaries.mjs, src/components/ui/markdown.tsx, src/lib/*, src/test/*, vercel.json, docs/operations.md
The client clears persisted drafts on reset, redacts share codes from Sentry events, and renders Markdown images as links. CSP and frame headers are tightened, and the build check scans output for source maps and secret patterns.
Security audit record
plans/2026-10-16-security-preship-pass.md, plans/README.md, supabase/README.md
The report records the audit checklist, findings, unresolved checks, and declarative-schema decision. Supabase documentation describes webhook claims and client-grant guidance.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Stripe
  participant StripeWebhook
  participant SupabaseDB
  Stripe->>StripeWebhook: Deliver event
  StripeWebhook->>SupabaseDB: Claim event
  StripeWebhook->>Stripe: Retrieve subscription state when required
  StripeWebhook->>SupabaseDB: Complete claim after successful processing
Loading
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

Programcode and others added 3 commits October 16, 2026 21:18
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The webhook now reads subscription state from Stripe, so it honours
STRIPE_API_URL like the account function, and the fake serves
GET /v1/subscriptions/:id from state the test sets. The billing journey
also checks that a late active update cannot revive a canceled plan.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The form gained visible Title and Content labels, so the placeholder is
no longer the accessible name. This was already failing on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex review round 1:
- A retry that found an unfinished claim got 200 duplicate, which ends
  Stripe's retries; it now gets 409 so a dead delivery is retried.
- The abandon cutoff came from the Edge clock but was compared with a
  Postgres timestamp. claim_webhook_event and complete_webhook_event now
  decide on the database clock.
- The new-object grant test checked pg_default_acl rows; it now checks
  the effective privileges of a newly created table and sequence. A
  global function revoke would also strip extension functions (pgTAP
  broke), so functions keep their explicit per-migration revokes, which
  the anon EXECUTE test enforces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Programcode and others added 2 commits October 16, 2026 21:35
Codex review round 2: if a failed delivery released its claim between a
concurrent delivery's insert and its read, the second delivery found no
row and answered duplicate (HTTP 200) for an event nobody completed.
Claim again when the row is gone, and fall back to in_progress.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Programcode and others added 2 commits October 17, 2026 19:13
# Conflicts:
#	e2e/full-stack/case-generation.spec.ts
#	e2e/full-stack/current-debate.spec.ts
#	e2e/full-stack/debate-context.spec.ts
#	e2e/full-stack/stack-core.ts
#	supabase/functions/ai/index.ts
Production and staging carry rate_limits policies that no migration
created (rate_limits_select_own_or_global, "Users can read own limits"),
both allowing global rows. Dropping by the migration's own name would
have left them. Drop every policy on the two tables, and revoke client
writes on rate_limits. Rehearsed on read-only dumps of both projects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – debate-navigator — ebb3f404 Deployed Oct 17, 2026 by vercel[bot]
Preview – debatelab-staging — ebb3f404 Deployed Oct 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants