Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,9 @@ The staging browser suite disables trace, screenshot, and video capture. It crea

Do not enable the staging repository variable before every prerequisite is present. A skipped staging job is not a pass.

`STAGING_E2E_ENABLED` has been `true` since 2026-09-18. The job runs after every successful `Quality` run from a push to `main`. Quality runs from pull requests also trigger it, and those show as skipped by design. Check the latest real result with `gh run list --workflow staging-e2e.yml` and `npm run ci:failures -- <run-id>`, not this page. The run uploads no artifact, so the job log is the only evidence.
`STAGING_E2E_ENABLED` has been `true` since 2026-09-18. The job runs after every successful `Quality` run from a push to `main`. Quality runs from pull requests also trigger it, and those show as skipped by design. Check the latest real result with `gh run list --workflow staging-e2e.yml` and `npm run ci:failures -- <run-id>`, not this page. The run uploads no artifact, so the job log is the only evidence. On failure the log names the failed step and the URL path, for example `Staging step "add a source to the debate" failed at path /app/library`.

No staging run passed after 2026-09-23. Until 2026-10-01 the job did not run, because the `Quality` run it waits for never started. Once Actions came back, it failed on every `main` push through 2026-10-02. The cause was the spec, not staging: since the Current Debate change (#72), Library disables **Add Source** until a debate is current, and the spec never started one. The spec now starts a uniquely named debate and deletes it afterwards. The suite drives the real UI, so a change to a flow it covers can break it with no backend drift; run it locally before blaming staging.

## Staging sync

Expand Down Expand Up @@ -211,7 +213,7 @@ Do not expose protected data in artifacts. Record the commit, environment, opera
## Incident clues

- Every Actions job fails in 2 to 5 seconds with no logs: Actions minutes or billing, not the code. Verify locally.
- Green local E2E plus staging failure: schema, functions, provider access, demo code, or CORS may differ.
- Green local E2E plus staging failure: schema, functions, provider access, demo code, or CORS may differ. Run the staging suite locally first; if it fails there too on a UI step, the spec is stale, not staging.
- New frontend errors against Supabase right after a merge: the matching migration or function may not be applied to production yet. Run `npm run backend:status` from the linked checkout.
- Function returns unauthorized after deploy: check handler auth and environment propagation before changing gateway settings.
- Browser works on one route only: check the `vercel.json` SPA rewrite and cache headers.
Expand Down
12 changes: 8 additions & 4 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,11 +294,15 @@ The suite:
- verifies that the anon key belongs to the supplied staging URL;
- authenticates through `demo-code-login`;
- proves session reload;
- writes and reads one uniquely named source;
- makes one bounded AI request;
- deletes only the source it created.
- removes debates an interrupted earlier run left behind (only motions starting `E2E staging debate `);
- starts one uniquely named debate from Library, because sources belong to a Current Debate;
- writes one uniquely named source, then reads it back after clearing local state;
- makes one bounded AI request in Ask;
- deletes the source through the UI and the debate through the REST API with the test user's own session (the app cannot delete the Current Debate), which also removes the chat Ask saved to it.

Temporary auth state lives under `test-results/.auth/`. The staging projects disable traces, screenshots, and video because browser captures can retain authenticated data.
Use port `18080`: the staging Edge Functions answer browser requests only from allowlisted origins (see the [staging contract](operations.md#staging-contract)). The CI demo code exists only in the GitHub `staging` environment, so a local run needs its own bounded staging code.

Temporary auth state lives under `test-results/.auth/`. The staging projects disable traces, screenshots, and video because browser captures can retain authenticated data. Actions time out after 30 seconds, well inside the 180-second test timeout, so cleanup still runs after a failure. Each failure logs the step name and the URL path, never the query or hash. Playwright still writes an ARIA snapshot to `test-results/**/error-context.md` on failure; the workflow does not upload it, and it can contain staging account text.

The GitHub staging workflow runs only after an eligible main-branch quality result or a main-branch manual dispatch, and only when `STAGING_E2E_ENABLED=true`. The protected environment must already contain current secrets, functions, schema, provider access, a dedicated test code, and a main-only deployment rule.

Expand Down
221 changes: 168 additions & 53 deletions e2e/staging/authed-flows.spec.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,73 @@
import { expect, test } from '@playwright/test';
import { createClient } from '@supabase/supabase-js';
import { expect, test, type Page } from '@playwright/test';

test.describe.configure({ mode: 'serial' });
// A failed action throws well inside the test timeout, so the cleanup in `finally` still runs.
test.use({ actionTimeout: 30_000, navigationTimeout: 30_000 });

test('real staging session persists, writes and removes a source, and completes one AI request', async ({ page }) => {
const MOTION_PREFIX = 'E2E staging debate ';

/**
* Runs one named phase as a Playwright step. On failure it logs the phase and
* the URL path only: the query and hash can carry auth handoff tokens, and the
* staging projects keep no traces, screenshots or video.
*/
async function step<T>(page: Page, name: string, body: () => Promise<T>): Promise<T> {
return test.step(name, async () => {
try {
return await body();
} catch (error) {
const path = page.url().startsWith('http') ? new URL(page.url()).pathname : page.url();
console.error(`Staging step "${name}" failed at path ${path}`);
test.info().annotations.push({ type: 'failed-step', description: `${name} at ${path}` });
throw error;
}
});
}

/** A REST client acting as the signed-in staging test user, for cleanup only. */
async function userClient(page: Page) {
const session = await page.evaluate(() => localStorage.getItem('debate-auth-staging'));
const accessToken = session ? (JSON.parse(session) as { access_token?: string }).access_token : undefined;
expect(accessToken, 'staging session for cleanup').toBeTruthy();
return createClient(process.env.STAGING_SUPABASE_URL!, process.env.STAGING_SUPABASE_ANON_KEY!, {
auth: { persistSession: false, autoRefreshToken: false, detectSessionInUrl: false },
global: { headers: { Authorization: `Bearer ${accessToken}` } },
});
}

/**
* The debate the spec creates is the Current Debate, and the app only lets a
* student delete a debate that is not current, so cleanup uses the same
* signed-in session through the REST API and deletes that one row by id.
* Its sources and saved chat go with it (ON DELETE CASCADE).
*/
async function deleteOwnDebate(page: Page, debateId: string) {
const { data, error } = await (await userClient(page)).from('debates').delete().eq('id', debateId).select('id');
expect(error, error?.message).toBeNull();
expect(data).toHaveLength(1);
}

/**
* A run killed before its cleanup (a cancelled job, a crashed runner) leaves
* its debate behind as the account's Current Debate. Remove only debates this
* suite names, so every run starts from the same state.
*/
async function deleteLeftoverDebates(page: Page) {
const { data, error } = await (await userClient(page))
.from('debates').delete().like('motion', `${MOTION_PREFIX}%`).select('id');
expect(error, error?.message).toBeNull();
if (data?.length) console.log(`Removed ${data.length} debate(s) left by an earlier staging run`);
}

test('real staging session persists, starts a debate, writes and removes a source, and completes one AI request', async ({ page }) => {
test.setTimeout(180_000);
const consoleErrors: string[] = [];
const sourceTitle = `E2E staging source ${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const runId = `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`;
const motion = `${MOTION_PREFIX}${runId}`;
const sourceTitle = `E2E staging source ${runId}`;
const sourceContent = 'A dedicated Playwright staging record used to verify authenticated persistence.';
let debateId: string | null = null;
let sourceCreated = false;

page.on('pageerror', (error) => consoleErrors.push(`pageerror: ${error.message}`));
Expand All @@ -15,62 +76,116 @@ test('real staging session persists, writes and removes a source, and completes
});

try {
await page.goto('/app/settings');
const testEmail = await page.evaluate(() => localStorage.getItem('debate-staging-e2e-email'));
expect(testEmail).toBeTruthy();
await expect(page.getByText(testEmail!, { exact: true })).toBeVisible();
await step(page, 'session persists across a reload', async () => {
await page.goto('/app/settings');
const testEmail = await page.evaluate(() => localStorage.getItem('debate-staging-e2e-email'));
expect(testEmail).toBeTruthy();
await expect(page.getByText(testEmail!, { exact: true })).toBeVisible();

await page.reload();
await expect(page).toHaveURL(/\/app\/settings$/);
await expect(page.getByText(testEmail!, { exact: true })).toBeVisible();
await page.reload();
await expect(page).toHaveURL(/\/app\/settings$/);
await expect(page.getByText(testEmail!, { exact: true })).toBeVisible();
});

await page.goto('/app/library');
await page.getByRole('button', { name: 'Add Source' }).click();
await page.getByPlaceholder('Source title...').fill(sourceTitle);
await page.getByPlaceholder('Paste source content...').fill(sourceContent);
const sourceInsert = page.waitForResponse((response) =>
response.request().method() === 'POST' && response.url().includes('/rest/v1/sources')
);
await page.getByRole('button', { name: 'Save to Library' }).click();
expect((await sourceInsert).ok()).toBe(true);
sourceCreated = true;
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible();
await step(page, 'start a debate', async () => {
await page.goto('/app/library');
await deleteLeftoverDebates(page);
await page.reload();
// Sources belong to a debate, so Library asks for one before it allows adding a source.
await expect(page.getByTestId('library-debate')).toContainText('Select or create a debate', { timeout: 15_000 });
await expect(page.getByRole('button', { name: 'Add Source' })).toBeDisabled();
await page.getByTestId('library-debate').getByRole('button', { name: 'Start debate' }).click();
const dialog = page.getByRole('dialog', { name: 'Start new debate' });
await dialog.getByLabel('Motion').fill(motion);
const debateInsert = page.waitForResponse((response) =>
response.request().method() === 'POST' && response.url().includes('/rest/v1/debates')
);
await dialog.getByRole('button', { name: 'Start debate' }).click();
const insertResponse = await debateInsert;
expect(insertResponse.ok()).toBe(true);
const created = await insertResponse.json() as { id?: string; motion?: string };
expect(created.motion).toBe(motion);
debateId = created.id ?? null;
expect(debateId).toBeTruthy();
await expect(dialog).toBeHidden();
await expect(page.getByTestId('library-debate')).toContainText(motion);
await expect(page.getByTestId('current-debate-desktop')).toHaveAttribute('aria-label', `Current debate: ${motion}`);
});

await page.evaluate(() => localStorage.removeItem('debate-trainer-storage'));
const sourceHydration = page.waitForResponse((response) =>
response.request().method() === 'GET' && response.url().includes('/rest/v1/sources?')
);
await page.reload();
expect((await sourceHydration).ok()).toBe(true);
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible({ timeout: 15_000 });
await page.getByText(sourceTitle, { exact: true }).click();
await expect(page.locator('#source-detail-content')).toHaveValue(sourceContent);
await page.getByRole('button', { name: 'Close' }).click();
await step(page, 'add a source to the debate', async () => {
await page.getByRole('button', { name: 'Add Source' }).click();
await page.getByPlaceholder('Source title...').fill(sourceTitle);
await page.getByPlaceholder('Paste source content...').fill(sourceContent);
const sourceInsert = page.waitForResponse((response) =>
response.request().method() === 'POST' && response.url().includes('/rest/v1/sources')
);
await page.getByRole('button', { name: 'Save to Library' }).click();
expect((await sourceInsert).ok()).toBe(true);
sourceCreated = true;
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible();
});

await page.goto('/app/ask');
await page.getByLabel('Pros/Cons').click();
await page.getByPlaceholder('Ask your debate coach...').fill('Reply with one short sentence confirming this staging request.');
const aiResponse = page.waitForResponse((response) =>
response.request().method() === 'POST' && response.url().includes('/functions/v1/ai')
);
await page.getByRole('button', { name: 'Send message' }).click();
expect((await aiResponse).ok()).toBe(true);
const saveResponseButton = page.getByRole('button', { name: 'Save message' }).last();
await expect(saveResponseButton).toBeAttached({ timeout: 90_000 });
const assistantBubble = saveResponseButton.locator('..');
await expect(assistantBubble.locator(':scope > div.text-sm')).toHaveText(/\S/, { timeout: 90_000 });
} finally {
if (sourceCreated) {
await page.goto('/app/library');
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible({ timeout: 15_000 });
const sourceDelete = page.waitForResponse((response) =>
response.request().method() === 'DELETE' && response.url().includes('/rest/v1/sources?')
await step(page, 'source and debate hydrate from staging after local state is cleared', async () => {
// Sources and debates persist under separate keys; clear both so every value below comes from staging.
await page.evaluate(() => {
localStorage.removeItem('debate-trainer-storage');
localStorage.removeItem('debatelab-current-debate');
});
const hydrationRead = (table: string) => page.waitForResponse((response) =>
response.request().method() === 'GET' && response.url().includes(`/rest/v1/${table}?`)
);
await page.getByRole('button', { name: `Delete ${sourceTitle}`, exact: true }).click();
expect((await sourceDelete).ok()).toBe(true);
await page.evaluate(() => localStorage.removeItem('debate-trainer-storage'));
const tables = ['sources', 'debates', 'current_debates'];
const reads = Promise.all(tables.map(hydrationRead));
await page.reload();
await expect(page.getByText(sourceTitle, { exact: true })).toHaveCount(0, { timeout: 15_000 });
(await reads).forEach((read, index) => expect(read.ok(), `${tables[index]} read`).toBe(true));
await expect(page.getByTestId('library-debate')).toContainText(motion, { timeout: 15_000 });
await expect(page.getByTestId('current-debate-desktop')).toHaveAttribute('aria-label', `Current debate: ${motion}`);
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible({ timeout: 15_000 });
await page.getByText(sourceTitle, { exact: true }).click();
await expect(page.locator('#source-detail-content')).toHaveValue(sourceContent);
await page.getByRole('button', { name: 'Close' }).click();
});

await step(page, 'complete one AI request in Ask', async () => {
await page.goto('/app/ask');
await page.getByLabel('Pros/Cons').click();
await page.getByPlaceholder('Ask your debate coach...').fill('Reply with one short sentence confirming this staging request.');
const aiResponse = page.waitForResponse((response) =>
response.request().method() === 'POST' && response.url().includes('/functions/v1/ai'),
{ timeout: 90_000 });
await page.getByRole('button', { name: 'Send message' }).click();
expect((await aiResponse).ok()).toBe(true);
const saveResponseButton = page.getByRole('button', { name: 'Save message' }).last();
await expect(saveResponseButton).toBeAttached({ timeout: 90_000 });
const assistantBubble = saveResponseButton.locator('..');
await expect(assistantBubble.locator(':scope > div.text-sm')).toHaveText(/\S/, { timeout: 90_000 });
});
} finally {
try {
if (sourceCreated) {
await step(page, 'remove the source', async () => {
await page.goto('/app/library');
await expect(page.getByText(sourceTitle, { exact: true })).toBeVisible({ timeout: 15_000 });
const sourceDelete = page.waitForResponse((response) =>
response.request().method() === 'DELETE' && response.url().includes('/rest/v1/sources?')
);
await page.getByRole('button', { name: `Delete ${sourceTitle}`, exact: true }).click();
expect((await sourceDelete).ok()).toBe(true);
await page.evaluate(() => localStorage.removeItem('debate-trainer-storage'));
const sourceHydration = page.waitForResponse((response) =>
response.request().method() === 'GET' && response.url().includes('/rest/v1/sources?')
);
await page.reload();
expect((await sourceHydration).ok()).toBe(true);
await expect(page.getByTestId('library-debate')).toContainText(motion, { timeout: 15_000 });
await expect(page.getByText(sourceTitle, { exact: true })).toHaveCount(0, { timeout: 15_000 });
});
}
} finally {
if (debateId) {
const id = debateId;
await step(page, 'delete the debate', () => deleteOwnDebate(page, id));
}
}
}

Expand Down
Loading