Delay Jam and align cache, CSP, and robots

docs/operations.md

@@ -42,23 +42,23 @@ Vercel builds and serves the frontend from the Git repository. A push to `main`
 
 - automatic Git deployments: `main` only. `git.deploymentEnabled` sets `main` to true and `**` to false. A branch deploys when any matching rule is true, and a branch that matches nothing defaults to true, so the `**` rule has to stay. `*` does not match a slash, which would leave branches such as `cursor/…` deploying. A push that matches no true rule creates no deployment and does not count toward the 100-deployments-per-day cap;
 - the Vite build command and `dist` output;
-- rewrites of the app routes (`/app`, `/login`, `/share`, `/expocit/sources`) to `app-shell.html`; any other path that is not a file gets `404.html` with a real 404 status;
+- rewrites of the app routes (`/app`, `/login`, `/share`, `/expocit/sources`) to `app-shell.html`; any other path that is not a file gets `404.html` with a real 404 status. `/expocit/sources` is also prerendered, and the filesystem is served before rewrites, so that file is what visitors and crawlers get. The rewrite remains for a build that did not emit it;
 - `trailingSlash: false`, which 308-redirects `/app/settings/` to `/app/settings` before routing, since rewrite and redirect sources match exactly;
 - 301 redirects from the retired brand pages (`/debatelab`, `/debatelab-by-programcode`, and the rest) to `/`;
 - 301 redirects from retired guide slugs to the guide that replaced them, such as `/debate-weighing-guide` to `/how-to-weigh-in-debate`. The `trailingSlash` 308 runs first, so `/debate-weighing-guide/` reaches the same page in two permanent hops and needs no rule of its own;
-- no-cache headers for `/`, `index.html`, `app-shell.html`, `theme-init.js`, and `sw.js`, and immutable caching for hashed `/assets/`;
-- `noindex` headers for `/app`, `/login`, `/share` and `/embed/debate-timer`;
-- Content Security Policy and the other response security headers, including the `Permissions-Policy` that allows the microphone for the app's own origin only. Every path except `/embed/debate-timer` refuses to be framed (`X-Frame-Options: DENY` and `frame-ancestors 'none'`). `/embed/debate-timer` allows any site to frame it and gets its own CSP, which permits only same-origin scripts and requests. The two rules use complementary sources (`/((?!embed/debate-timer$).*)` and `/embed/debate-timer`) so no path gets both, a new embed needs its path added to both, and `src/test/vercel-deployment-config.test.ts` fails if two matching rules ever set the same header.
+- revalidating cache headers (`public, max-age=0, must-revalidate`, without `no-store`, so a document can enter the back/forward cache) for `/`, `index.html`, `app-shell.html`, and `theme-init.js`. `sw.js` stays `no-store` so a new service worker is not stuck in the HTTP cache. Hashed `/assets/` are immutable. The unhashed images in `public/` (`favicon.ico`, `favicon.png`, `favicon-72.png`, `hero-visual.png`, `placeholder.svg`) are cached for 30 days. Jam's capture iframe is removed on `pagehide` (never via `unload`) so that cross-origin frame cannot veto the cache;
+- `noindex` headers for `/app`, `/login`, `/share` and `/embed/debate-timer`. The app shell's robots meta is `noindex, nofollow`, the same policy. Public pages, including the prerendered `/expocit/sources`, are `index, follow`. A route deletes the platform `Access-Control-Allow-Origin: *` header on HTML responses and leaves it on static assets;
+- Content Security Policy and the other response security headers, including the `Permissions-Policy` that allows the microphone for the app's own origin only. Jam is limited to `https://js.jam.dev` and `https://recorder.jam.dev`; `script-src` does not include `unsafe-eval`. Web Analytics and Speed Insights stay on this origin (`/_vercel/insights` and `/_vercel/speed-insights`), which `'self'` already allows. Every path except `/embed/debate-timer` refuses to be framed (`X-Frame-Options: DENY` and `frame-ancestors 'none'`). `/embed/debate-timer` allows any site to frame it and gets its own CSP, which permits only same-origin scripts and requests. The two rules use complementary sources (`/((?!embed/debate-timer$).*)` and `/embed/debate-timer`) so no path gets both, a new embed needs its path added to both, and `src/test/vercel-deployment-config.test.ts` fails if two matching rules ever set the same header.
 
-After `vite build`, [`scripts/prerender.mjs`](../scripts/prerender.mjs) renders the landing page, `/terms`, `/privacy`, and the 404 page in Node through [`src/prerender/entry.tsx`](../src/prerender/entry.tsx). It writes them to `dist/index.html`, `dist/terms/index.html`, `dist/privacy/index.html`, and `dist/404.html`, each with its own title, description, and canonical URL from [`src/lib/seo/pages.ts`](../src/lib/seo/pages.ts); the landing page also carries the Organization, SoftwareApplication (with the trial and Pro offers) and FAQPage JSON-LD. The built shell is kept as `dist/app-shell.html`. On load, the client replaces the prerendered HTML with the same page. A new public SPA page needs an entry in `PAGE_HEADS`, in the prerender entry, and a route kept out of the 404.
+After `vite build`, [`scripts/prerender.mjs`](../scripts/prerender.mjs) renders the landing page, `/terms`, `/privacy`, `/expocit/sources`, and the 404 page in Node through [`src/prerender/entry.tsx`](../src/prerender/entry.tsx). It writes them to `dist/index.html`, `dist/terms/index.html`, `dist/privacy/index.html`, `dist/expocit/sources/index.html`, and `dist/404.html`, each with its own title, description, and canonical URL from [`src/lib/seo/pages.ts`](../src/lib/seo/pages.ts); the landing page also carries the Organization, SoftwareApplication (with the trial and Pro offers) and FAQPage JSON-LD. The built shell is kept as `dist/app-shell.html`. On load, the client replaces the prerendered HTML with the same page. A new public SPA page needs an entry in `PAGE_HEADS`, in the prerender entry, and a route kept out of the 404.
 
 The free tools under `/tools` ([`src/pages/tools/`](../src/pages/tools/)) are prerendered the same way, listed in [`src/prerender/tool-pages.tsx`](../src/prerender/tool-pages.tsx) with heads in [`src/lib/seo/tool-pages.ts`](../src/lib/seo/tool-pages.ts). They render their own JSON-LD in the page body, so the dev server and the prerendered HTML carry the same structured data, and their `dateModified` is `TOOL_PAGES_MODIFIED` in that file: bump it with a substantive content change. Their speech times come from [`src/lib/free-tools/formats.ts`](../src/lib/free-tools/formats.ts), which cites a league source per format.
 
 `/embed/debate-timer` is a second Vite HTML entry ([`embed/debate-timer/index.html`](../embed/debate-timer/index.html)), not an app route: it loads only the timer, never the app shell, Jam, analytics or the service worker, so a site that frames it sees no third-party request. It is `noindex` with its canonical URL on `/tools/debate-timer`, and the service worker leaves `/embed/` alone.
 
 [`scripts/sitemap.mjs`](../scripts/sitemap.mjs) then writes `dist/sitemap.xml` from every page in `dist/` that names itself as canonical and is not `noindex`. Each `<lastmod>` is the `dateModified` the page declares in its JSON-LD: the guide pages' `CONTENT_DATE` in [`scripts/generate_seo_content.py`](../scripts/generate_seo_content.py), the legal pages' "last updated" dates, the tool pages' `TOOL_PAGES_MODIFIED`, and the landing page's last source commit. A page without a date is listed without `<lastmod>`.
 
-[`scripts/check-site.mjs`](../scripts/check-site.mjs) runs next and fails the build when an internal link, a `#fragment`, a sitemap URL or an `llms.txt` URL does not resolve (it applies the `vercel.json` redirects and rewrites), when JSON-LD does not parse or carries HTML entities, when an indexed page other than `/terms` and `/privacy` lacks its question H2 (`#short-answer`) as the first H2, or when an image has no alt text, a button has no name or a form field has no label.
+[`scripts/check-site.mjs`](../scripts/check-site.mjs) runs next and fails the build when an internal link, a `#fragment`, a sitemap URL or an `llms.txt` URL does not resolve (it applies the `vercel.json` redirects and rewrites), when JSON-LD does not parse or carries HTML entities, when an indexed page other than `/terms`, `/privacy`, and `/expocit/sources` lacks its question H2 (`#short-answer`) as the first H2, or when an image has no alt text, a button has no name or a form field has no label.
 
 The guide pages in `public/<slug>/index.html` are generated, not hand-edited: run `python3 scripts/generate_seo_content.py` and commit its output. Short pages live in its `PAGES` list; pages with long-form copy, their own highlight-card descriptions, and their own FAQs (which also feed the page's FAQPage JSON-LD) live in [`scripts/seo_guides.py`](../scripts/seo_guides.py). Keep `PAGES` in order, because each page's related-guide links are picked by position. A guide entry can set its own `modified` date, which overrides `CONTENT_DATE` for that page alone.
 
@@ -106,7 +106,7 @@ Admins see a **Developer** card in Settings that signs out, sets the key, and re
 
 A build can pin one project with the `VITE_SUPABASE_ENV` build variable (`production` or `staging`; any other non-empty value fails the build). A pinned build ignores the localStorage key, hides the Settings card and the strip's **Switch to Production** button, and does not reload when another tab switches. The strip still shows on a pinned staging build. The staging Vercel project uses this so a requested preview is pinned to staging; see [Opt-in previews](#opt-in-previews).
 
-The switch only works while the `connect-src` directive in [`vercel.json`](../vercel.json) allows the staging project origin. It lists exactly the two project origins from [`src/lib/supabase/projects.ts`](../src/lib/supabase/projects.ts), and `src/test/vercel-deployment-config.test.ts` fails if they drift apart.
+The switch only works while the `connect-src` directive in [`vercel.json`](../vercel.json) allows the staging project origin. It lists exactly the two project origins from [`src/lib/supabase/projects.ts`](../src/lib/supabase/projects.ts), and `src/test/vercel-deployment-config.test.ts` fails if they drift apart. The same file is deployed to production and to `debatelab-staging`, and an unpinned production preview can still select staging, so the staging origin stays in the production site's CSP.
 
 ## Opt-in previews
 

e2e/loading-performance.spec.ts

@@ -4,15 +4,49 @@ import { setLocale, stubSupabaseSession } from './helpers/auth';
 test('a stalled Jam recorder cannot delay the real application', async ({ page, browserName }) => {
   test.skip(browserName === 'webkit', 'Jam recording requires MediaRecorder, which this WebKit runtime does not expose.');
   await page.route('https://js.jam.dev/**', async () => new Promise(() => undefined));
-  const jamRequest = page.waitForRequest('https://js.jam.dev/**');
+  const jamRequests: string[] = [];
+  page.on('request', (request) => {
+    if (request.url().startsWith('https://js.jam.dev/')) jamRequests.push(request.url());
+  });
 
   await page.goto('/', { waitUntil: 'domcontentloaded' });
 
-  await jamRequest;
+  // Jam waits for an interaction, so the landing page is up before any recorder request.
   await expect(page.getByRole('heading', { name: /Prepare Cases and Practice Rounds/ })).toBeVisible({ timeout: 2_000 });
+  expect(jamRequests).toEqual([]);
+
+  const jamRequest = page.waitForRequest('https://js.jam.dev/**');
+  await page.evaluate(() => window.dispatchEvent(new PointerEvent('pointerdown', { bubbles: true })));
+  await jamRequest;
+  await expect(page.getByRole('heading', { name: /Prepare Cases and Practice Rounds/ })).toBeVisible();
   await expect(page.getByText('DebateLab by Pi-Stack is the official')).toHaveCount(0);
 });
 
+test('pagehide removes the Jam frame and a persisted pageshow does not re-add it (synthetic events, stubbed Jam)', async ({ page, browserName }) => {
+  test.skip(browserName === 'webkit', 'Jam recording requires MediaRecorder, which this WebKit runtime does not expose.');
+  await page.route('https://js.jam.dev/**', (route) => route.fulfill({ status: 200, contentType: 'text/javascript', body: '' }));
+
+  await page.goto('/', { waitUntil: 'load' });
+  const heading = page.getByRole('heading', { name: /Prepare Cases and Practice Rounds/ });
+  await expect(heading).toBeVisible();
+  const jamRequest = page.waitForRequest('https://js.jam.dev/**');
+  await page.evaluate(() => window.dispatchEvent(new PointerEvent('pointerdown', { bubbles: true })));
+  await jamRequest;
+
+  // Stand-in for the capture frame Jam injects; the real one is cross-origin.
+  const frames = await page.evaluate(() => {
+    const frame = document.createElement('iframe');
+    frame.dataset.jamCapture = 'true';
+    document.body.append(frame);
+    window.dispatchEvent(new PageTransitionEvent('pagehide', { persisted: true }));
+    const afterHide = document.querySelectorAll('iframe[data-jam-capture]').length;
+    window.dispatchEvent(new PageTransitionEvent('pageshow', { persisted: true }));
+    return { afterHide, afterShow: document.querySelectorAll('iframe[data-jam-capture]').length };
+  });
+  expect(frames).toEqual({ afterHide: 0, afterShow: 0 });
+  await expect(heading).toBeVisible();
+});
+
 test('public routes keep the real UI after refresh', async ({ page }) => {
   await setLocale(page, 'en');
   for (const [path, label] of [

index.html

@@ -12,7 +12,9 @@
     <meta name="description" content="DebateLab is an AI debate prep and practice app for students and competitive debaters: cases, rebuttals, evidence, coaching and timed practice rounds against an AI opponent." />
     <meta name="keywords" content="DebateLab, debate lab, AI debate practice, debate practice platform, debate drills, rebuttal practice, argumentation training, debate coach, debate preparation" />
     <meta name="author" content="DebateLab by Pi-Stack" />
-    <meta name="robots" content="index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1" />
+    <!-- The shell for /app, /login and /share. Prerender replaces this with the
+         public index directive on pages that have a canonical URL. -->
+    <meta name="robots" content="noindex, nofollow" />
     <meta name="language" content="English" />
     <meta name="theme-color" content="#f1f5f9" media="(prefers-color-scheme: light)" />
     <meta name="theme-color" content="#0f172a" media="(prefers-color-scheme: dark)" />

public/theme-init.js

@@ -20,9 +20,19 @@
 
   try {
     var pathname = window.location.pathname;
+    var privateRoute = pathname === '/app' || pathname.indexOf('/app/') === 0
+      || pathname === '/login'
+      || pathname === '/share' || pathname.indexOf('/share/') === 0;
     if (pathname === '/app' || pathname.indexOf('/app/') === 0 || pathname === '/login') {
       document.documentElement.classList.add('app-shell');
     }
+    // The shell HTML is noindex. Private routes keep that. The sources page is
+    // public, including when the app-shell rewrite serves this document.
+    var robots = document.querySelector('meta[name="robots"]');
+    if (robots && privateRoute) robots.setAttribute('content', 'noindex, nofollow');
+    else if (robots && pathname === '/expocit/sources') {
+      robots.setAttribute('content', 'index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1');
+    }
   } catch {
     // Ignore unavailable location or document globals.
   }

scripts/check-site.mjs

@@ -13,7 +13,7 @@ const DIST = 'dist';
 const SITE_ORIGIN = JSON.parse(readFileSync('src/config/site.json', 'utf8')).origin;
 const vercel = JSON.parse(readFileSync('vercel.json', 'utf8'));
 // Indexed pages that answer no question of their own.
-const NO_SHORT_ANSWER = new Set(['/terms', '/privacy']);
+const NO_SHORT_ANSWER = new Set(['/terms', '/privacy', '/expocit/sources']);
 
 const problems = [];
 const report = (where, message) => problems.push(`${where}: ${message}`);

scripts/prerender.mjs

@@ -74,9 +74,12 @@ function pageHtml(shell, page, assets) {
     [/<meta name="twitter:title" content="[^"]*" \/>/, `<meta name="twitter:title" content="${escapeAttribute(head.title)}" />`, 'twitter:title'],
     [/<meta name="twitter:description" content="[^"]*" \/>/, `<meta name="twitter:description" content="${escapeAttribute(head.description)}" />`, 'twitter:description'],
   ]) html = replaceOnce(html, pattern, value, label);
-  if (!head.canonicalPath) {
-    html = replaceOnce(html, /<meta name="robots" content="[^"]*" \/>/, '<meta name="robots" content="noindex" />', 'robots');
-  }
+  // The shell document is noindex (app, login, share). A public page puts the
+  // index directive back; a page without a canonical URL stays noindex.
+  const robots = head.canonicalPath
+    ? 'index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1'
+    : 'noindex, nofollow';
+  html = replaceOnce(html, /<meta name="robots" content="[^"]*" \/>/, `<meta name="robots" content="${robots}" />`, 'robots');
 
   const added = [];
   if (head.canonicalPath) {

scripts/test-service-worker-offline.mjs

@@ -132,7 +132,7 @@ async function main() {
       headers: { 'Cache-Control': 'no-store' },
       body: V2_WORKER,
     }));
-    await upgradePage.goto(origin, { waitUntil: 'networkidle' });
+    await upgradePage.goto(origin, { waitUntil: 'load' });
     await upgradePage.evaluate(async () => navigator.serviceWorker.ready);
     await upgradePage.waitForFunction(async () => {
       const registration = await navigator.serviceWorker.getRegistration();
@@ -153,7 +153,7 @@ async function main() {
 
     const activatedPage = await context.newPage();
     await prepare(activatedPage);
-    await activatedPage.goto(origin, { waitUntil: 'networkidle' });
+    await activatedPage.goto(origin, { waitUntil: 'load' });
     await activatedPage.evaluate(async () => navigator.serviceWorker.ready);
     await activatedPage.waitForFunction(async () => {
       const keys = await caches.keys();
@@ -167,7 +167,7 @@ async function main() {
 
     const page = await context.newPage();
     await prepare(page);
-    await page.goto(`${origin}/app`, { waitUntil: 'networkidle' });
+    await page.goto(`${origin}/app`, { waitUntil: 'load' });
     await page.locator('main h1', { hasText: 'What are you working on?' }).waitFor({ timeout: CONTENT_TIMEOUT_MS });
     await page.evaluate(async () => navigator.serviceWorker.ready);
     await page.waitForFunction(async (paths) => {
@@ -192,7 +192,7 @@ async function main() {
     await context.setOffline(false);
     const prefetchedPage = await context.newPage();
     await prepare(prefetchedPage);
-    await prefetchedPage.goto(`${origin}/app`, { waitUntil: 'networkidle' });
+    await prefetchedPage.goto(`${origin}/app`, { waitUntil: 'load' });
     await prefetchedPage.locator('main h1', { hasText: 'What are you working on?' }).waitFor({ timeout: CONTENT_TIMEOUT_MS });
     const attackLink = prefetchedPage.locator('main [data-tour="mode-attack"]');
     await attackLink.hover();