Skip to content

Return 404 for calculator paths whose first segment is not a country - #1365

Merged
MaxGhenis merged 4 commits into
mainfrom
fix/calculator-unknown-country-404
Oct 20, 2026
Merged

MaxGhenis merged 4 commits into
mainfrom
fix/calculator-unknown-country-404

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

app.policyengine.org returns a 200 HTML calculator page for any first path segment. Production, 2026-10-20:

Path Status x-matched-path
/llms.txt 200 text/html /[countryId]
/favicon.ico 200 text/html /[countryId]
/US 200 text/html /[countryId]
/xx 200 text/html /[countryId]

calculator-app/src/app/[countryId]/layout.tsx was a client component. For an unsupported country it rendered null and called router.replace("/") in the browser. The server had already sent a 200 shell by then. These soft 200s hide real 404s from crawlers and monitoring, and they make the calculator answer for paths it doesn't own. One was found while porting llms.txt to the website in #1332.

Change

  • [countryId]/layout.tsx is now a server component, like the website's. It awaits params and calls notFound() unless isCountryId(countryId).
  • [countryId]/CountryShell.tsx (new, "use client") holds the Country, Navigation and Location providers and CalculatorProviders, moved over unchanged.
  • app/src/libs/countries.ts adds isCountryId(value: unknown): value is CountryId. It is an exact match against countryIds, so US and " us" are not countries.
  • calculator-app/src/app/not-found.tsx (new) is the calculator's 404 page. It has a "Page not found | PolicyEngine calculator" title and a link back to /, styled like error.tsx. Next adds <meta name="robots" content="noindex"> to it.
  • ARCHITECTURE-COUNTRY-ID.md now describes the 404 behaviour.
  • fast-check is added to app devDependencies at the same ^4.10.2 the website already uses. The lockfile diff is that one line.

Review nits (round 2): the 404 page uses colors.text.secondary and typography.lineHeight.none, and NotFound.test.tsx renders through @test-utils. CountryLayout.test.tsx keeps @testing-library/react on purpose: @test-utils wraps a CountryProvider('us') of its own, which would hide a layout that failed to provide the country.

The supported list stays countryIds (us, uk, ca, ng, il). If #1345 removes countries, the 404 follows that list automatically.

Nothing legitimate moves

Next serves public/ files (calculator-app/public → app/public), metadata routes (/robots.txt) and static route folders (/api/*) before the dynamic [countryId] route, so the layout never sees those paths.

Route table. next build gives the same rendering mode for every route on main and on this branch: every /[countryId]/* route is still ƒ (rendered on demand), and / and /robots.txt are still static. The only difference is that /_not-found is now the custom page.

Local production builds (next start) of main and of this branch were each sent 43 paths. 30 return the same status on both. The 13 that changed all went from 200 to 404, and every one of them has a first segment that isn't a supported country:

Unchanged Changed, 200 → 404
/ 200 · /us /uk /ca /ng /il 200 · /us/reports /uk/policies /us/build /us/ask /us/account /us/households /us/simulations/create /us/report-output/abc /us/report-output/abc/config /us/report/123 /us/callback 200 · /us/no-such-page 404 · /robots.txt 200 · /favicon.svg /assets/logos/policyengine/teal.svg /assets/logos/policyengine/teal-square.png 200 · /api/geolocate 307 (rewrite) · /api/calibration-targets /api/model-validation 400 · /api/scorecard-comparisons 405 · /api/does-not-exist /.well-known/security.txt /data/geojson /_next/static/does-not-exist.js 404 /llms.txt /llms-full.txt /favicon.ico /sitemap.xml /wp-login.php /US /Uk /usa /gb /xx /%20us /xx/reports /llms.txt/reports

Invariants

  1. The layout 404s exactly when the segment isn't a supported country. For every first path segment s, the layout renders the calculator if s ∈ countryIds. Otherwise it calls notFound() exactly once and renders nothing. This is a fast-check property over ASCII and full-Unicode strings, country ids with padding around them, and upper-cased ids. Example tests cover each country, and real crawler paths such as llms.txt, favicon.ico and wp-login.php.
  2. isCountryId matches exactly. For any JS value (fc.anything()), isCountryId(v) implies v ∈ countryIds. A country id with any non-empty padding is never a country id. Upper case never matches.
  3. Nothing shadows a country route. No top-level entry in calculator-app/public or calculator-app/src/app is named exactly after a country. Next serves those before [countryId], so one would shadow that country's calculator. A test also checks that calculator-app/public still resolves to app/public.

Mutation check. Each mutant was swapped in, the suite was run, and the fix was restored:

  • No notFound() call: 14 tests fail. Every example test fails with "promise resolved instead of rejecting", and the property fails with expected 'rendered' to be 'not found'.
  • Case-insensitive check: 3 tests fail, with fast-check counterexample ["US"].
  • main's client layout: 19 tests fail. The calculator-shell tests fail with TypeError because a client component can't run as a server layout, so this one is a structural catch rather than a behavioural one.

Checks run locally

  • Prettier and ESLint (app): pass.
  • Typecheck: app and calculator both pass.
  • next build (calculator): passes.
  • Website tests: 336 of 336 pass.
  • App tests: 5,247 pass. The 2 failures are in ScorecardComparisons.test.tsx, which this PR doesn't touch. They fail the same way on untouched main under local Node 26 (£1B vs £1.0B), and Pin the scorecard's compact money format and run CI tests on Node 26 #1363 fixes them. CI's runner Node passes them.

Preview verification

The same 43 paths were curled on the policyengine-calculator-next preview of 19b1022, through the project's existing automation-bypass key, and on production:

  • The 13 paths above go from 200 in production to 404 on the preview. x-matched-path is still /[countryId], but the response is now the 404 page: title "Page not found | PolicyEngine calculator", with <meta name="robots" content="noindex">.
  • The other 30 paths return the same status as production, apart from /api/geolocate below. That covers every country, the deep calculator routes, assets, /robots.txt and the /api/* handlers.
  • /api/geolocate is 307 on the preview and 200 in production. This predates the PR: Pin the scorecard's compact money format and run CI tests on Node 26 #1363's preview, which doesn't have this change, 307s the same way, to https://www.policyengine.org/api/geolocate. Previews proxy to the apex policyengine.org, which redirects to www, while production points straight at a host that answers 200.

Re-check at head b060a4b (after the review nits and a merge of main): the 43 paths were curled again on its preview against production. The same 13 paths go from 200 to 404, /api/geolocate still shows the preview-only 307, and every other path is unchanged.

Independent review

GPT-6.1 Sol reviewed the PR in two rounds, and both returned VERDICT: APPROVE:

  • Round 1 (head 19b1022): no blockers and four nits. It confirmed from the installed Next 15.5.12 source that notFound() in this layout sets a 404 status, because there is no Suspense boundary above it. It also confirmed that the root not-found.tsx and its metadata apply.
  • Round 2 (delta to b060a4b): the nit fixes and the merge of main. It agreed that CountryLayout.test.tsx should keep @testing-library/react.

Sol's lane had no shell, so I ran the checks it couldn't: the merge's diff of app/package.json and bun.lock against main is only the fast-check line, the calculator tests pass, and calculator Prettier and ESLint pass. The FlagshipGate soft 404 that Sol raised can't be reached in production, because the US flagship flag is on unless set to "false".

axiom: n/a: calculator routing (UI infrastructure, no policy logic)

🤖 Generated with Claude Code

app.policyengine.org answered any first path segment with a 200 HTML
calculator shell (x-matched-path: /[countryId]) because the country layout
was a client component that rendered nothing and redirected to / in the
browser. /llms.txt, /favicon.ico, /US and /xx/reports all looked like real
pages to crawlers and monitors.

The layout is now a server component that calls notFound() unless the
segment is a supported country (isCountryId, exact match against
countryIds), so those paths are HTTP 404s. The client providers move to
CountryShell unchanged. A calculator not-found page replaces Next's
default, with a "Page not found" title.

Static files, robots.txt, /api routes and every supported country keep
their status: Next serves those before the [countryId] route, and a local
production build answers 43 paths identically to main except the 13
unknown-first-segment ones, which go from 200 to 404.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 20, 2026 8:55pm UTC
policyengine-website Ready Ready Preview Oct 20, 2026 8:55pm UTC

Request Review

MaxGhenis and others added 2 commits October 20, 2026 16:38
…tils

Review nits on #1365: the 404 page's helper text now uses
colors.text.secondary and its heading typography.lineHeight.none instead
of gray[500] and a literal 1. NotFound.test.tsx renders through
@Test-Utils. CountryLayout.test.tsx keeps @testing-library/react, with a
comment saying why: @Test-Utils wraps its own CountryProvider('us'), which
would hide a layout that failed to provide the country.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merging under the merge gates in Max's agent instructions. The only open block is the ruleset's required approving review. This PR is under Max's account, so it can't approve itself, so I'm admin-merging pinned to --match-head-commit cc5dd03026c36835ce0c9cf233718993ca176b2d.

@MaxGhenis
MaxGhenis merged commit c50f96b into main Oct 20, 2026
10 checks passed
@MaxGhenis
MaxGhenis deleted the fix/calculator-unknown-country-404 branch October 20, 2026 21:13

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — cc5dd030 Deployed Oct 20, 2026 by vercel[bot]
Preview – policyengine-calculator-next — cc5dd030 Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant