Skip to content

Blog charts: waterfall and US state tile map; draw the 2025 SPM post's figures with them - #1232

Merged
MaxGhenis merged 5 commits into
mainfrom
max/spm-post-design-system-charts
Sep 29, 2026
Merged

MaxGhenis merged 5 commits into
mainfrom
max/spm-post-design-system-charts

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #1231 and includes its commit (59d35092), so a merge commit lands both. Its review fixes for #1231 are in the second commit here. I retargeted this PR to main so the full CI runs on the whole stack.

What changes

The 2025 SPM post embedded three static PNGs. They are now ```chart blocks drawn with Recharts and the @policyengine/ui-kit chart defaults (axis, grid, tooltip and legend styles, --chart-* / --primary tokens, Inter, PolicyEngineWatermark), so they match the app's charts and reflow on phones. The post's text is unchanged, and the three PNGs are deleted.

  • waterfall type (BlogWaterfallChart.tsx): one small-multiple panel per data row, all on a shared zero-based axis. Steps are level (a bar from zero) or change (a bar from the running total). Bar geometry comes from ui-kit's computeWaterfallData, with dashed connectors at each running total as in PEWaterfallChart. A later level must equal the running total, or the spec is rejected.
  • stateMap type (BlogStateMap.tsx): the 11 by 8 tile layout from the slides' state map (the same positions as state_tile_map.py). Tiles are shaded by bin from the background to --primary. There is a summary line, a legend, an optional note, hover tooltips for a mouse and tap-to-toggle tooltips for touch, and an aria-label on each tile.
  • Shared helpers move to blogChartUtils.ts, and BlogChart.tsx becomes a dispatcher with one figure frame (title, subtitle, watermark, source) for all three types.

Why not use ui-kit's components directly

  • PEWaterfallChart (ui-kit 0.4.0, and the app's WaterfallChart, which copies it) prints labels in white inside bars and only on bars at least 20 px tall. The post's steps are 0.4 to 1.2 points on a 0 to 20 axis, about 5 to 15 px tall, so every step label would disappear. Its connector stroke is var(--color-gray-300), which is undefined on the website: that variable is emitted as --tw-color-gray-300 under prefix(tw), which I checked in the running page. The blog waterfall therefore uses ui-kit's geometry and labels bars above and below, like the blog bar chart, with connectors in --border-dark.
  • HexagonalMap sizes its SVG in fixed pixels, with no responsive scaling. It positions its layout, watermark and tooltip with unprefixed classes (inline-flex, absolute bottom-1 right-2, fixed z-50 pointer-events-none) that the website's Tailwind build does not generate. It also ships no state layout (only congressional districts), and its hexes show IDs without values. The tile map is built from inline token styles instead.

Numbers

Every chart value is copied by script from analysis/asec-2026-anchored-thresholds at 4b8df66 on the branch of spm-threshold-paper#8, which is still open; the post already cites that PR:

  • Waterfall: waterfall_data.json, all four groups and all four steps.
  • State averages: census_table17_state_3yr.csv. I checked all 51 rows against the Census Table 17 xlsx: 0 mismatches. The "United States: 13.0%" line is the xlsx's US row.
  • Threshold effect: by_state.csv, which also agrees with state_age_results.json. The US figure, 0.8, is by_age.all.effect_pp.

An independent script parsed the three blocks back out of the markdown and checked them against those files: 187 checks, all passing. It also confirmed that the post's prose agrees with the chart data (Alabama 1.7, Nebraska 0.0, NY, MT, IN, DE and AZ, 28 states above twice their SE, and the Maine to Louisiana range with CA, MS and FL). JS toFixed and Python .1f produce the same label for all 118 displayed labels (102 tiles and 16 waterfall labels). JS binIndex and the script's v < bound rule put every state in the same bin. So tiles, bars and shading match the old PNGs.

Invariants (property-tested with fast-check)

  • Waterfall. For any step sequence whose later levels equal the running total, the spec parses. Every bar height is at least 0, each change bar spans the running totals before and after it, and each level bar spans zero to the level. Moving a later level off the running total rejects the spec.
  • Bins. Every finite value lands in exactly one bin. The value is below that bin's bound and at or above the previous bin's bound. The bin index never decreases as the value rises.
  • Shades. Each higher bin mixes in strictly more primary, from 12% to 100%, and only the top bin uses inverse text.
  • niceTicks. The domain holds zero and every value, the ticks start and end on the domain, and they are evenly spaced.
  • Corpus. Every ```chart block in every post parses. An invalid spec renders nothing, so a broken block would otherwise vanish silently.

The shared chartColor helper now checks own properties only, so a color name like "constructor" no longer resolves to a function.

Review fixes (second commit)

An independent Opus review of #1231 and #1232 made three findings, and adversarial refuters reproduced each one before I fixed it:

  • Reading time counted the chart JSON. The SPM post's header moved from "10 min read" to "14 min read". A chart block now counts as the prose in it (title, subtitle, note, source), which keeps the SPM post at 10 minutes and the prediction post at 13, the same as main. A test pins both.
  • Every research article shipped Recharts. MarkdownFormatter imported BlogChart eagerly. It now goes through LazyBlogChart (next/dynamic, still server-rendered). I loaded two chart-free articles headless: both return 200 and request 0 Recharts or BlogChart modules. The SPM post's figures are in the server HTML.
  • Design-system bar charts for blog posts; use one in the graded prediction post #1231's bar chart body never rendered in tests. It does now, with labels on either side of zero, ticks and the legend.

Smaller fixes:

  • Specs that parsed could still throw during render. Examples are format.decimals outside 0 to 100, a non-string title or legend, and null or string bar values. The parsers now reject them, and formatChartValue clamps decimals.
  • The positive color used an undefined --success. It now falls back to the series palette.
  • The waterfall legend lost LEGEND_STYLE's top padding. It has it back.
  • Figures took their accessible name from the source line. They now take it from the title.
  • Map tooltips could run off the map on narrow phones. Tooltips near either edge now anchor to that edge.
  • A tapped tooltip could only be closed by tapping the same tile. It now also closes on a tap outside the map.
  • The prediction chart's source line misdescribed Table 5. It said Table 5 gives changes; it now reads "Table 5 (rates to two decimals; changes are their differences)", because Table 5 publishes rates.

Round two was a fresh Opus reviewer. It confirmed findings 1 to 3 are resolved:

  • Reading time matches main for all 203 posts.
  • No chart module is reachable from the static import graph of ArticleClient.
  • The new bar tests catch all 10 mutations of BlogChart tried against them.
  • The server render and hydration work under next/dynamic.

It approved #1232, and approved #1231 only if it lands together with #1232, which this merge does.

It also left three nits, fixed in the third commit:

  • niceTicks now falls back to a finite axis near the largest double, where it used to throw "Invalid array length".
  • The chart-fence pattern now accepts trailing spaces or a CR on the fence lines.
  • The reading-time test now calls the same readingTimeLabel that ArticleClient uses.

The last delta, e675f80d, fixes one remaining nit from the delta review of d3058d6f: mixed-sign values near the largest double, such as -1.29e308 and 1, still threw, so niceTicks now guards on end - start. The same reviewer approved this head after these checks:

  • 0 throws over about 300k full-range inputs.
  • 0 differences from fea5501d over 20,009 realistic inputs.
  • tsc, Vitest and ESLint all pass.

Two cases remain unhandled; no post uses either:

  • a closing fence of 4 or more backticks;
  • a closing fence indented 1 to 3 spaces.

Round-one findings I left as they are:

  • The effect map's summary now reads "United States: 0.8 points". The PNG said "0.8". I added the unit to match the tooltip.
  • Tiles near a bin edge. A tile can read "0.8" and sit in the bin below, because bins use the raw value (0.75). The PNG did the same.
  • Raw JSON outside the website. The legacy app/ renderer and the llms.txt and prerender scripts still show chart blocks as raw JSON. The app/ blog renderer isn't served in production, and the llms/prerender output is a separate follow-up.

Verification

  • cd website && bunx tsc --noEmit && bunx vitest run: 31 files, 195 tests pass. bunx eslint . --max-warnings 0 is clean. bun run scripts/check-markdown-drift.ts passes. app Prettier check passes.
  • Local Next dev server, headless Chromium at 1280 px and 375 px:
    • All three figures render: 51 tiles on each map and 4 waterfall panels. No PNG or raw JSON is left on the page, and no element overflows its figure.
    • The page has no horizontal scroll at 375 px.
    • Map tiles are 69 by 52 px at 14 px text on desktop, and 29 by 22 px at 8 px text on mobile.
    • The waterfall shows 2 by 2 panels on desktop and 1 column on mobile.
    • The mouse hover tooltip and the tap-toggle tooltip both work. A first build toggled the tooltip off on the tap's own click; that is fixed and covered by a test.
  • The page's one console error is a hydration mismatch in ShareLinks, where share URLs are built from window.location. This PR does not touch that code.

axiom: n/a: blog presentation (UI) change, no policy logic

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits September 29, 2026 11:44
…n post

Adds a ```chart block to the website's blog renderer. The block holds a
JSON spec (series, data, number format) and BlogChart renders it with
Recharts and the ui-kit chart defaults the app's charts use: axis, grid,
tooltip and legend styles, the --chart-* color tokens, Inter, and the
PolicyEngine watermark. Header and label styles use token CSS variables
inline because the website's Tailwind build uses a prefix that ui-kit's
utility classes lack.

The graded prediction post replaces its matplotlib PNG with a bar chart of
the graded quantity: the 2024-to-2025 change, Census against the
registered prediction, by group.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s figures with them

The 2025 SPM post embedded three static PNGs (a four-panel waterfall and two
state tile maps). They are now ```chart blocks rendered with Recharts and the
ui-kit chart defaults, so they use Inter, the chart color tokens and the
PolicyEngine watermark like the app's charts, and they reflow on phones.

- BlogChart gains a "waterfall" type (small-multiple panels on a shared
  zero-based axis; geometry from ui-kit's computeWaterfallData, dashed
  connectors at each running total) and a "stateMap" type (the slides'
  11 x 8 tile layout, binned primary-color shades, hover/tap tooltips).
- ui-kit's PEWaterfallChart labels only bars at least 20px tall, which drops
  the post's 0.4 to 1.2 point steps, and its HexagonalMap sizes its SVG in
  fixed pixels and positions its layout and tooltip with unprefixed utility
  classes the website does not generate; hence token-styled bodies.
- Shared helpers move to blogChartUtils.ts. niceTicks now steps by index and
  gives spans under 1e-9 a unit axis (a subnormal value produced a NaN axis;
  values under ~1e-100 made toFixed throw).
- The chart numbers are copied verbatim from spm-threshold-paper
  (waterfall_data.json, by_state.csv, state_age_results.json,
  census_table17_state_3yr.csv); the post's text is unchanged. The three
  PNGs are deleted.
- Tests: parsing, geometry, rendering and tooltips, with fast-check
  properties for the waterfall running total, bin assignment and niceTicks,
  plus a check that every chart block in every post parses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Sep 29, 2026 5:22pm UTC
policyengine-website Ready Ready Preview Sep 29, 2026 5:22pm UTC

Request Review

…hrow

Fixes from the independent review of #1231 and #1232:

- Reading time counted chart JSON as words, moving the 2025 SPM post from
  "10 min read" to "14 min read". A chart block now counts as the prose a
  reader reads in it (title, subtitle, note, source), which keeps the SPM
  post at 10 minutes and the graded prediction post at 13, as on main.
- MarkdownFormatter imported BlogChart eagerly, so every research article
  shipped Recharts. It now goes through LazyBlogChart (next/dynamic, still
  server-rendered); articles without a chart block load none of it.
- Specs that parsed could still throw during render (format.decimals outside
  0 to 100, non-string title or legend, null or string bar values). The
  parsers now reject them, and formatChartValue clamps decimals.
- The bar chart body is now rendered in tests (labels either side of zero,
  ticks, legend), alongside tests for the new rejections, reading-time
  parity, and pinned values from the published SPM figures.
- Smaller: "positive" (an undefined --success on the website) falls back to
  the series color; the waterfall legend keeps LEGEND_STYLE's top padding;
  figures take their accessible name from the title; map tooltips near
  either edge anchor to it, and a tapped tooltip closes on a tap outside the
  map; the prediction chart's source says Table 5 publishes rates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- niceTicks falls back to a [lo, hi] axis when rounding out to a step
  overflows (values near the largest double threw an invalid array length),
  and finds the range with a loop rather than spreading every value.
- The chart-fence pattern accepts trailing spaces or a CR on either fence
  line, so a loosely closed block no longer swallows the prose after it.
- Reading time lives in chartReadingText.ts as readingTimeLabel, used by
  ArticleClient and called directly by the test that pins the SPM posts at
  10 and 13 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mixed-sign values near the largest double (for example -1.29e308 and 1)
round to finite ends whose difference overflows, which still threw an
invalid array length. The fallback now checks end - start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merge audit (merged by an agent session on Max's standing rule for his own PRs; the only unmet gate was the required approving review, which Max can't give on his own PR):

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — e675f80d Deployed Sep 29, 2026 by vercel[bot]
Preview – policyengine-calculator-next — e675f80d Deployed Sep 29, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant