Fix how v3 reports run, rerun and record their results by PavelMakarchuk · Pull Request #1282 · PolicyEngine/policyengine-app-v2

PavelMakarchuk · 2026-10-15T13:01:16Z

Summary

Fixes eight problems in how the v3 (flagship) report page runs, reruns and records a report. Each fix has a test that failed first. Legacy v2 pages are not changed. The shared calculation code gets one opt-in flag only, and its default stays as it was.

Problems and fixes

A shared link rebuilt the reform for 2026 without its dated values (medium). With no local stash, provenanceFromPolicy read the reform in CURRENT_YEAR and dropped parameter.values. Recompute, "Edit in Build" and "Edit the reform" then ran one flat value from 2026. Now the page passes report.year. The rebuilt provisions keep the policy's intervals, and their value and current law are read on 1 January of the report year (reportProvenance.ts, Report.page.tsx).
The adjust panel kept empty values when the reform policy loaded after the results (high). Recompute was enabled and sent a reform with no values. Now the page waits for the reform policy (unless that policy or the metadata failed to load). The panel also starts over when its provisions or title change: it is keyed on them (Report.page.tsx, ReportAdjustPanel.tsx).
Recompute applied an edited value from 2026, not from the report year (medium). For a 2022–2025 report, the recompute showed no change. Now an edited value holds from 1 January of the report year, on top of the provision's earlier dated values, as Build's inline edit does. The saved reform gets the same intervals (ReportAdjustPanel.tsx).
Recompute linked to the wrong saved reform, or saved a flattened reform (medium). The panel's own reformMatches compared only names and the first value. It now uses findMatchingReform on the intervals that run, and it saves those same intervals (ReportAdjustPanel.tsx).
A failed society-wide run was never recorded, so "Failed" never showed in Your reforms (low). I added an opt-in flag, CalcStartConfig.recordReportErrors. Only the flagship report sets it. With the flag, CalcOrchestrator calls the new ResultPersister.persistReportError. This marks the report record as error, with the reason. It does so both when the first answer is a failure and when a later poll is. A record that already says error is not written again (CalcOrchestrator.ts, ResultPersister.ts, CalcStartConfig.ts, Report.page.tsx).
Every open re-ran the economy request and PATCHed about 100 KB of output to the public record, even when it was complete (medium). The page now seeds the calculation cache from report.output with the existing, unchanged useHydrateCalculationCache, when the record is complete and the report is society-wide. So no run starts and nothing is written (Report.page.tsx).
The v3 route accepted household report ids, overwrote their stored output and crashed (high). The page now finds household simulations, starts no calculation for them, and says "This report cannot open here" (Report.page.tsx, new ReportNotSocietyWide.tsx).
Your reforms did not list a new report for up to 5 minutes (medium). A successful run in useRunFlagshipReport now invalidates the reports list and the report statuses. Their query keys are now in one place, flagshipReportQueryKeys in runReport.ts, which Reforms.page.tsx and ReformReports.tsx use.

Not fixed

PATCH writes a stale row while GET serves a newer row (the 4326 → 4333 alias), so the record stays pending. The cause is in policyengine-api: update_report_output selects the requested row, and resolve_report_output maps an old row to the current one. The fix belongs in the API (PATCH must resolve the same row as GET). The app has no safe workaround: it cannot see the current row id, and getting the badge from the economy endpoint would cost one economy request for each report in the list.

Seen in passing, left as is (shared v2 code)

In CalcOrchestrator.startPolling, when the first economy answer is already error, the observer calls its listener synchronously inside subscribe(). Then unsubscribe is still in its temporal dead zone. The ReferenceError is swallowed, and the manager never cleans up that orchestrator. The v3 opt-in path handles a failure at once before it starts to poll. The default path is not changed.

v2 impact

None by default. recordReportErrors is off unless a caller sets it, and only pages/flagship/Report.page.tsx sets it. The new tests show that without it a failed run writes nothing, both at once and after polling. persistReportError is a new method, and persist() did not change. useHydrateCalculationCache did not change, and no v2 page uses it today. All other changes are in flagship files.

Tests

New: ReportRunLifecycle.test.tsx (real calculation machinery, with only the API stubbed: pending, failed and already-complete reports), ReportSharedLink.test.tsx, useRunFlagshipReport.test.tsx.
Extended: ReportAdjustPanel.test.tsx, reportProvenance.test.ts, ReportLoadStates.test.tsx, CalcOrchestrator.test.ts (default-unchanged and opt-in cases), ResultPersister.test.ts.
bun run typecheck: pass. eslint --max-warnings 0 and prettier --check on the changed files: pass. bun run vitest: 431 files, 4955 passed, 10 skipped.

Note: this PR is based on the open #1274 (fix/v3-relevant-comparisons, which now contains main). Retarget it to main after #1274 merges.

🤖 Generated with Claude Code

Each paged read of a tracker table now sends an order on the table's key. Without ORDER BY, Postgres gave the two 1,000-row pages of processed_bills in a different order on each load, so some rows came back twice and others were never read. Those bills lost their stage badge, their bill text link and their place in the stage filter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The provisions rebuilt from the reform policy took today's current law, the 2026 value (or the first interval), and no dates, so a shared 2027 report read 'From Jan 1, 2026'. They now carry the policy's dated values and pass through provisionsForYear for the report year, as the run's own stash does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Each paged tracker read now names a stable order by the table's key. Without one, page 2 of processed_bills could repeat rows of page 1 and skip others, so bills lost their stage and bill-text link. When two status rows share a bill number, the one with the latest action wins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An id with words after the number (hi-sb3125-hd1, nj-a1185-marriage-penalty-elimination, ut-sb60-bill) matched no processed_bills row, so the bill showed no stage. After the id, the join now tries the title's bill number, as the tracker does, and then the id's first part. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The stored budget card on Economic impacts now takes the bill and uses the overview's own wording: "Annual revenue loss" for a federal bill, "Annual state revenue loss" for a state bill, with the amount unsigned next to the loss or gain word, as the overview shows it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Each paged tracker read now names a unique order column (bill_id for processed_bills, id for the rest). Without one, Postgres gave page 2 of processed_bills rows already on page 1, so about 418 of 1,418 status rows were lost on each load, and a different set each time. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Open as draft reform started a national draft for this year, so a bill from 2027 scored no change from Build and a state bill ran nationwide. The draft now opens with the bill's run year and state, as the bill's report and Run impact report do. loadProvisionsIntoDraft takes an opt-in source; without one the draft stays hand-built. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…tage When two processed_bills rows share one normalized bill number (NY "S 4487" and "S04487"), the feed kept whichever row it read last. It now keeps the row with the higher LegiScan bill id, the later bill, so the stage and the bill text link no longer depend on the read order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The Economic impacts tab labelled every stored budget 'Annual state revenue', so HR 1425's -$225.5bn read as state revenue while the Overview said 'Annual revenue loss'. The stored budget now takes the bill's state and words it as the Overview does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Delete in the reform detail now opens a confirm dialog that names the reform and says how many of its reports leave the list (they still open from their links). A failed delete or duplicate shows an error, and the error clears when another reform opens. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The tracker's fiscal_note_source was read but never shown, and a note stored as a range without a point estimate lost its link, so 'Within fiscal-note range' cited a note no one could open. The fiscal-note row now takes the stored source as its name, and a note without an estimate is named and linked under the accepted range, or in a row of its own when no range is stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…its dates A report opened without its local stash rebuilt the reform for 2026 and dropped its dated values, so the overview showed the 2026 value against today's law, and Recompute, Edit in Build and Edit the reform ran a flat value from 2026. The rebuilt provisions now keep the policy's intervals, and their value and current law are those of the report's year. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

reform / baseline - 1 is NaN (or Infinity) for a group with no one in poverty before. Recharts drew no bar for it, so the value labels, which go by the index of the bars drawn, moved onto the next group: 'NaN%' sat on All and All's own value was lost. Such a group now draws a bar of zero that reads 'n/a', and its hover and the title say the change in points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

On a shared link the results can arrive before the reform policy, which waits for the parameter metadata. The page rendered with no provisions, and the adjust panel kept its first, empty values and name: Recompute was enabled and ran a reform with no values. The page now waits for the reform policy (unless it or the metadata failed), and the panel starts over whenever the reform it adjusts changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… re-run UT SB60's tracker row scores 0.045, the law before the bill, under a Policy changes table of 4.5% to 4.45%, so Load full results showed an $85.0m revenue gain for a rate cut. The bill report now compares the values a run would score with the write-up's 'after' values. When they disagree, the page keeps the tracker's stored estimate and says why the full results are not available. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…lead The API's winner shares weight each household by its people, so the overview tile read a share of people as 'Households gaining' and the winners tooltip named the row 'All households'. They now say 'People gaining income' and 'All people'. When more people lose than gain, the tile shows 'People losing income' instead of a bare 0.0% gaining. The tracker's stored figure on a bill report reads the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

An edited provision ran, and was saved, as one value from 2026, so a 2024 or 2025 report recomputed to no change. An edit now holds from 1 January of the report's year, over the provision's earlier dated values, as Build's inline edit does, and the saved reform has the same dates as the run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Recompute matched saved reforms on parameter names and the first value alone, so a phased reform could run linked to, and titled as, a flat one. It now uses findMatchingReform, as Build does: a saved reform is reused only when it sets the same values over the same dates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Recharts' default tick interval hid category labels that crowd, so on a 375px screen the Hispanic bar, the Top 10% share bar and the state tax bar had a value and no name. The report's bar charts and the budget waterfall now show every tick: each label wraps by word to its own slot, the font shrinks until the longest word fits, and the x-axis grows to hold the label with the most lines. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The flagship report route accepted any API report id. For a household report it started a report-level calculation, which scored one household and wrote that over the report's stored per-simulation outputs, then crashed reading the household result as society-wide. The page now runs only society-wide reports and says a household one cannot open here. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

react-simple-maps gives every district shape tabindex 0, with no name and, here, no focus outline, so a keyboard user pressed Tab hundreds of times to pass the map. The districts now take no focus, and the map reads as one image whose name points to the ranked district lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This reverts commit 9ae8107. The paged tracker reads get their stable order in PR #1277 (fix/v3-audit-security), so this branch does not carry its own copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The card's SVG download skipped every SVG under 40px as an icon, so the compact 28px All row was dropped, and the legend was HTML, which the download does not copy. The legend now draws in SVG, and the chart marks its All row and legend with CHART_EXPORT_KEEP, an opt-in marker the download keeps at any visible size. Unmarked SVGs are filtered as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…aders The search input is a combobox whose focus stays in the input while the arrow keys move the highlight, but nothing told a screen reader which row was highlighted, so Enter added a parameter the user had not heard. Each result now has its own id, the input points at the highlighted one with aria-activedescendant (none while a folder is pointed at, the list is out of date or no results show), and it says it filters a list with aria-autocomplete="list". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…o fix/v3-audit-states-a11y Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…o fix/v3-audit-report-content

…o wip/audit2-security

Every open of a report ran the economy request again and wrote the full results back over the public record, even when that record was already complete. The page now seeds its calculation from the record's stored output when the record is complete, so no run starts and nothing is written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…o wip/audit2-bills

The Relative view, its legend and its rankings rounded to one decimal, so a state reform that moves each district by thousandths of a percent read 0.0% throughout, and '-0.0%' at the legend's end. Percentages now take the decimals the largest value needs (one to four), and a value that rounds to zero reads without a minus. A state with fewer than ten districts splits them between the two rankings instead of listing some in both, and a view where every district is zero, such as Loser % when no one loses, says so instead of ranking zeros. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

billScoresOtherThanWriteUp flagged a bill when any paired provision held no value near the write-up's "after". Correct bills lost "Load full results": a filing-status change written with one status's value, a yearly write-up of a monthly parameter, a step ("+1%") or a percent stored as a whole number. The check now needs evidence of a swap: the named provisions together hold the write-up's "before" value and none of its "after" value. A signed "after" next to an unsigned "before" is a step and is not checked. A stored value also reads as a percent the other way (4.45 for "0.0445"). UT SB60 (0.045 for "4.5% → 4.45%") still flags. changeNamesPath is exported, so the Policy changes table can share it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The 1,001-bill test now gives every reform_impacts row its baseline params and checks that every bill's provisions carry that baseline. Before, the baselines read could lose its row order and the test still passed; now that read drops the 1,001st bill's baseline and the test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…red helper ReportChanges copied changeNamesPath line by line as namesPath. It now imports the one in billProvisions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The federal page test passes with the page's state prop removed, since no state already gives the federal words. This test fails then: a Utah bill must read "Annual state revenue loss" on Economic impacts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The feed maps a null fiscal_note_url to undefined but keeps "". With no estimate, an empty link then gave an "Official fiscal note · No estimate" row with no link and no source. The estimates table now treats a blank link as absent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Your reforms cached its list of reports for the app's five minutes, and nothing told it a run had added one: back on the page, the reform still said "No reports yet". A run now invalidates the list and the reports' statuses, whose query keys live in one place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…o fix/v3-audit-run-lifecycle Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The test asserted the draft's year, which draftYear already gave before the fix. It now reads the year the draft runs in, and asserts what the fix changed: the draft opens with source "bill" and no state region. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… no draft The bill report stopped re-running UT SB60, but on the Reforms page "Open as draft reform" still loaded its swapped 0.045 into Build, and a run there scored the same +$85m increase. The bill panel now uses the same check: "Run impact report" and "Open as draft reform" are disabled, and a note says why. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel · 2026-10-15T13:01:22Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-calculator-next	Ready	Preview	Oct 15, 2026 1:02pm UTC
policyengine-website	Ready	Preview	Oct 15, 2026 1:02pm UTC

Validation: baseline comparisons for the programs the reform changes

…rity

…es-a11y

…rt-content

# Conflicts: # app/src/api/billFeed.ts # app/src/tests/unit/api/billFeed.test.ts

…mp/audit-int # Conflicts: # app/src/components/flagship/report/StoredImpactCharts.tsx # app/src/pages/flagship/BillReport.page.tsx # app/src/tests/unit/components/flagship/report/StoredImpactCharts.test.tsx

…o tmp/audit-int # Conflicts: # app/src/components/flagship/report/charts/DeepPovertyImpactByAgeSubPage.tsx # app/src/components/flagship/report/charts/DeepPovertyImpactByGenderSubPage.tsx # app/src/components/flagship/report/charts/InequalityImpactSubPage.tsx # app/src/components/flagship/report/charts/PovertyImpactByAgeSubPage.tsx # app/src/components/flagship/report/charts/PovertyImpactByGenderSubPage.tsx # app/src/components/flagship/report/charts/PovertyImpactByRaceSubPage.tsx # app/src/components/flagship/report/charts/ReportWaterfallChart.tsx

… tmp/audit-int # Conflicts: # app/src/libs/flagship/reportProvenance.ts # app/src/pages/flagship/Report.page.tsx # app/src/tests/unit/libs/flagship/reportProvenance.test.ts

PavelMakarchuk · 2026-10-15T15:56:53Z

Duplicate of #1283, which merged the same branch (fix/v3-audit-run-lifecycle).

🤖 Generated with Claude Code

PavelMakarchuk and others added 30 commits October 14, 2026 18:11

Reforms: format the bill feed and validation changes

e36a199

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Bills: leave the tracker row order to #1277

78ab5cd

This reverts commit 9ae8107. The paged tracker reads get their stable order in PR #1277 (fix/v3-audit-security), so this branch does not carry its own copy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…

564383a

…o fix/v3-audit-states-a11y Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…

2e03ccd

…o fix/v3-audit-report-content

Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…

1d9ad46

…o wip/audit2-security

Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…

bbd0020

…o wip/audit2-bills

PavelMakarchuk and others added 10 commits October 15, 2026 08:53

Report: the Policy changes table reads change parameters with the sha…

b45f2d1

…red helper ReportChanges copied changeNamesPath line by line as namesPath. It now imports the one in billProvisions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…

90443cf

…o fix/v3-audit-run-lifecycle Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel Bot deployed to Preview – policyengine-website October 15, 2026 13:01

View deployment

vercel Bot deployed to Preview – policyengine-calculator-next October 15, 2026 13:02

View deployment

PavelMakarchuk and others added 8 commits October 15, 2026 09:40

Merge pull request #1274 from PolicyEngine/fix/v3-relevant-comparisons

ae32545

Validation: baseline comparisons for the programs the reform changes

Merge remote-tracking branch 'origin/main' into tmp/fix/v3-audit-secu…

4e413aa

…rity

Merge remote-tracking branch 'origin/main' into tmp/fix/v3-audit-bills

ac1b7a0

Merge remote-tracking branch 'origin/main' into tmp/fix/v3-audit-stat…

5e14e63

…es-a11y

Merge remote-tracking branch 'origin/main' into tmp/fix/v3-audit-repo…

a4b97ac

…rt-content

Merge branch 'tmp/fix/v3-audit-security' into tmp/bills-on-1277

ba9a650

# Conflicts: # app/src/api/billFeed.ts # app/src/tests/unit/api/billFeed.test.ts

Merge commit '78f54f96' into tmp/bills2

73fce16

# Conflicts: # app/src/api/billFeed.ts # app/src/tests/unit/api/billFeed.test.ts

Merge remote-tracking branch 'origin/main' into tmp/rl

9004535

vercel Bot deployed to Preview – policyengine-website October 15, 2026 13:46

View deployment

vercel Bot deployed to Preview – policyengine-calculator-next October 15, 2026 13:46

View deployment

PavelMakarchuk added 3 commits October 15, 2026 09:47

Merge remote-tracking branch 'origin/fix/v3-audit-states-a11y' into t…

ade72f8

…mp/audit-int # Conflicts: # app/src/components/flagship/report/StoredImpactCharts.tsx # app/src/pages/flagship/BillReport.page.tsx # app/src/tests/unit/components/flagship/report/StoredImpactCharts.test.tsx

Merge remote-tracking branch 'origin/fix/v3-audit-run-lifecycle' into…

8172f5c

… tmp/audit-int # Conflicts: # app/src/libs/flagship/reportProvenance.ts # app/src/pages/flagship/Report.page.tsx # app/src/tests/unit/libs/flagship/reportProvenance.test.ts

vercel Bot deployed to Preview – policyengine-website October 15, 2026 13:52

View deployment

vercel Bot deployed to Preview – policyengine-calculator-next October 15, 2026 13:52

View deployment

PavelMakarchuk closed this Oct 15, 2026

Navigation Menu

Repository navigation

Fix how v3 reports run, rerun and record their results - #1282

Fix how v3 reports run, rerun and record their results#1282
PavelMakarchuk wants to merge 56 commits into
fix/v3-relevant-comparisonsfrom
fix/v3-audit-run-lifecycle

PavelMakarchuk commented Oct 15, 2026

Uh oh!

vercel Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

PavelMakarchuk commented Oct 15, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

PavelMakarchuk commented Oct 15, 2026

Summary

Problems and fixes

Not fixed

Seen in passing, left as is (shared v2 code)

v2 impact

Tests

Uh oh!

vercel Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PavelMakarchuk commented Oct 15, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 15, 2026 •

edited

Loading