Skip to content

Bill feed: complete stages, matching run settings and named fiscal notes - #1278

Merged
PavelMakarchuk merged 25 commits into
mainfrom
fix/v3-audit-bills
Oct 15, 2026
Merged

Bill feed: complete stages, matching run settings and named fiscal notes#1278
PavelMakarchuk merged 25 commits into
mainfrom
fix/v3-audit-bills

Conversation

@PavelMakarchuk

@PavelMakarchuk PavelMakarchuk commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes six confirmed problems in the v3 bill feed, bill report and Reforms page. All changes are in v3 (flagship) code. One shared helper, loadProvisionsIntoDraft, gets an opt-in option; its default is unchanged and tested.

Problems and fixes

  1. The tracker feed lost about 30% of bill-status rows. trackerSelect paged with limit/offset and no order, so page 2 of processed_bills (1,418 rows) could repeat rows of page 1 and skip others. WV SB392 and UT HB290 read "Analyzed", the Enacted filter matched nothing, and 13 cards lost their bill-text link.
    • Fix: each paged read names a stable order by the table's key (id.asc; processed_bills: bill_id.asc,state.asc,bill_number.asc). When two status rows share one bill number (81 live keys, such as MI HB4170), the row with the latest last_action_date wins.
  2. UT SB60 showed a +$85.0m revenue gain under a "4.5% → 4.45%" rate cut. The tracker row stores the law before the bill as reform_params, so a full run scored a tax increase.
    • Fix: the bill report compares the values a run would score with the write-up's "after" values (billScoresOtherThanWriteUp). When they disagree, the page does not offer "Load full results", keeps the tracker's stored estimate, and says why. The check only pairs a change with a provision when the change names the parameter, or when the bill has one provision and one change. It only reads an "after" that is one plain number, and it allows for the write-up's rounding.
    • With fix 5, UT SB60 now also gets its stage ("Signed into Law").
  3. Open as draft reform dropped the bill's run year and state. The draft opened national and in 2026. A 2027 bill then scored no change from Build, and a state bill ran nationwide.
    • Fix: the draft now opens with billRunYear and the bill's state region, the same as the bill report and Run impact report. The source stays bill and the source reference stays the bill id. These come from a new opt-in source / sourceRef option on loadProvisionsIntoDraft.
  4. A federal bill's stored budget read "Annual state revenue loss" on the Economic impacts tab (HR 1425, HR 904), while the Overview said "Annual revenue loss".
    • Fix: StoredImpactCharts takes the bill's state and uses the same words as storedBillMetrics.
  5. The stage join read only the id, so HI SB3125 HD1, NJ A1185 and UT SB60 showed no stage.
    • Fix: when the id finds no row, the join tries the bill number in the title, as the tracker's extractBillNumber does, and then the first part of the id (sb3125 from sb3125-hd1). The state must still match.
  6. Validation hid the fiscal note's source and, for a range-only note, its link. On GA HB1001, "Within fiscal-note range" referred to a note that the page did not show or link.
    • Fix: the fiscal-note row now uses the stored fiscal_note_source as its name, and falls back to "Official fiscal note". A note with no point estimate is named and linked under "Accepted range", or in its own row when no range is stored.

v2 impact

None. Every changed file is v3 code, or a v3 helper that only flagship pages import. loadProvisionsIntoDraft keeps source: 'manual' and no sourceRef when the new option is not passed.

Not fixed here

  • The UT SB60 tracker row itself: reform_params and baseline_params look swapped. That fix belongs in the tracker repo. This PR only stops the app from re-scoring a bill whose parameters disagree with its write-up.
  • The adjust panel on a bill like UT SB60 still starts from the stored value (4.5%). This PR does not hide it, so that ReportView stays unchanged.

Tests

  • New tests, each written to fail first:
    • unordered paging that repeats rows
    • duplicate bill numbers
    • stage join by title and by the first part of the id
    • a 2027 state bill and a federal bill opened as drafts
    • the default and opt-in source of loadProvisionsIntoDraft
    • the federal stored-budget label, in the component and on the page
    • range-only, link-only and named-source fiscal notes
    • billScoresOtherThanWriteUp cases
    • the UT SB60 bill report with no run offered
  • bun run typecheck, bunx eslint and bunx prettier --check on all changed files: clean.
  • bun run vitest: 428 files, 4,939 tests pass.

Note: this PR is based on the open #1274 stack (fix/v3-relevant-comparisons). Retarget it to main after #1273 and #1274 merge.

🤖 Generated with Claude Code

PavelMakarchuk and others added 9 commits October 14, 2026 18:13
@PavelMakarchuk @claude
Reforms: the tracker feed reads every bill-status row once
2e15b9f 
Each paged tracker read now names a stable order by the table's key.
Without one, page 2 of processed_bills could repeat rows of page 1 and
skip others, so bills lost their stage and bill-text link. When two
status rows share a bill number, the one with the latest action wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: bills join their stage by the bill number in the title
303fbf3 
An id with words after the number (hi-sb3125-hd1,
nj-a1185-marriage-penalty-elimination, ut-sb60-bill) matched no
processed_bills row, so the bill showed no stage. After the id, the
join now tries the title's bill number, as the tracker does, and then
the id's first part.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: the tracker feed reads its pages in a fixed row order
e5f7b37 
Each paged tracker read now names a unique order column (bill_id for
processed_bills, id for the rest). Without one, Postgres gave page 2 of
processed_bills rows already on page 1, so about 418 of 1,418 status
rows were lost on each load, and a different set each time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: a bill opened as a draft keeps its run year and state
075a4d7 
Open as draft reform started a national draft for this year, so a bill
from 2027 scored no change from Build and a state bill ran nationwide.
The draft now opens with the bill's run year and state, as the bill's
report and Run impact report do. loadProvisionsIntoDraft takes an
opt-in source; without one the draft stays hand-built.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: a bill filed twice under one number keeps the later bill's s…
9984e07 
…tage

When two processed_bills rows share one normalized bill number (NY
"S 4487" and "S04487"), the feed kept whichever row it read last. It
now keeps the row with the higher LegiScan bill id, the later bill, so
the stage and the bill text link no longer depend on the read order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Report: a federal bill's stored budget is not called state revenue
1ecbe59 
The Economic impacts tab labelled every stored budget 'Annual state
revenue', so HR 1425's -$225.5bn read as state revenue while the
Overview said 'Annual revenue loss'. The stored budget now takes the
bill's state and words it as the Overview does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Validation: the fiscal note names its source and links it
cab31f9 
The tracker's fiscal_note_source was read but never shown, and a note
stored as a range without a point estimate lost its link, so
'Within fiscal-note range' cited a note no one could open. The
fiscal-note row now takes the stored source as its name, and a note
without an estimate is named and linked under the accepted range, or
in a row of its own when no range is stored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Report: a bill whose stored parameters contradict its write-up is not…
f721182 
… re-run

UT SB60's tracker row scores 0.045, the law before the bill, under a
Policy changes table of 4.5% to 4.45%, so Load full results showed an
$85.0m revenue gain for a rate cut. The bill report now compares the
values a run would score with the write-up's 'after' values. When they
disagree, the page keeps the tracker's stored estimate and says why the
full results are not available.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: format the bill feed and validation changes
e36a199 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 14, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 15, 2026 1:45pm UTC
policyengine-website Ready Ready Preview Oct 15, 2026 1:45pm UTC

Request Review

PavelMakarchuk and others added 12 commits October 15, 2026 08:44
@PavelMakarchuk
Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…
1d9ad46 
…o wip/audit2-security
@PavelMakarchuk
Merge remote-tracking branch 'origin/fix/v3-relevant-comparisons' int…
bbd0020 
…o wip/audit2-bills
@PavelMakarchuk @claude
Reforms: leave the tracker row order to the security fix
a1146d2 
Revert 2e15b9f. The paged processed_bills read and the rule for two
status rows under one bill number belong to #1277 (fix/v3-audit-security)
only. The feed reads processed_bills without last_action_date again, and
the title and id-prefix stage join stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: test a bill that only its title joins to its stage
e811d54 
The id "sc-income-tax-cut" names no bill number, so only the title's
"H. 3492" finds the processed_bills row. The test fails when the title
candidate is removed from processedBillFor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: a bill held twice keeps the row with the later action
835e255 
When two status rows share one bill number (NY "S04487" from LegiScan
and "S 4487" from OpenStates), the row with the later last_action_date
now wins. On the same day, or with no dates, the row that links to
LegiScan wins. The bill id is only a fixed tie-break after that: the
OpenStates and Congress monitors hash their ids above 10,000,000, so
a higher id does not mean a later bill, and the OpenStates row always
won before. The test rows now carry realistic ids for each source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Report: a bill loses its run only when it scores the law before it
2224463 
billScoresOtherThanWriteUp flagged a bill when any paired provision held
no value near the write-up's "after". Correct bills lost "Load full
results": a filing-status change written with one status's value, a
yearly write-up of a monthly parameter, a step ("+1%") or a percent
stored as a whole number.

The check now needs evidence of a swap: the named provisions together
hold the write-up's "before" value and none of its "after" value. A
signed "after" next to an unsigned "before" is a step and is not checked.
A stored value also reads as a percent the other way (4.45 for "0.0445").
UT SB60 (0.045 for "4.5% → 4.45%") still flags.

changeNamesPath is exported, so the Policy changes table can share it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: the paged feed test checks each bill's baseline read
33a2768 
The 1,001-bill test now gives every reform_impacts row its baseline
params and checks that every bill's provisions carry that baseline.
Before, the baselines read could lose its row order and the test
still passed; now that read drops the 1,001st bill's baseline and the
test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Report: the Policy changes table reads change parameters with the sha…
b45f2d1 
…red helper

ReportChanges copied changeNamesPath line by line as namesPath. It now
imports the one in billProvisions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Report: test that a state bill's stored budget reads as state revenue
7f189d9 
The federal page test passes with the page's state prop removed, since
no state already gives the federal words. This test fails then: a Utah
bill must read "Annual state revenue loss" on Economic impacts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Validation: an empty fiscal-note link is no fiscal note
2d94303 
The feed maps a null fiscal_note_url to undefined but keeps "". With no
estimate, an empty link then gave an "Official fiscal note · No
estimate" row with no link and no source. The estimates table now
treats a blank link as absent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: the federal draft test checks the bill stays the draft's source
040e7ed 
The test asserted the draft's year, which draftYear already gave before
the fix. It now reads the year the draft runs in, and asserts what the
fix changed: the draft opens with source "bill" and no state region.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk @claude
Reforms: a bill whose stored parameters contradict its write-up opens…
78f54f9 
… no draft

The bill report stopped re-running UT SB60, but on the Reforms page
"Open as draft reform" still loaded its swapped 0.045 into Build, and a
run there scored the same +$85m increase. The bill panel now uses the
same check: "Run impact report" and "Open as draft reform" are disabled,
and a note says why.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk
PavelMakarchuk changed the base branch from fix/v3-relevant-comparisons to main October 15, 2026 13:40
@PavelMakarchuk
Merge branch 'tmp/fix/v3-audit-security' into tmp/bills-on-1277
ba9a650 
# Conflicts:
#	app/src/api/billFeed.ts
#	app/src/tests/unit/api/billFeed.test.ts
@PavelMakarchuk
Merge commit '78f54f96' into tmp/bills2
73fce16 
# Conflicts:
#	app/src/api/billFeed.ts
#	app/src/tests/unit/api/billFeed.test.ts
@PavelMakarchuk
PavelMakarchuk merged commit 2c0b895 into main Oct 15, 2026
12 checks passed
anth-volk added a commit that referenced this pull request Oct 15, 2026
@anth-volk @claude
Merge origin/main: keep v3 storage on the Public API, bring over the …
957783f 
…audit and report re-runs

Brought over from main: the v3 audit fixes (#1273, #1274, #1277, #1278,
#1280, #1281, #1283, #1284, #1285), run progress from the API, a report
that opens from its stored output and records a failed run, a wait for
the reform policy, the report year switcher and All years view (#1292),
dated edits in the adjust panel (#1294), each country's own draft, and
Your reforms' read-only reform detail with a parameters table, "View
results", "Re-run" and the delete prompt (#1288), with the other merged
fixes (#1289, #1290, #1293, #1295, #1296).

How the conflicts were resolved:

- Report page: a link is still a saved report's ID, read through the
  credential-free shared-report route, with the v1 report's number only
  without accounts. The v1 report it names (the shared row's
  legacy_report_id) gets main's run lifecycle: stored output, recorded
  failures, the household guard, the wait for the reform policy (for any
  link that resolved), the unknown-baseline state and the year switcher.
- Reforms page and ReformReports: main's read-only detail, parameters
  table, delete prompt, View results and Re-run, on this session's stores
  and session-scoped query keys. "View results", "Open" and "View full
  impact report" go to reportLink(report), never the v1 report's number.
  The detail keeps this branch's guards: a reform whose changes did not
  load cannot run and says so, a reform that changes nothing cannot run,
  and the run sends what Build would. A record without a year reads it
  from the API's record.
- Build's run bar: still "Save and run", which saves first and runs
  nothing when the save fails; it now saves only the values that differ
  from current law, leaves a reused reform (and its name) as it is, and
  names a budget window's years. A new reform has nothing to save when
  nothing differs.
- Adjust panel: main's dated edits, budget window and unknown-baseline
  guard on this session's reform store; recompute still waits until a
  value differs from current law.
- draftReform: loadProvisionsIntoDraft takes both the saved reform it
  edits and main's window and source.
- Tests: main's run-hook tests sit beside this branch's store tests; the
  run bar test that ran a report after a failed save is dropped, as a
  failed save runs nothing here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk anth-volk mentioned this pull request Oct 15, 2026
Merged

This branch was successfully deployed

2 active deployments
Preview – policyengine-calculator-next — 73fce16b Deployed Oct 15, 2026 by vercel[bot]
View deployment
Preview – policyengine-website — 73fce16b Deployed Oct 15, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@PavelMakarchuk