Skip to content

Reforms: tracker bill stages stay the same on every load - #1277

Merged
PavelMakarchuk merged 6 commits into
mainfrom
fix/v3-audit-security
Oct 15, 2026
Merged

PavelMakarchuk merged 6 commits into
mainfrom
fix/v3-audit-security

Conversation

@PavelMakarchuk

Copy link
Copy Markdown
Contributor

Summary

The Reforms feed read the tracker's tables page by page with no row order. Postgres gives no stable order without ORDER BY, so the second page of processed_bills (1,418 rows) repeated rows from the first page. About 418 status rows were lost on each load, and a different set each time. Bills then showed "Analyzed" in place of their real stage, lost their bill text link, and the stage filter gave different bills on each reload.

Problems and fixes

  1. Paged tracker reads had no row order (app/src/api/billFeed.ts). trackerSelect now takes a required order argument and sends order= on every page. processed_bills reads by bill_id; research, reform_impacts (both reads, and the district read) and validation_metadata read by id. This also protects research and reform_impacts when they pass 1,000 rows.
  2. Two status rows under one bill number (for example NY "S 4487" and "S04487"). processedByKey kept whichever row it read last. The feed now sorts status rows by LegiScan bill id (the higher id is the later bill), with the row contents as a final tie-break. The later bill's row wins in any read order. A LegiScan link on the research entry still wins first, as before.

v2 impact

None. billFeed.ts serves only the v3 Reforms surface and bill reports. No v2 page or route changes.

Tests

  • New mockPagedTrackerFetch fixture: it serves pages as PostgREST does, and a read with no order repeats the first rows on every page, as the live tracker did.
  • given a status row past the first page then every load finds its stage and link: failed before (status "Analyzed", no link), passes now.
  • given every table past one page then each bill reads once, with its run, check and stage: 1,001 bills in each table; failed before (1,000 unique bills), passes now.
  • given two status rows under one bill number then the later bill wins in any read order: failed before (the reversed order gave "Introduced"), passes now.
  • Checks: bun run typecheck, bunx eslint and bunx prettier --check on the changed files, bunx vitest run on src/tests/unit/api, useTrackedBills and pages/flagship (493 passed), full bun run vitest (4,921 passed, 10 skipped).

Note: this PR is based on the open #1274 stack. Retarget it to main after #1273 and #1274 merge.

🤖 Generated with Claude Code

PavelMakarchuk and others added 2 commits October 14, 2026 18:15
Each paged tracker read now names a unique order column (bill_id for
processed_bills, id for the rest). Without one, Postgres gave page 2 of
processed_bills rows already on page 1, so about 418 of 1,418 status
rows were lost on each load, and a different set each time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tage

When two processed_bills rows share one normalized bill number (NY
"S 4487" and "S04487"), the feed kept whichever row it read last. It
now keeps the row with the higher LegiScan bill id, the later bill, so
the stage and the bill text link no longer depend on the read order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 15, 2026 1:42pm UTC
policyengine-website Ready Ready Preview Oct 15, 2026 1:42pm UTC

Request Review

PavelMakarchuk and others added 2 commits October 15, 2026 08:53
When two status rows share one bill number (NY "S04487" from LegiScan
and "S 4487" from OpenStates), the row with the later last_action_date
now wins. On the same day, or with no dates, the row that links to
LegiScan wins. The bill id is only a fixed tie-break after that: the
OpenStates and Congress monitors hash their ids above 10,000,000, so
a higher id does not mean a later bill, and the OpenStates row always
won before. The test rows now carry realistic ids for each source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 1,001-bill test now gives every reform_impacts row its baseline
params and checks that every bill's provisions carry that baseline.
Before, the baselines read could lose its row order and the test
still passed; now that read drops the 1,001st bill's baseline and the
test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PavelMakarchuk added a commit that referenced this pull request Oct 15, 2026
Revert 2e15b9f. The paged processed_bills read and the rule for two
status rows under one bill number belong to #1277 (fix/v3-audit-security)
only. The feed reads processed_bills without last_action_date again, and
the title and id-prefix stage join stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PavelMakarchuk
PavelMakarchuk merged commit b95a0bd into main Oct 15, 2026
12 checks passed
anth-volk added a commit that referenced this pull request Oct 15, 2026
…audit and report re-runs

Brought over from main: the v3 audit fixes (#1273, #1274, #1277, #1278,
#1280, #1281, #1283, #1284, #1285), run progress from the API, a report
that opens from its stored output and records a failed run, a wait for
the reform policy, the report year switcher and All years view (#1292),
dated edits in the adjust panel (#1294), each country's own draft, and
Your reforms' read-only reform detail with a parameters table, "View
results", "Re-run" and the delete prompt (#1288), with the other merged
fixes (#1289, #1290, #1293, #1295, #1296).

How the conflicts were resolved:

- Report page: a link is still a saved report's ID, read through the
  credential-free shared-report route, with the v1 report's number only
  without accounts. The v1 report it names (the shared row's
  legacy_report_id) gets main's run lifecycle: stored output, recorded
  failures, the household guard, the wait for the reform policy (for any
  link that resolved), the unknown-baseline state and the year switcher.
- Reforms page and ReformReports: main's read-only detail, parameters
  table, delete prompt, View results and Re-run, on this session's stores
  and session-scoped query keys. "View results", "Open" and "View full
  impact report" go to reportLink(report), never the v1 report's number.
  The detail keeps this branch's guards: a reform whose changes did not
  load cannot run and says so, a reform that changes nothing cannot run,
  and the run sends what Build would. A record without a year reads it
  from the API's record.
- Build's run bar: still "Save and run", which saves first and runs
  nothing when the save fails; it now saves only the values that differ
  from current law, leaves a reused reform (and its name) as it is, and
  names a budget window's years. A new reform has nothing to save when
  nothing differs.
- Adjust panel: main's dated edits, budget window and unknown-baseline
  guard on this session's reform store; recompute still waits until a
  value differs from current law.
- draftReform: loadProvisionsIntoDraft takes both the saved reform it
  edits and main's window and source.
- Tests: main's run-hook tests sit beside this branch's store tests; the
  run bar test that ran a report after a failed save is dropped, as a
  failed save runs nothing here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — 4e413aa9 Deployed Oct 15, 2026 by vercel[bot]
Preview – policyengine-calculator-next — 4e413aa9 Deployed Oct 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant