diff --git a/README.md b/README.md index 4e81c6b..caa12bc 100644 --- a/README.md +++ b/README.md @@ -28,8 +28,9 @@ noise alone materially affects tier assignment. ## What v0.1 finds (FY 2024 file, all 53 jurisdictions) - **Tier assignment is noisy.** Several states' tiers are near coin flips - under QC sampling variation (Colorado: official rate 9.97%, 0.03 points - from the 15% boundary, with a ±0.9-point sampling SD). + under QC sampling variation (Colorado: official FY 2024 rate 9.97%, 0.03 + points below the 10% rate where the 15% cost share begins, with a + ±0.9-point sampling SD; its FY 2025 rate, 10.09%, is above it). - **Simplification options carry large expected values** where they can move a state across a boundary: about $609M/yr in combined expected state cost-share reduction nationally at 50% category-suppression effectiveness diff --git a/analysis/ENGINE_COMPARISON.md b/analysis/ENGINE_COMPARISON.md index 94ce6b6..4f309b6 100644 --- a/analysis/ENGINE_COMPARISON.md +++ b/analysis/ENGINE_COMPARISON.md @@ -20,7 +20,7 @@ Stage concepts are pinned per state in the suite reports (Colorado exposes `snap ## Formula-benefit divergence catalog -`|RAWBEN - BENFIX|` equals the recorded error amount for every official-universe case in the seven states (asserted at build), and nationally for 99.997% of weighted FY2024 cases (n=44,800). The Minimodel's formula benefit taken as the deviation anchor — `|RAWBEN - FSBEN|` — matches the recorded error for only 83.64% weighted, because the formula recomputation conflates recorded allotment adjustments with error. The catalog partitions each state's `FSBEN != BENFIX` cases three ways: +`|RAWBEN - BENFIX|` equals the recorded error amount for every official-universe case in the seven states (asserted at build), and nationally for 99.997% of weighted FY2024 cases (n=44,800). The formula benefit (FSBEN) taken as the deviation anchor — `|RAWBEN - FSBEN|` — matches the recorded error for only 83.64% weighted, because the formula recomputation conflates recorded allotment adjustments with error. The catalog partitions each state's `FSBEN != BENFIX` cases three ways: 1. **Coded allotment adjustment** — `ALLADJ` records a prorated (2) or other (3) adjustment; the recorded allotment is not the full-month formula amount by design. 2. **Error-correction arithmetic** — no coded adjustment and `AMTERR > 0`: the reviewer's corrected allotment differs from the full recomputation (about two-thirds within $2, rounding-scale). diff --git a/analysis/engine_comparison.py b/analysis/engine_comparison.py index da2f391..141d34c 100644 --- a/analysis/engine_comparison.py +++ b/analysis/engine_comparison.py @@ -9,8 +9,8 @@ each state's FY2024 QC public-use reviews through the Axiom RuleSpec SNAP composition and compare six recorded stages — gross income, standard deduction, excess-shelter deduction, net income, maximum - allotment, and the benefit — at zero tolerance against the file's own - Minimodel-recomputed values (``FSBEN`` and companions). This module + allotment, and the benefit — at zero tolerance against the file's + computed values (``FSBEN`` and companions). This module re-derives the per-state parity counts from those reports and asserts the exactness the app displays (every compared case matches at every stage; excluded cases are enumerated program-structure classes). @@ -18,8 +18,8 @@ * **The formula-benefit divergence catalog** — the public file records three benefit anchors per case: ``RAWBEN`` (the allotment the state issued), ``BENFIX`` (the allotment corrected for the reviewer's - findings), and ``FSBEN`` (the Minimodel's full-formula recomputation - from the edited inputs). ``|RAWBEN - BENFIX|`` equals the recorded + findings), and ``FSBEN`` (the formula benefit Mathematica computes for + USDA from the edited inputs). ``|RAWBEN - BENFIX|`` equals the recorded error amount ``AMTERR`` for every official-universe case in the seven states; ``FSBEN`` differs from ``BENFIX`` for a catalogued minority, so a formula recomputation alone conflates recorded allotment @@ -495,8 +495,8 @@ def render_report(artifact: dict[str, Any]) -> str: "official-universe case in the seven states (asserted at " "build), and nationally for " f"{100 * national['concordance_weighted']['benfix']:.3f}% of " - f"weighted FY2024 cases (n={national['n']:,}). The Minimodel's " - "formula benefit taken as the deviation anchor — " + f"weighted FY2024 cases (n={national['n']:,}). The formula " + "benefit (FSBEN) taken as the deviation anchor — " "`|RAWBEN - FSBEN|` — matches the recorded error for only " f"{100 * national['concordance_weighted']['fsben']:.2f}% " "weighted, because the formula recomputation conflates " diff --git a/app/public/app.js b/app/public/app.js index 70006e5..b825cc6 100644 --- a/app/public/app.js +++ b/app/public/app.js @@ -7,7 +7,7 @@ const TIERS = [[6, 0], [8, 5], [10, 10], [Infinity, 15]]; const TIER_LABELS = { 0: "0% share", 5: "5% share", 10: "10% share", 15: "15% share" }; const TIER_VARS = { 0: "--tier-0", 5: "--tier-5", 10: "--tier-10", 15: "--tier-15" }; const DRAWS = 4000; -const ASSET_V = "20261003a"; // bump with index.html's app.js?v= on every deploy that changes any asset +const ASSET_V = "20261004a"; // bump with index.html's app.js?v= on every deploy that changes any asset const SCEN_SCHEMA = "snap_qc_sim.model_scenarios.v1"; const ENGINE_SCHEMA = "snap_qc_sim.engine_comparison.v1"; const INTERVENTIONS_SCHEMA = "snap_qc_sim.interventions.v1"; @@ -813,7 +813,7 @@ function renderFy2027(code) { // ---- Axiom rules-engine comparison mode ---------------------------------- // The engine mode surfaces precomputed verification artifacts: the committed // axiom-oracles suite reports (stage-by-stage parity of the Axiom RuleSpec -// computation against the QC file's Minimodel-recomputed benefit chain) and +// computation against the QC file's computed benefit chain) and // the formula-benefit divergence catalog built by analysis/engine_comparison.py. // It does not change the Monte Carlo simulation — engine runs are precomputed, // never recomputed in the browser. @@ -866,8 +866,8 @@ function enginePanelBody(code) { const conflates = `

What the formula benefit conflates with error

` + `

The file records three benefit anchors per case: the allotment the state issued (RAWBEN), ` + - `the allotment corrected for the reviewer's findings (BENFIX), and the Minimodel's full-formula ` + - `recomputation (FSBEN). |RAWBEN − BENFIX| equals the recorded error amount for every ` + + `the allotment corrected for the reviewer's findings (BENFIX), and the formula benefit Mathematica ` + + `computes for USDA from the edited record with the QC Minimodel's benefit formula (FSBEN). |RAWBEN − BENFIX| equals the recorded error amount for every ` + `official-universe case in the seven verified states — nationally ${pct(nat.concordance_weighted.benfix, 3)} of ` + `weighted FY 2024 cases. Taking the formula benefit as the error anchor instead — |RAWBEN − FSBEN| — matches ` + `the recorded error for only ${pct(nat.concordance_weighted.fsben, 1)} weighted: a formula recomputation alone ` + diff --git a/app/public/index.html b/app/public/index.html index f4720f9..eef4396 100644 --- a/app/public/index.html +++ b/app/public/index.html @@ -118,7 +118,7 @@

Targeted review scenarios

Benefit recomputation -
Shows how the open, certified Axiom RuleSpec computation compares against the QC file's FNA Minimodel benefit chain, stage by stage, with the formula-benefit divergence catalog. Precomputed artifacts; the simulation is unchanged.
+
Shows how the open, certified Axiom RuleSpec computation compares against the QC file's computed benefit chain, stage by stage, with the formula-benefit divergence catalog. Precomputed artifacts; the simulation is unchanged.
Uncertainty @@ -129,8 +129,8 @@

Targeted review scenarios

Sampling engine: resamples the state's observed FY 2024 QC error dollars, centered on the official rate.

@@ -236,6 +236,6 @@

All states at a glance

- + diff --git a/app/public/paper/index.html b/app/public/paper/index.html index 8fc0947..506aae1 100644 --- a/app/public/paper/index.html +++ b/app/public/paper/index.html @@ -76,23 +76,28 @@

Administrative quality control data as an oracle

failures, and the fiscal 2025 rates landing where the sampling-noise mechanism says realized years should: 18 of 53 jurisdictions changed cost-share tiers, only 7 of those changes beyond two-year sampling - noise. This page embeds the revision-7 manuscript (updated 2026-08-13), which - carries five rounds of adversarial review, archived unedited, and adds - the engine-adoption accounting bounds, the official-vs-file wedge - decomposition, and the pre-registered model-repair experiments. + noise. This page embeds revision 10 (2026-10-03), which corrects how + revision 9 described the QC file's computed benefit and the Colorado + error-case replay, adds Colorado's fiscal 2025 rate beside its fiscal + 2024 margin, and withdraws a claim of errata in the federal technical + documentation that no source records; the fact catalog lists each + change. The manuscript carries six rounds of adversarial review, + archived unedited, and adds the engine-adoption accounting bounds, the + official-vs-file wedge decomposition, and the pre-registered + model-repair experiments.

- Revision 9 · 2026-08-16 · Max Ghenis, PolicyEngine + Revision 10 · 2026-10-03 · Max Ghenis, PolicyEngine
-