Flag the replay's weakly identified matches and reset-off misses in the paper

app/public/paper/index.html

@@ -76,7 +76,7 @@ <h1>Administrative quality control data as an oracle</h1>
       failures, and the fiscal 2025 rates landing where the sampling-noise
       mechanism says realized years should: 18 of 53 jurisdictions changed
       cost-share tiers, only 7 of those changes beyond two-year sampling
-      noise. This page embeds revision 10 (2026-10-03), which corrects how
+      noise. This page embeds revision 10 (2026-10-04), which corrects how
       revision 9 described the QC file's computed benefit and the Colorado
       error-case replay, adds Colorado's fiscal 2025 rate beside its fiscal
       2024 margin, and withdraws a claim of errata in the federal technical
@@ -86,18 +86,18 @@ <h1>Administrative quality control data as an oracle</h1>
       official-vs-file wedge decomposition, and the pre-registered
       model-repair experiments.
     </p>
-    <span class="paper-tag">Revision 10 · 2026-10-03 · Max Ghenis, PolicyEngine</span>
+    <span class="paper-tag">Revision 10 · 2026-10-04 · Max Ghenis, PolicyEngine</span>
   </header>
 
   <nav class="paper-actions" aria-label="Paper actions">
-    <a class="primary" href="web/index.html?v=r10-20261003">Open standalone HTML</a>
+    <a class="primary" href="web/index.html?v=r10-20261004">Open standalone HTML</a>
     <a href="web/index.pdf">Download PDF</a>
     <a href="../">Live simulator</a>
     <a href="https://github.com/PolicyEngine/snap-qc-sim">Code and artifacts</a>
   </nav>
 
   <div class="paper-frame-card">
-    <iframe src="web/index.html?v=r10-20261003"
+    <iframe src="web/index.html?v=r10-20261004"
             title="SNAP payment error working paper (embedded manuscript)"
             loading="lazy"
             sandbox="allow-same-origin allow-popups allow-popups-to-escape-sandbox"
@@ -106,7 +106,7 @@ <h1>Administrative quality control data as an oracle</h1>
 
   <p class="paper-foot">
     <a href="#paper-top">↑ Back to top</a> ·
-    <a href="web/index.html?v=r10-20261003">Open manuscript in a new page</a>
+    <a href="web/index.html?v=r10-20261004">Open manuscript in a new page</a>
   </p>
 </main>
 

app/public/paper/web/index.html

@@ -7,7 +7,7 @@
 <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes">
 
 <meta name="author" content="Max Ghenis">
-<meta name="dcterms.date" content="2026-10-03">
+<meta name="dcterms.date" content="2026-10-04">
 
 <title>Administrative quality control data as an oracle: verifying encoded SNAP rules and simulating payment error under cost sharing</title>
 <style>
@@ -138,10 +138,10 @@
 — implies.
 ">
 <meta name="citation_author" content="Max Ghenis">
-<meta name="citation_publication_date" content="2026-10-03">
-<meta name="citation_cover_date" content="2026-10-03">
+<meta name="citation_publication_date" content="2026-10-04">
+<meta name="citation_cover_date" content="2026-10-04">
 <meta name="citation_year" content="2026">
-<meta name="citation_online_date" content="2026-10-03">
+<meta name="citation_online_date" content="2026-10-04">
 <meta name="citation_language" content="en">
 <meta name="citation_reference" content="citation_title=Technical documentation for the fiscal year 2024 supplemental nutrition assistance program quality control database and the QC minimodel;,citation_author=Joshua Leftin;,citation_author=Mia Monkovic;,citation_author=Francisco Yang;,citation_author=Sam Bonelli;,citation_author=John Carlo Maula;,citation_publication_date=2026-08;,citation_cover_date=2026-08;,citation_year=2026;,citation_fulltext_html_url=https://snapqcdata.net/datafiles;,citation_technical_report_institution=Mathematica, for the U.S. Department of Agriculture;">
 <meta name="citation_reference" content="citation_title=Supplemental nutrition assistance program payment error rates, fiscal year 2024;,citation_author=U.S. Department of Agriculture, Food and Nutrition Service;,citation_publication_date=2025;,citation_cover_date=2025;,citation_year=2025;,citation_fulltext_html_url=https://www.fns.usda.gov/snap/qc/per;">
@@ -204,7 +204,7 @@ <h1 class="title">Administrative quality control data as an oracle: verifying en
                 <div>
             <div class="quarto-title-meta-heading">Published</div>
             <div class="quarto-title-meta-contents">
-              <p class="date">October 3, 2026</p>
+              <p class="date">October 4, 2026</p>
             </div>
           </div>
 
@@ -483,8 +483,8 @@ <h2 class="anchored" data-anchor-id="sec-decompose">Decomposing measured error:
 </div>
 </figure>
 </div>
-<p>The third layer adapts the public reconstruction solver of <span class="citation" data-cites="giannella_snapqc">Giannella and Molin (<a href="#ref-giannella_snapqc" role="doc-biblioref">2026</a>)</span> to fiscal 2024 and replays its output through the verified engine. The solver starts from the file’s edited inputs and moves only the input named by the case’s first finding (<code>ELEMENT1</code>). Household size moves once, by one person: one fewer for natures 12, 14 or 16 and one more for nature 7 (element 150). Earned or unearned income, rent, the utility allowance, and the medical, dependent care or child-support deduction move in $3 steps toward the issued benefit (<code>RAWBEN</code>) until a stop rule binds, and a stepped utility amount is then reset to a commonly reported value.<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a> When the first finding falls outside those element and nature codes, nothing moves. Of 283 deviation cases surviving the solver’s consistency filters (22 of 305 are excluded), 246 reproduce the issued amount within the file’s $5 editing tolerance, which is consistent with correct arithmetic applied to a wrong input. In 230 of the 246 the solver moved an input; in the other 16 it took no step, and the issued benefit was already within $5 of <code>FSBEN</code>, so the match restates the parity result. In 19 of the 230 the issued benefit was also within $5 of <code>FSBEN</code> before the solver moved, so those matches did not need the move. The rate differs by threshold status: among the 97 above-threshold official error cases 76 reproduce (78.4%, binomial standard error about 4 points), and among the 186 sub-threshold deviations 170 do (91.4%). The solver’s own recomputation and the engine agree on the within-$5 classification for all 283 cases, two implementations with distinct codebases reading the same file.</p>
-<p>Neither outcome identifies a computation error. A reproduced case shows that one value of the first-listed input, run through correct arithmetic, yields a benefit within $5 of the issued benefit; it does not show which input the agency had wrong, and a moved input can absorb a computation error. A non-reproduced case can reflect the solver’s limits: it tries one input in one direction, and in 17 of the 37 it moved nothing (14 cases whose first finding falls outside those codes, and 3 where it stopped before moving). Of the 26 cases that carry a layer-2 computational finding, 13 reproduce, 10 do not and 3 were not replayed. The complete per-case replay output (283 rows), the layer-1 and layer-2 classification outputs, the reconstruction scripts and <code>audit_claims.py</code>, which regenerates the replay counts into <code>claims_audit.json</code>, are committed with this paper.</p>
+<p>The third layer adapts the public reconstruction solver of <span class="citation" data-cites="giannella_snapqc">Giannella and Molin (<a href="#ref-giannella_snapqc" role="doc-biblioref">2026</a>)</span> to fiscal 2024 and replays its output through the verified engine. The solver starts from the file’s edited inputs and moves only the input named by the case’s first finding (<code>ELEMENT1</code>). Household size moves once, by one person: one fewer for natures 12, 14 or 16 and one more for nature 7 (element 150). Earned or unearned income, rent, the utility allowance, and the medical, dependent care or child-support deduction move in $3 steps toward the issued benefit (<code>RAWBEN</code>) until a stop rule binds, and the utility amount is then reset to a commonly reported value.<a href="#fn3" class="footnote-ref" id="fnref3" role="doc-noteref"><sup>3</sup></a> When the first finding falls outside those element and nature codes, nothing moves. Of 283 deviation cases surviving the solver’s consistency filters (22 of 305 are excluded), 246 reproduce the issued amount within the file’s $5 editing tolerance, which is consistent with correct arithmetic applied to a wrong input. In 230 of the 246 the solver moved an input; in the other 16 it took no step, and the issued benefit was already within $5 of <code>FSBEN</code>, so the match restates the parity result. In 19 of the 230 the issued benefit was also within $5 of <code>FSBEN</code> before the solver moved, so those matches did not need the move. The rate differs by threshold status: among the 97 above-threshold official error cases 76 reproduce (78.4%, binomial standard error about 4 points), and among the 186 sub-threshold deviations 170 do (91.4%). In 60 of the 230 moved matches, 31 of them above the threshold, the issued benefit lies within $5 of a stretch where the solver’s benefit formula is flat in the moved input (the maximum allotment in 53, the minimum benefit in 4 and rent past the shelter-deduction cap in 3). By that formula every amount of the input on the stretch reproduces the issued benefit, so the match bounds the input on one side at most and is weakly identified. In 34 of the 53 the issued benefit is the maximum allotment and the solver lowered an income; the solver pays the maximum whenever net income is zero or less, so every value of that income up to the point where net income reaches zero yields it. The solver’s own recomputation and the engine agree on the within-$5 classification for all 283 cases, two implementations with distinct codebases reading the same file.</p>
+<p>Neither outcome identifies a computation error. A reproduced case shows that one value of the first-listed input, run through correct arithmetic, yields a benefit within $5 of the issued benefit; it does not show which input the agency had wrong, and a moved input can absorb a computation error. A non-reproduced case can reflect the solver’s limits: it tries one input in one direction, and in 17 of the 37 it moved nothing (14 cases whose first finding falls outside those codes, and 3 where it stopped before moving). In 6 more, its $3 steps reached within $3 of the issued benefit before the utility reset moved the input off that match; in 9 a stop rule ended the steps short of it, and in 5 the one-person household-size move missed. Of the 26 cases that carry a layer-2 computational finding, 13 reproduce, 10 do not and 3 were not replayed. Of the 10, 2 are among the 6 the utility reset moved off a match. The complete per-case replay output (283 rows), the layer-1 and layer-2 classification outputs, the reconstruction scripts and <code>audit_claims.py</code>, which regenerates the replay counts into <code>claims_audit.json</code>, are committed with this paper.</p>
 <p>Layers 1 and 2 agree on the structure: most measured SNAP payment error among eligible cases is information failure, not calculation failure — consistent with the administrative-burden literature’s account of where compliance costs bind <span class="citation" data-cites="herd2018burden">(<a href="#ref-herd2018burden" role="doc-biblioref">Herd and Moynihan 2018</a>)</span>. Correct-computation guarantees address the smaller share directly — roughly 3 to 10% of error dollars by layers 1–2 — plus whatever share of information failures better tooling prevents indirectly through verification prompts and documentation-requirement computation at intake; <a href="#sec-adoption" class="quarto-xref">Section&nbsp;7.2</a> prices the direct shares through the formula’s tiers, elections, and delay clause. Ineligibility errors, excluded from the file, are predominantly facts-driven, which would push the information-failure share higher still; that inference, unlike the in-file shares, is not directly computed here. The indirect channel is where policy design enters: rules determine what a caseworker must collect and verify, which is the mechanism the modeling section takes up.</p>
 </section>
 <section id="sec-model" class="level2">
@@ -757,7 +757,7 @@ <h2 class="anchored" data-anchor-id="acknowledgments">Acknowledgments</h2>
 <ol>
 <li id="fn1"><p>7 U.S.C. § 2013(a)(2)(B)(iii). The test applies mechanically to each year’s rate and is independent of the state’s election. The fiscal 2025 rates published in June 2026 settle the first leg: seven jurisdictions — Alaska (23.15%), the District of Columbia, Delaware, Georgia, Illinois, New Mexico, and Oregon — crossed on fiscal 2025 and owe nothing in FY2028, with no bill before FY2029 — keyed to the fiscal 2026 rate unless that rate also crosses, which pushes the start to FY2030 and is nearly certain for Alaska. Fiscal 2024 rates would have qualified ten, including New York (14.09%); New York’s fiscal 2025 rate of 13.18% misses the 13.33% threshold, so its delay now rides on the fiscal 2026 measurement. Any projection of FY2028 outlays must model the delay, and the simulator does: delayed years enter expected bills as zero.<a href="#fnref1" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
 <li id="fn2"><p>The published thresholds — $48, $54, $56, $57, $58 for fiscal 2022–26 — are uniquely reproduced by flooring the indexed value, which puts the fiscal 2027 threshold at an estimated $59: $59.58 unrounded on the May 2026 Thrifty Food Plan cost of $1,018.20, with $60 requiring a June figure of at least $1,025.40. The estimate hardens when the June figure publishes.<a href="#fnref2" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
-<li id="fn3"><p>Income steps stop once the solver’s recomputed benefit passes <code>RAWBEN</code>; rent, utility and deduction steps also stop within $3 of it. Steps that lower an input stop at zero; rent and utility steps also stop when the shelter deduction reaches its cap (when raising) or zero (when lowering); income-raising steps stop when the uncapped benefit falls below zero; and all steps stop when the benefit reaches $0 or after 1,000 steps. Where <code>RAWBEN</code> is the maximum allotment, which the benefit cannot pass, income-lowering steps continue until the income reaches zero or the step limit. Each case whose first finding is the utility allowance and whose <code>RAWBEN</code> differs from <code>FSBEN</code> then has its utility amount reset among the amounts that more than five filtered cases of any review status in the same state and calendar year report: those above the file’s amount when <code>RAWBEN</code> exceeds <code>FSBEN</code>, those below it when <code>RAWBEN</code> is lower. The solver takes the candidate nearest its stepped amount, keeps the stepped amount if none lies above, and sets zero if none lies below; in 2 cases this moves an amount the steps left unchanged (<code>reconstruct_co_fy2024.R</code>).<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
+<li id="fn3"><p>Income steps stop once the solver’s recomputed benefit passes <code>RAWBEN</code>; rent, utility and deduction steps also stop within $3 of it. Steps that lower an input stop at zero; rent and utility steps also stop when the shelter deduction reaches its cap (when raising) or zero (when lowering); income-raising steps stop when the uncapped benefit falls below zero; and all steps stop when the benefit reaches $0 or after 1,000 steps. Where <code>RAWBEN</code> is at or above the maximum allotment, which the benefit cannot pass, income-lowering steps continue until the income reaches zero or the step limit; where it is at or below a one- or two-person unit’s minimum benefit, which the benefit cannot fall below, income-raising steps continue until the uncapped benefit falls below zero or the step limit. Each case whose first finding is the utility allowance and whose <code>RAWBEN</code> differs from <code>FSBEN</code> then has its utility amount reset among the amounts that more than five filtered cases of any review status in the same state and calendar year report: those above the file’s amount when <code>RAWBEN</code> exceeds <code>FSBEN</code>, those below it when <code>RAWBEN</code> is lower. The solver takes the candidate nearest its stepped amount, keeps the stepped amount if none lies above, and sets zero if none lies below; in 2 cases this moves an amount the steps left unchanged (<code>reconstruct_co_fy2024.R</code>).<a href="#fnref3" class="footnote-back" role="doc-backlink">↩︎</a></p></li>
 </ol>
 </section></div></main>
 <!-- /main column -->