From 8714770979394463b193caddb366de8d18f54dc8 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Mon, 5 Oct 2026 10:48:35 -0400 Subject: [PATCH 1/3] Audit optional employer pass-through in the payroll tax references; draft both records Four scored payroll_tax references (scenario_032 MN, 043 CO, 081 MA, 082 NY) count an employee share of a state paid-leave premium that the law lets the employer deduct but does not require. policyengine-us 2.15.17 counts the largest share the employer may deduct. The output asks for "mandatory employee state payroll taxes". reference_audit/2026-10-05-payroll classifies every state program in the payroll references from primary law (two adversarial reviews per program, none refuted), sweeps all 1,984 outputs with an output-scope adapter that drops the optional shares (four outputs move, nothing else), and drafts both candidate records: proposed_exclusions.json and proposed_regenerations.json. It scores each through the analyze CLI; the unchanged copy reproduces the published payload. Changes no reference or published number. Draft README; review pending. Co-Authored-By: Claude Opus 5.5 --- reference_audit/2026-10-05-payroll/README.md | 171 + .../fixes/payroll_mandatory_scope.py | 78 + .../program_classification.json | 346 ++ .../proposed_exclusions.json | 61 + .../proposed_regenerations.json | 86 + .../scripts/decompose_payroll.py | 232 + .../scripts/leaderboard_impact.py | 401 ++ .../scripts/model_answers.py | 135 + .../scripts/propose_changes.py | 246 + .../scripts/sweep_payroll_scope.py | 269 + .../verification/explanations.json | 3462 ++++++++++++ .../verification/law_research.json | 4794 +++++++++++++++++ .../verification/leaderboard_impact.json | 1054 ++++ .../leaderboard_impact_exclude_models.csv | 47 + ...rboard_impact_exclude_plus_salt_models.csv | 47 + ...oard_impact_exclude_plus_salt_programs.csv | 2 + .../leaderboard_impact_exclude_programs.csv | 2 + .../leaderboard_impact_regenerate_models.csv | 47 + ...ard_impact_regenerate_plus_salt_models.csv | 47 + ...d_impact_regenerate_plus_salt_programs.csv | 2 + ...leaderboard_impact_regenerate_programs.csv | 2 + .../leaderboard_impact_salt_models.csv | 47 + .../leaderboard_impact_salt_programs.csv | 2 + .../verification/model_answers.csv | 553 ++ .../verification/model_answers_summary.json | 280 + .../verification/payroll_decomposition.csv | 101 + .../verification/payroll_decomposition.json | 1985 +++++++ .../verification/sweep_payroll_scope.csv | 1985 +++++++ .../sweep_payroll_scope_households.json | 506 ++ 29 files changed, 16990 insertions(+) create mode 100644 reference_audit/2026-10-05-payroll/README.md create mode 100644 reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py create mode 100644 reference_audit/2026-10-05-payroll/program_classification.json create mode 100644 reference_audit/2026-10-05-payroll/proposed_exclusions.json create mode 100644 reference_audit/2026-10-05-payroll/proposed_regenerations.json create mode 100644 reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py create mode 100644 reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py create mode 100644 reference_audit/2026-10-05-payroll/scripts/model_answers.py create mode 100644 reference_audit/2026-10-05-payroll/scripts/propose_changes.py create mode 100644 reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py create mode 100644 reference_audit/2026-10-05-payroll/verification/explanations.json create mode 100644 reference_audit/2026-10-05-payroll/verification/law_research.json create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact.json create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_models.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_models.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_programs.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_programs.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_models.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_models.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_programs.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_programs.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_models.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_programs.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/model_answers.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/model_answers_summary.json create mode 100644 reference_audit/2026-10-05-payroll/verification/payroll_decomposition.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/payroll_decomposition.json create mode 100644 reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope.csv create mode 100644 reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json diff --git a/reference_audit/2026-10-05-payroll/README.md b/reference_audit/2026-10-05-payroll/README.md new file mode 100644 index 00000000..f0318a1b --- /dev/null +++ b/reference_audit/2026-10-05-payroll/README.md @@ -0,0 +1,171 @@ +# Optional employer pass-through in the payroll tax references, October 5, 2026 + +This directory audits the state part of PolicyBench's US `payroll_tax` references in release `dashboard-data-20260930`. Four scored references count an employee share of a state paid-leave premium that the law lets the employer deduct but does not require. The directory proposes two alternative records for those four outputs, excluding them or regenerating them, and changes no reference. Either one changes published scores, so they wait for Max's ruling (cos decision DECISION_ID). + +## Recommendation + +Exclude the four outputs (scenario_032 MN, scenario_043 CO, scenario_081 MA, scenario_082 NY) from scoring in the next release, with `proposed_exclusions.json`. In the next prompt version, state how optional employee shares count, so these households can be scored again on a fresh run. + +- **Keep fails.** Under each state's law the premium is the employer's, and the employee pays only what the employer chooses to deduct, from zero up to a cap (16 of 16 independent reviews agree). The frozen reference assumes the cap. policyengine-us documents that assumption for Minnesota and Massachusetts ("assuming the employer withholds the maximum permitted employee share"), and the prompt never states the employer's choice. The reference therefore rests on a fact the prompt does not give. +- **Regenerate picks a side of a real split.** The prompt asks for "mandatory employee state payroll taxes" and says to exclude employer payroll taxes. On that wording the frozen value is too high. But the same laws speak of the employee share in mandatory terms: Minnesota's statute says employees "must pay the remaining portion, if any", New York's says every employee "shall contribute", and Massachusetts tells employers they "must withhold PFML contributions". 22 of 46 models included the Massachusetts and New York contributions. That is the case the exclusion rule covers: a careful reader could take the stated facts the other way. +- **Regenerating would mostly reward answers that never engaged with the program.** Across the four outputs it would newly credit 82 answers. One gives the optional-deduction reason; 36 say the state has no employee payroll tax and 39 never mention a state program, the same pattern as on Pennsylvania's mandatory contribution, where 17 and 16 models say the state has none. It would take credit from 35 answers that include the program at exactly the engine's amount. +- **Exclusion is the symmetric remedy and moves the board least.** GPT-6 Sol stays first (95.00% to 95.09%); five adjacent pairs swap. Regeneration moves GPT-6 Sol to 94.63% and puts Claude Opus 5.5 first on the within-1% metric. + +## How the reference is built + +These files hold for policyengine-us 2.15.17, as installed in the reference venv, plus `latest_final`, the system that built the published references. + +- `payroll_tax` is `spm_unit_payroll_tax` (`policybench/benchmark_specs.json`), which sums `employee_payroll_tax` over the SPM unit's tax units (`variables/household/expense/tax/spm_unit_payroll_tax.py`). +- `employee_payroll_tax` adds employee Social Security tax, employee Medicare tax, Additional Medicare Tax and `employee_state_payroll_tax` (`variables/gov/irs/tax/payroll/employee_payroll_tax.py`). +- `employee_state_payroll_tax` adds one `_employee_state_payroll_tax` per state for 14 states. Its documentation reads "Employee-side mandatory state payroll taxes and payroll-funded contributions" (`variables/gov/states/tax/payroll/employee_state_payroll_tax.py`). +- Each state aggregate adds its programs' employee contributions. For Minnesota, Massachusetts, Delaware, Maine and Vermont, the contribution's documentation says it assumes "the employer withholds the maximum permitted employee share". Colorado's, New York's and Washington's say no such thing, though each computes the largest permitted share. +- Nothing else in the engine reads these contributions except through `employee_payroll_tax`, which also feeds `spm_unit_paycheck_withholdings` and `household_tax_before_refundable_credits`. No PolicyBench output other than `payroll_tax` moves when they change (the sweep below). + +## Every payroll reference with a state component + +`scripts/decompose_payroll.py` recomputes all 100 payroll references, reproduces every one, and splits each into its federal and state parts. Twelve carry a state component; eleven are scored. + +| Output | State program | State part | Federal part | Reference | Employee share | Status | +|---|---|---:|---:|---:|---|---| +| scenario_005 | CA SDI | 5,590.00 | 30,454.00 | 36,044.00 | mandatory | scored | +| scenario_022 | CA SDI | 684.26 | 4,026.58 | 4,710.83 | mandatory | scored | +| scenario_023 | CA SDI | 226.75 | 1,334.36 | 1,561.12 | mandatory | scored | +| scenario_099 | CA SDI | 2,145.00 | 12,622.50 | 14,767.50 | mandatory | scored | +| scenario_028 | PA UC employee contribution | 42.00 | 4,590.00 | 4,632.00 | mandatory | scored | +| scenario_123 | PA UC employee contribution | 101.50 | 11,092.50 | 11,194.00 | mandatory | scored | +| scenario_120 | CT Paid Leave | 827.98 | 12,668.17 | 13,496.16 | mandatory | scored | +| scenario_008 | NJ TDI 50.92 + FLI 61.64 | 112.56 | 2,050.20 | 2,162.76 | mandatory | excluded (r05, NJ worker UI omitted) | +| scenario_032 | MN Paid Leave | 127.60 | 2,218.50 | 2,346.10 | **optional** | scored; proposed | +| scenario_043 | CO FAMLI | 17.99 | 312.74 | 330.73 | **optional** | scored; proposed | +| scenario_081 | MA PFML | 805.01 | 13,387.65 | 14,192.66 | **optional** | scored; proposed | +| scenario_082 | NY PFL 411.91 + DBL 31.20 | 443.11 | 7,664.92 | 8,108.03 | **optional** | scored; proposed | + +Every household with wages in a state with a modeled program carries that program; no benchmark household lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington. The four proposed references have had these values since the first run (2026-06-15). + +## The law, program by program + +The test is who owes the contribution. A **mandatory** employee contribution is one the statute puts on the worker and requires the employer to withhold. An **optional employer pass-through** is a premium the statute puts on the employer, which may deduct up to a share from wages and may instead pay it, so the employee amount runs from zero to a cap at the employer's choice. `program_classification.json` has each program's citation, 2026 rate and engine amounts; `verification/law_research.json` has every record, quote and URL. + +| Program | Who owes it | Employee share | Class | Primary source | +|---|---|---|---|---| +| MN Paid Leave | Premiums "accrue and become payable by each employer"; employer pays at least 50% | Employees pay "the remaining portion, if any, of the premium not paid by the employer": 0 to 0.44% | optional | [Minn. Stat. 268B.14](https://www.revisor.mn.gov/statutes/cite/268B.14) subds. 1, 3, 5a | +| CO FAMLI | Employer "shall remit" premiums | Employer "may deduct up to 50 percent" (0.44%); one that deducts less "is considered to have elected to pay" it | optional | [C.R.S. 8-13.3-507](https://olls.info/crs/crs2025-title-08.pdf)(2), (5); 7 CCR 1107-1 s. 1.4(6)(A) | +| MA PFML | Employer "shall remit" contributions | Employer may deduct up to 100% of family and 40% of medical (0.46%) and may deduct less | optional | [M.G.L. c. 175M, s. 6](https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6)(a), (c); 458 CMR 2.05(5)(c) | +| NY PFL and DBL | Covered employer provides the coverage and pays its cost beyond what it collects | Employer "is authorized to collect" up to the caps (PFL 0.432%, at most $411.91; DBL $0.60 a week) | optional | [N.Y. WCL s. 209](https://www.nysenate.gov/legislation/laws/WKC/209)(3)-(4), s. 210(1) | +| CT Paid Leave | Each employee "shall contribute" | Employer "shall deduct and withhold" 0.5%; no employer share | mandatory | [Conn. Gen. Stat. 31-49g](https://www.cga.ct.gov/2026/sup/chap_557.htm)(b)(1), (3) | +| CA SDI | Each worker "shall pay worker contributions" | Employer must withhold 1.3% in trust; one that pays it instead still owes the worker's contribution, and the payment counts as wages | mandatory | [Cal. UIC s. 984](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.)(a)(1), s. 986(a) | +| PA UC | Each employee "shall pay contributions" | Employer "shall withhold, in trust" 0.07% of all wages | mandatory | [UC Law s. 301.4](https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3)(a)-(b) | +| NJ TDI and FLI | Each worker "shall contribute" | Employer "shall ... withhold in trust" (TDI 0.19%, FLI 0.23%) | mandatory | [N.J.S.A. 43:21-7](https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf)(d)(1)(E), (G) | + +For the record of an upstream fix, the same test classifies the modeled programs no benchmark household reaches; these were researched once and not adversarially reviewed. Delaware Paid Leave, Maine PFML and the Vermont Child Care Contribution are optional pass-throughs; Washington Cares, Paid Leave Oregon, the Oregon statewide transit tax and Rhode Island TDI are mandatory; Washington PFML is mixed (the employer "may deduct" the employee share and "may elect to pay" it). + +Every 2026 rate, share and cap above was published before the 2026-07-03 reference freeze, and each engine amount equals the law's largest employee share to the cent. Points that cut the other way, all recorded in `verification/law_research.json`: + +- **Minnesota.** Subd. 3 says employees "must pay the remaining portion, if any", and subd. 5a(b)(2) that the employer "must make wage deductions as necessary". The remainder is whatever the employer leaves above its 50% floor, and the Department of Employment and Economic Development tells employers they "can deduct up to 0.44%" and "may choose to pay more than their required portion". +- **New York.** WCL s. 209(1) says every employee "shall contribute ... to the extent and in the manner herein provided"; s. 209(4) then only authorizes collection, and the Workers' Compensation Board says an employer "is allowed, but not required" to collect. +- **Massachusetts.** The Department of Family and Medical Leave's overview says employers "must withhold PFML contributions from employees' paychecks"; the same page says they "may choose to cover a larger share", and 458 CMR 2.05(5)(c) covers an employer that "opts to deduct a lower percentage". Acts 2026 c. 101 (approved 2026-06-12, before the freeze) amends the deduction caps in s. 6. The two reviewers read its scope differently, and its s. 45 applies it to taxable years beginning on or after 2026-01-01, while the department, after the freeze, dates the change to 2027. If it reached 2026, the most a large employer could deduct for scenario_081 would be $1,351.02 (0.772%) rather than $805.01. The share stays optional either way, so this matters only if the reference is kept. +- **California.** An employer may pay a worker's SDI instead of deducting it (s. 986(a)(2)), but the contribution remains the worker's and the payment counts as wages, the same structure as an employer paying the employee's FICA. + +## What the models answered + +`scripts/model_answers.py` scores every answer on the twelve outputs with `policybench.analysis.row_hit_scores`, the scorer the board uses (exact within $1), against the frozen reference and against the reference without the optional shares. Its frozen-reference scores equal the payload's own `exact` field on every row. + +| Output | Frozen | Without optional share | Exact now | Exact if regenerated | Most common answers | +|---|---:|---:|---:|---:|---| +| scenario_032 MN | 2,346.10 | 2,218.50 | 6 | 33 | 2,218.50 (29), 2,346.10 (6), 2,219 (4) | +| scenario_043 CO | 330.73 | 312.74 | 16 | 26 | 312.73 (18), 331.13 (9), 330.72 (7) | +| scenario_081 MA | 14,192.66 | 13,387.65 | 9 | 9 | 13,387.65 (7), 14,192.66 (6), 14,193 (3) | +| scenario_082 NY | 8,108.03 | 7,664.92 | 4 | 14 | 7,664.92 (8), 7,665 (6), 8,108.03 (3) | + +Of 46 models, the explanations on each output (`verification/explanations.json`, one reviewer per group of outputs, every model classified): + +| Output | Include the state program | Leave it out as optional or employer-paid | Say the state has no employee payroll tax | Never mention it | Other or unparsed | +|---|---:|---:|---:|---:|---:| +| scenario_032 MN | 7 | 1 | 19 | 16 | 3 | +| scenario_043 CO | 18 | 0 | 14 | 11 | 3 | +| scenario_081 MA | 22 | 1 | 7 | 9 | 7 | +| scenario_082 NY | 22 | 0 | 6 | 12 | 6 | + +Where the law requires the employee contribution, the explanations look the same (mandatory programs, all scored): + +| Output | Include the state program | Leave it out as optional or employer-paid | Say the state has no employee payroll tax | Never mention it | Other or unparsed | +|---|---:|---:|---:|---:|---:| +| scenario_005 CA SDI | 31 | 0 | 2 | 8 | 5 | +| scenario_022 CA SDI | 29 | 2 | 4 | 9 | 2 | +| scenario_023 CA SDI | 30 | 0 | 5 | 7 | 4 | +| scenario_099 CA SDI | 29 | 0 | 2 | 10 | 5 | +| scenario_028 PA UC | 12 | 0 | 17 | 15 | 2 | +| scenario_123 PA UC | 15 | 0 | 16 | 11 | 4 | +| scenario_120 CT Paid Leave | 20 | 0 | 11 | 9 | 6 | + +So a federal-only answer usually means the model did not know or did not apply the program: on Pennsylvania's mandatory contribution, 17 and 16 models say the state has no employee payroll tax. Across the four optional outputs, a regenerated reference would newly credit 82 answers. One of them gives the optional-deduction reason (Grok 4.7 on scenario_032: "paid-leave withholding is optional"); 36 say the state has no employee payroll tax, 39 never mention a state program and 6 are other or unparsed. It would take credit from 35 answers that include the program at exactly the engine's amount. + +Model by model, the readings split by family. Claude Opus 5.5, Claude Fable 5.1, Ox Alpha and Gemini 3.8 Flash answer federal tax alone on all four. GPT-6.1 Sol answers the frozen reference on all four. GPT-6 Sol answers federal tax alone on Minnesota and the frozen reference on Colorado, Massachusetts and New York; it also matches all seven mandatory references. Claude Fable 5.1 and Ox Alpha leave out every state program, mandatory ones included. + +## Board impact + +`scripts/leaderboard_impact.py` copies the frozen run to scratch directories and scores it with `python -m policybench.cli analyze`, the command the freeze runs. The unchanged copy reproduces the published payload's modelStats (except the cost and latency fields the freeze overlays), programStats, heatmap, globalWeights and failureModes exactly. Household-impact-weighted exact-match rate, the headline: + +| Published rank | Model | Published | Exclude (rank) | Regenerate (rank) | Payroll exact: published, exclude, regenerate | +|---:|---|---:|---:|---:|---| +| 1 | GPT-6 Sol | 95.00 | 95.09 (1) | 94.63 (1) | 98, 95, 96 | +| 2 | Claude Opus 5.5 | 93.70 | 94.32 (2) | 94.39 (2) | 92, 92, 96 | +| 3 | GPT-5.6 Sol | 93.57 | 93.70 (3) | 93.27 (3) | 94, 91, 92 | +| 4 | GPT-6 Luna | 92.10 | 92.34 (5) | 92.07 (5) | 91, 89, 91 | +| 5 | Claude Sonnet 5.5 | 92.08 | 92.52 (4) | 92.44 (4) | 92, 91, 94 | +| 6 | GPT-6 Astra | 91.67 | 91.80 (6) | 91.18 (7) | 86, 83, 83 | +| 7 | Claude Fable 5.1 | 90.83 | 91.38 (7) | 91.52 (6) | 88, 88, 92 | +| 8 | GPT-6.1 Sol | 90.60 | 90.53 (9) | 89.91 (9) | 86, 82, 82 | +| 9 | Kimi K3 | 90.43 | 90.54 (8) | 89.92 (8) | 92, 89, 89 | +| 10 | GPT-5.6 Luna | 89.31 | 89.55 (10) | 89.13 (11) | 89, 87, 88 | +| 11 | Ox Alpha | 88.49 | 89.09 (11) | 89.18 (10) | 87, 87, 91 | +| 12 | Grok 4.7 | 88.34 | 88.61 (13) | 88.35 (13) | 92, 90, 92 | +| 13 | Inkling | 88.22 | 88.65 (12) | 88.37 (12) | 81, 80, 82 | + +Payroll exact counts are out of 99 scored payroll outputs (95 under exclusion). + +**Exclude** (1,928 to 1,924 scored outputs per model): + +- Every model's headline exact rate moves by -0.07 to +0.62 points. GPT-6 Sol: 95.00% to 95.09%, still first. +- Five adjacent pairs swap: Claude Sonnet 5.5 and GPT-6 Luna (#4/#5), Kimi K3 and GPT-6.1 Sol (#8/#9), Inkling and Grok 4.7 (#12/#13), Gemini 3 Flash Preview and Claude Opus 4.7 (#29/#30), and Gemini 3.1 Flash Lite Preview and DeepSeek V4 Pro (#35/#36). +- The always-zero baseline moves from 69.98% to 70.53%, so GPT-6 Sol's lead over it goes from 25.02 to 24.57 points. +- Payroll exact match across models rises from 78.5% to 81.0%. + +**Regenerate** (1,928 scored outputs per model): + +- Headline exact rates move by -0.69 to +0.69 points. GPT-6 Sol: 95.00% to 94.63%, still first; Claude Opus 5.5: 93.70% to 94.39%, so the gap closes from 1.31 to 0.24 points. +- Ten adjacent pairs swap on the headline, among them Claude Sonnet 5.5 and GPT-6 Luna (#4/#5) and Claude Fable 5.1 and GPT-6 Astra (#6/#7). On the within-1% metric Claude Opus 5.5 passes GPT-6 Sol for first. +- The always-zero baseline is unchanged at 69.98%; GPT-6 Sol's lead over it goes from 25.02 to 24.65 points. + +**With the state income tax withholding proposal.** PolicyEngine/policybench#191 (cos d963) proposes three federal exclusions and may land in the same release. Its records alone reproduce #191's figures here (GPT-6 Sol 95.62%; Claude Sonnet 5.5 passes GPT-6 Luna). On top of them, exclusion takes GPT-6 Sol to 95.75% and Claude Opus 5.5 to 94.97% with three adjacent swaps; regeneration takes them to 95.19% and 95.00%. `verification/leaderboard_impact.json` has every figure. + +## Method + +Run from this checkout. `` is `results/local/adds202609/triage` in the main checkout, which holds the policyengine-us 2.15.17 venv; `` is the repo's `.venv`. + +1. **Decompose.** `scripts/decompose_payroll.py` recomputes every payroll reference on policyengine-us 2.15.17 with `latest_final` (`../2026-09-28/fixes/`) and splits it into federal and per-program state parts (`verification/payroll_decomposition.csv` and `.json`). All 100 reproduce the published reference, and the parts add to the total. +2. **Classify.** One researcher per program read the statute, rules, agency pages and engine source; two reviewers per program, one on the statute and one on the rules and agency guidance, then tried to refute each classification, re-fetching the sources and re-running the engine amounts. None refuted. `program_classification.json` is the result; `verification/law_research.json` holds every record and verdict. +3. **Sweep.** `scripts/sweep_payroll_scope.py` recomputes all 1,984 outputs with and without `fixes/payroll_mandatory_scope.py`, an output-scope adapter in the form of the Maryland one (`../2026-09-28/fixes/latest_md_local_output_scope.py`). It drops the five optional contributions from the list behind `employee_state_payroll_tax` and changes no formula or rate. The baseline reproduces all 1,928 scored references; the 19 excluded outputs that differ keep the values they were decided on, each listed in the sidecar's `excluded_outputs_rechecked`. The adapter moves four outputs, the four payroll references above, and no other output by even a cent (`verification/sweep_payroll_scope.csv` and `.log`). `CountryTaxBenefitSystem` applies a reform twice to the same system, so the adapter is idempotent. +4. **Propose.** `scripts/propose_changes.py` writes both alternatives in the format a release installs: `proposed_exclusions.json` (four `reference_depends_on_unlisted_input` records) and `proposed_regenerations.json` (four regenerated values with the adapter's sha256). It refuses a sweep that moves any output the proposal does not cover. +5. **Answers.** `scripts/model_answers.py` writes `verification/model_answers.csv` and `model_answers_summary.json`. +6. **Impact.** `scripts/leaderboard_impact.py --with-salt` writes `verification/leaderboard_impact*.csv`, `leaderboard_impact.json` and `.log`. It reads #191's records from its commit `8af912a0`. + +``` +PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH=$PWD /.venv-pe21517/bin/python reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py --out-dir reference_audit/2026-10-05-payroll/verification +PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH=$PWD /.venv-pe21517/bin/python reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py --out-dir reference_audit/2026-10-05-payroll/verification +PYTHONPATH=$PWD /bin/python reference_audit/2026-10-05-payroll/scripts/propose_changes.py +PYTHONPATH=$PWD /bin/python reference_audit/2026-10-05-payroll/scripts/model_answers.py +PYTHONPATH=$PWD /bin/python reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py --scratch --with-salt +``` + +INDEPENDENT_REVIEW + +## What a release adopting either option must also do + +RELEASE_CHECKLIST + +## Related findings + +- **Published annotations.** The case notes for these outputs treat the frozen reference as settled. scenario_081's calls the Massachusetts contribution "the mandatory Massachusetts Paid Family and Medical Leave (PFML) employee contribution", and scenario_032's and scenario_043's count the optional-deduction reasoning as a model error. Both options need them reworded. +- **Engine.** policyengine-us has no input for the employer's deduction choice, employer size (except Massachusetts' employer side) or a private plan. An upstream change could add one and default it to the largest share, so users can model either reading. The Connecticut rate parameter cites 31-49e, where 31-49g(b)(1) holds the rate; California's SDI rate parameter carries 1.3% back to 0000-01-01, wrong for years before 2026 (1.2% in 2025). Neither moves a benchmark number. +- **SALT.** policyengine-us 2.15.17 leaves state payroll contributions out of the federal SALT deduction, the defect PolicyEngine/policybench#191 records. diff --git a/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py b/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py new file mode 100644 index 00000000..f79881f5 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py @@ -0,0 +1,78 @@ +"""Candidate output-definition adapter: keep optional employer pass-through out of payroll_tax. + +PolicyBench defines payroll_tax as "annual household employee-side payroll tax: employee +Social Security tax, employee Medicare tax, Additional Medicare Tax, and mandatory +employee state payroll taxes. Exclude employer payroll taxes, ..." +(policybench/benchmark_specs.json, output prompt). + +policyengine-us 2.15.17 builds the output from ``employee_state_payroll_tax`` +(variables/gov/states/tax/payroll/employee_state_payroll_tax.py), whose documentation +reads "Employee-side mandatory state payroll taxes and payroll-funded contributions". +It adds one ``_employee_state_payroll_tax`` aggregate per state, and those add each +program's employee contribution. For programs whose premium the law places on the +employer, with the employer allowed but not required to deduct a share from wages, the +engine counts the largest share the employer may deduct. Its Minnesota variable says so: +"assuming the employer withholds the maximum permitted employee share" +(variables/gov/states/mn/tax/payroll/paid_leave/mn_employee_paid_leave_contribution.py). + +This module removes those programs (``OPTIONAL``, classified from primary law in +``../program_classification.json``) from the list behind ``employee_state_payroll_tax``, +so the scored output again matches its definition. Only the list changes; no formula, +rate or amount changes, and every mandatory employee contribution stays in. It mirrors +the 2026-09-28 Maryland output-scope adapter +(reference_audit/2026-09-28/fixes/latest_md_local_output_scope.py). +""" + +from policyengine_core.reforms import Reform +from policyengine_core.variables import Variable + +FIX_ID = "payroll_mandatory_scope" +DESCRIPTION = ( + "Exclude employee shares the employer may but need not deduct from the payroll " + "tax output (output definition)" +) +AGGREGATE = "employee_state_payroll_tax" +# Employee shares the employer may, but need not, deduct from wages +# (program_classification.json, classification optional_employer_pass_through). +OPTIONAL = ( + "co_employee_famli_contribution", + "ma_employee_paid_leave_contribution", + "mn_employee_paid_leave_contribution", + "ny_employee_disability_benefits_contribution", + "ny_employee_paid_family_leave_contribution", +) + + +def leaves(system) -> list[str]: + """Every program contribution under the engine's state aggregates, in order. + + CountryTaxBenefitSystem applies a reform twice to the same system, so on the second + pass the aggregate already lists programs; those are kept as they are. + """ + out = [] + for name in system.variables[AGGREGATE].adds: + if name.endswith("_" + AGGREGATE): + out.extend(system.variables[name].adds) + else: + out.append(name) + return out + + +class reform(Reform): + def apply(self): + listed = self.variables[AGGREGATE].adds + if not any(name.endswith("_" + AGGREGATE) for name in listed): + # Second pass: the list already holds the kept programs. + assert not set(OPTIONAL) & set(listed) + return + current = leaves(self) + unknown = sorted(set(OPTIONAL) - set(current)) + if unknown: + raise ValueError(f"not under {AGGREGATE}: {unknown}") + kept = [name for name in current if name not in OPTIONAL] + + class employee_state_payroll_tax(Variable): + adds = kept + + self.update_variable(employee_state_payroll_tax) + assert list(self.variables[AGGREGATE].adds) == kept diff --git a/reference_audit/2026-10-05-payroll/program_classification.json b/reference_audit/2026-10-05-payroll/program_classification.json new file mode 100644 index 00000000..85b494bd --- /dev/null +++ b/reference_audit/2026-10-05-payroll/program_classification.json @@ -0,0 +1,346 @@ +{ + "schema_version": 1, + "date": "2026-10-05", + "definition": "payroll_tax: annual household employee-side payroll tax: employee Social Security tax, employee Medicare tax, Additional Medicare Tax, and mandatory employee state payroll taxes. Exclude employer payroll taxes, FUTA, employer unemployment-insurance taxes, and self-employment tax (policybench/benchmark_specs.json)", + "engine": "policyengine-us 2.15.17: employee_state_payroll_tax adds one _employee_state_payroll_tax per state, and each adds the program contributions listed under engine_variables", + "test": "Who owes the contribution under the statute. mandatory_employee_withholding: the statute puts the contribution on the worker and requires the employer to withhold it. optional_employer_pass_through: the statute makes the premium the employer's, and lets the employer deduct up to a share from wages without requiring it, so the employee amount runs from zero to the cap at the employer's choice.", + "method": "One researcher per program read the statute, rules and agency pages and the engine source (2026-10-05). Two independent reviewers per program, one reading the statute and one the rules and agency guidance, then tried to refute each classification; review records each verdict. verification/law_research.json holds every record, quote, URL and verdict.", + "programs": [ + { + "key": "MN", + "program": "Minnesota Paid Leave", + "engine_variables": [ + "mn_employee_paid_leave_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "Minn. Stat. 268B.14, subds. 1, 3 and 5a", + "law_url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "employee_share_rule_text": "Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice.", + "rate_2026": "0.88% total (0.66% for small employers); employee share at most 0.44% of wages up to the Social Security base", + "benchmark_outputs": [ + "scenario_032" + ], + "engine_amounts": { + "scenario_032": 127.6 + }, + "review": { + "statute": { + "refuted": false, + "classification": "optional_employer_pass_through" + }, + "administration": { + "refuted": false, + "classification": "optional_employer_pass_through" + } + } + }, + { + "key": "CO", + "program": "Colorado FAMLI", + "engine_variables": [ + "co_employee_famli_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A)", + "law_url": "https://olls.info/crs/crs2025-title-08.pdf", + "employee_share_rule_text": "Colorado FAMLI premiums are remitted by the employer (C.R.S. 8-13.3-507(2)), which \"may deduct up to 50 percent of the premium\" from wages (507(5)); an employer that deducts less is considered to have elected to pay the rest (7 CCR 1107-1 s. 1.4(6)(A)).", + "rate_2026": "0.88% total; employee share at most 0.44% of wages up to the Social Security base", + "benchmark_outputs": [ + "scenario_043" + ], + "engine_amounts": { + "scenario_043": 17.99 + }, + "review": { + "statute": { + "refuted": false, + "classification": "optional_employer_pass_through" + }, + "administration": { + "refuted": false, + "classification": "optional_employer_pass_through" + } + } + }, + { + "key": "MA", + "program": "Massachusetts Paid Family and Medical Leave", + "engine_variables": [ + "ma_employee_paid_leave_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5)", + "law_url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "employee_share_rule_text": "Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)).", + "rate_2026": "0.88% total (0.18% family, 0.70% medical); employee share at most 0.46% of wages up to the Social Security base. Acts 2026 c. 101 ss. 25-26 (approved 2026-06-12) swap the caps to 40% family and 100% medical; s. 45 applies them to taxable years beginning on or after 2026-01-01, while DFML (after the freeze) dates the change 2027-01-01. Under the literal s. 45 reading the 2026 cap would be 0.772%.", + "benchmark_outputs": [ + "scenario_081" + ], + "engine_amounts": { + "scenario_081": 805.01 + }, + "review": { + "statute": { + "refuted": false, + "classification": "optional_employer_pass_through" + }, + "administration": { + "refuted": false, + "classification": "optional_employer_pass_through" + } + } + }, + { + "key": "NY", + "program": "New York Paid Family Leave and Disability Benefits", + "engine_variables": [ + "ny_employee_paid_family_leave_contribution", + "ny_employee_disability_benefits_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1)", + "law_url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "employee_share_rule_text": "New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them.", + "rate_2026": "PFL 0.432% of wages, at most $411.91; DBL 0.5% of wages, at most $0.60 a week", + "benchmark_outputs": [ + "scenario_082" + ], + "engine_amounts": { + "scenario_082": 443.11 + }, + "review": { + "statute": { + "refuted": false, + "classification": "optional_employer_pass_through" + }, + "administration": { + "refuted": false, + "classification": "optional_employer_pass_through" + } + } + }, + { + "key": "CT", + "program": "Connecticut Paid Leave", + "engine_variables": [ + "ct_employee_paid_leave_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "Conn. Gen. Stat. 31-49g(b)(1) and (3)", + "law_url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "employee_share_rule_text": "Each Connecticut employee \"shall contribute\" a percentage of subject earnings (Conn. Gen. Stat. 31-49g(b)(1)), and each employer \"shall deduct and withhold\" it (31-49g(b)(3)); there is no employer share.", + "rate_2026": "0.5% of wages up to the Social Security base", + "benchmark_outputs": [ + "scenario_120" + ], + "engine_amounts": { + "scenario_120": 827.98 + }, + "review": { + "statute": { + "refuted": false, + "classification": "mandatory_employee_withholding" + }, + "administration": { + "refuted": false, + "classification": "mandatory_employee_withholding" + } + } + }, + { + "key": "CA", + "program": "California State Disability Insurance", + "engine_variables": [ + "ca_employee_state_disability_insurance_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "Cal. Unemp. Ins. Code ss. 984(a)(1), 986(a) and 2901", + "law_url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "employee_share_rule_text": "Each California worker \"shall pay worker contributions\" (Cal. Unemp. Ins. Code s. 984(a)(1)), and each employer must withhold them in trust (s. 986(a)(1)); an employer that pays them for the worker still owes the worker's contribution, and the payment counts as wages (s. 986(a)(2)).", + "rate_2026": "1.3% of all wages, no cap", + "benchmark_outputs": [ + "scenario_005", + "scenario_022", + "scenario_023", + "scenario_099" + ], + "engine_amounts": { + "scenario_005": 5590.0, + "scenario_022": 684.26, + "scenario_023": 226.75, + "scenario_099": 2145.0 + }, + "review": { + "statute": { + "refuted": false, + "classification": "mandatory_employee_withholding" + }, + "administration": { + "refuted": false, + "classification": "mandatory_employee_withholding" + } + } + }, + { + "key": "PA", + "program": "Pennsylvania Unemployment Compensation employee contribution", + "engine_variables": [ + "pa_employee_unemployment_compensation_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "Pennsylvania UC Law s. 301.4(a)-(b), 43 P.S. 781.4", + "law_url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "employee_share_rule_text": "Each Pennsylvania employee \"shall pay contributions\" on all wages (UC Law s. 301.4(a)), and each employer \"shall withhold, in trust\" them (s. 301.4(b)).", + "rate_2026": "0.07% of all wages, no cap", + "benchmark_outputs": [ + "scenario_028", + "scenario_123" + ], + "engine_amounts": { + "scenario_028": 42.0, + "scenario_123": 101.5 + }, + "review": { + "statute": { + "refuted": false, + "classification": "mandatory_employee_withholding" + }, + "administration": { + "refuted": false, + "classification": "mandatory_employee_withholding" + } + } + }, + { + "key": "NJ", + "program": "New Jersey TDI and FLI worker contributions", + "engine_variables": [ + "nj_employee_temporary_disability_insurance_contribution", + "nj_employee_family_leave_insurance_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "N.J.S.A. 43:21-7(d)(1)(E) and (G)", + "law_url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "employee_share_rule_text": "Each New Jersey worker \"shall contribute\" to the TDI and FLI accounts (N.J.S.A. 43:21-7(d)(1)(G)), and each employer \"shall ... withhold in trust\" the contributions (43:21-7(d)(1)(E)).", + "rate_2026": "TDI 0.19% and FLI 0.23% of wages up to $171,100", + "benchmark_outputs": [ + "scenario_008" + ], + "engine_amounts": { + "scenario_008": 112.56 + }, + "note": "scenario_008's payroll output is already excluded (r05_nj_worker_ui: the engine omits the NJ worker UI and workforce contributions).", + "review": { + "statute": { + "refuted": false, + "classification": "mandatory_employee_withholding" + }, + "administration": { + "refuted": false, + "classification": "mandatory_employee_withholding" + } + } + } + ], + "programs_without_a_benchmark_household": { + "note": "No benchmark household lives in these states, so no reference depends on them. Each was researched once (verification/law_research.json, key OTHER) and not adversarially reviewed; payroll_mandatory_scope.py does not touch them.", + "programs": [ + { + "program": "Washington Paid Family and Medical Leave (PFML) premium, employee share", + "state": "WA", + "engine_variables": [ + "wa_employee_paid_leave_contribution" + ], + "classification": "mixed_or_unclear", + "law_citation": "RCW 50A.10.030(3)(a) (version effective until Jan. 1, 2028)", + "law_url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "quote": "For medical leave premiums, an employer may deduct from the wages of each employee up to the full amount of the premium required.", + "confidence": "medium" + }, + { + "program": "Washington WA Cares Fund (long-term services and supports) premium", + "state": "WA", + "engine_variables": [ + "wa_employee_long_term_care_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "RCW 50B.04.080(2)(a)", + "law_url": "https://app.leg.wa.gov/RCW/default.aspx?cite=50B.04.080", + "quote": "The employer must collect from the employees the premiums provided under this section through payroll deductions and remit the amounts collected to the employment security department.", + "confidence": "high" + }, + { + "program": "Oregon Paid Leave Oregon (family and medical leave insurance), employee contribution", + "state": "OR", + "engine_variables": [ + "or_employee_paid_leave_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "ORS 657B.150(2)(b)", + "law_url": "https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html", + "quote": "An employer shall deduct employee contributions from the wages of each employee in an amount that is equal to 60 percent of the total rate determined by the director.", + "confidence": "high" + }, + { + "program": "Oregon statewide transit tax", + "state": "OR", + "engine_variables": [ + "or_employee_statewide_transit_tax" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "ORS 320.550(2)", + "law_url": "https://www.oregonlegislature.gov/bills_laws/ors/ors320.html", + "quote": "A tax is imposed at the rate of one-tenth of one percent of the wages of an employee who is:", + "confidence": "high" + }, + { + "program": "Rhode Island Temporary Disability Insurance (TDI/TCI) employee contribution", + "state": "RI", + "engine_variables": [ + "ri_employee_temporary_disability_insurance_contribution" + ], + "classification": "mandatory_employee_withholding", + "law_citation": "R.I. Gen. Laws § 28-40-1(b) (version effective Jan. 1, 2026)", + "law_url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-1_28-40-1.htm", + "quote": "Each employee shall contribute ... an amount equal to the fund cost rate times the wages paid by the employer to the employee up to the taxable wage base", + "confidence": "high" + }, + { + "program": "Delaware Paid Leave (family and medical leave insurance) contributions, employee share", + "state": "DE", + "engine_variables": [ + "de_employee_paid_leave_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "19 Del. C. § 3705(d) (same wording in (e) and (f))", + "law_url": "https://delcode.delaware.gov/title19/c037/index.html", + "quote": "From the wages of an employee for whom a contribution is required under paragraph (b)(1) of this section, an employer shall deduct not greater than 50% of the contribution required for the employee.", + "confidence": "high" + }, + { + "program": "Maine Paid Family and Medical Leave premium, employee share", + "state": "ME", + "engine_variables": [ + "me_employee_paid_leave_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "26 M.R.S. § 850-F(5)(A)", + "law_url": "https://legislature.maine.gov/legis/statutes/26/title26sec850-F.html", + "quote": "An employer with 15 or more employees may deduct up to 50% of the premium required for an employee by subsection 3 from that employee's wages and shall remit 100% of the combined premium contribution", + "confidence": "high" + }, + { + "program": "Vermont Child Care Contribution, employee share", + "state": "VT", + "engine_variables": [ + "vt_employee_child_care_contribution" + ], + "classification": "optional_employer_pass_through", + "law_citation": "32 V.S.A. § 10553(a)(1)", + "law_url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "quote": "Each employer shall pay the Child Care Contribution on all covered wages paid to each of the employer’s employees", + "confidence": "high" + } + ] + } +} diff --git a/reference_audit/2026-10-05-payroll/proposed_exclusions.json b/reference_audit/2026-10-05-payroll/proposed_exclusions.json new file mode 100644 index 00000000..aaa8d1e9 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/proposed_exclusions.json @@ -0,0 +1,61 @@ +{ + "schema_version": 1, + "status": "proposed 2026-10-05; changes published scores, so it waits for Max's ruling (cos decision). decided_on is the draft date; a release sets it to the ruling's date. proposed_regenerations.json and proposed_exclusions.json are alternatives: a release adopts one of them, not both.", + "classification": "program_classification.json", + "sweep": "verification/sweep_payroll_scope.csv", + "basis": "policybench/reference_exclusions.py: an output is excluded for every model when its reference depends on an engine input the prompt never listed and a careful reader could take the stated facts the other way.", + "exclusions": [ + { + "scenario_id": "scenario_032", + "variable": "payroll_tax", + "reason_code": "reference_depends_on_unlisted_input", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The reference counts the largest employee share the employer may deduct (mn_employee_paid_leave_contribution $127.60), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "frozen_value": 2346.10009765625, + "alternative_value": 2218.5, + "engine_version": "policyengine-us 2.15.17", + "decided_on": "2026-10-05", + "decided_by": "developer", + "unlisted_input": "whether the employer deducts the employee share of Minnesota Paid Leave from wages, which the law permits but does not require (Minn. Stat. 268B.14, subds. 1, 3 and 5a)", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + }, + { + "scenario_id": "scenario_043", + "variable": "payroll_tax", + "reason_code": "reference_depends_on_unlisted_input", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Colorado FAMLI premiums are remitted by the employer (C.R.S. 8-13.3-507(2)), which \"may deduct up to 50 percent of the premium\" from wages (507(5)); an employer that deducts less is considered to have elected to pay the rest (7 CCR 1107-1 s. 1.4(6)(A)). The reference counts the largest employee share the employer may deduct (co_employee_famli_contribution $17.99), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "frozen_value": 330.7310791015625, + "alternative_value": 312.74322509765625, + "engine_version": "policyengine-us 2.15.17", + "decided_on": "2026-10-05", + "decided_by": "developer", + "unlisted_input": "whether the employer deducts the employee share of Colorado FAMLI from wages, which the law permits but does not require (C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A))", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + }, + { + "scenario_id": "scenario_081", + "variable": "payroll_tax", + "reason_code": "reference_depends_on_unlisted_input", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)). The reference counts the largest employee share the employer may deduct (ma_employee_paid_leave_contribution $805.01), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "frozen_value": 14192.662109375, + "alternative_value": 13387.65234375, + "engine_version": "policyengine-us 2.15.17", + "decided_on": "2026-10-05", + "decided_by": "developer", + "unlisted_input": "whether the employer deducts the employee share of Massachusetts Paid Family and Medical Leave from wages, which the law permits but does not require (M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5))", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + }, + { + "scenario_id": "scenario_082", + "variable": "payroll_tax", + "reason_code": "reference_depends_on_unlisted_input", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The reference counts the largest employee share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "frozen_value": 8108.02734375, + "alternative_value": 7664.91748046875, + "engine_version": "policyengine-us 2.15.17", + "decided_on": "2026-10-05", + "decided_by": "developer", + "unlisted_input": "whether the employer deducts the employee share of New York Paid Family Leave and Disability Benefits from wages, which the law permits but does not require (N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1))", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + } + ] +} diff --git a/reference_audit/2026-10-05-payroll/proposed_regenerations.json b/reference_audit/2026-10-05-payroll/proposed_regenerations.json new file mode 100644 index 00000000..abd2b0c0 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/proposed_regenerations.json @@ -0,0 +1,86 @@ +{ + "schema_version": 1, + "status": "proposed 2026-10-05; changes published scores, so it waits for Max's ruling (cos decision). decided_on is the draft date; a release sets it to the ruling's date. proposed_regenerations.json and proposed_exclusions.json are alternatives: a release adopts one of them, not both.", + "classification": "program_classification.json", + "sweep": "verification/sweep_payroll_scope.csv", + "basis": "The benchmark's output definitions hold (reference_audit/2026-09-28 rule 3): an output-scope adapter restores the payroll_tax definition, as latest_md_local_output_scope.py restored the state income tax output's.", + "regenerated": [ + { + "scenario_id": "scenario_032", + "variable": "payroll_tax", + "state": "MN", + "frozen_value": 2346.10009765625, + "regenerated_value": 2218.5, + "removed": { + "mn_employee_paid_leave_contribution": 127.6 + }, + "cause": "payroll_mandatory_scope", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The frozen reference counts the largest share the employer may deduct (mn_employee_paid_leave_contribution $127.60); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "law": "Minn. Stat. 268B.14, subds. 1, 3 and 5a", + "engine_version": "policyengine-us 2.15.17", + "fix_module": "payroll_mandatory_scope.py", + "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "applied_together_with": [ + "latest_final" + ] + }, + { + "scenario_id": "scenario_043", + "variable": "payroll_tax", + "state": "CO", + "frozen_value": 330.7310791015625, + "regenerated_value": 312.74322509765625, + "removed": { + "co_employee_famli_contribution": 17.99 + }, + "cause": "payroll_mandatory_scope", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. Colorado FAMLI premiums are remitted by the employer (C.R.S. 8-13.3-507(2)), which \"may deduct up to 50 percent of the premium\" from wages (507(5)); an employer that deducts less is considered to have elected to pay the rest (7 CCR 1107-1 s. 1.4(6)(A)). The frozen reference counts the largest share the employer may deduct (co_employee_famli_contribution $17.99); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "law": "C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A)", + "engine_version": "policyengine-us 2.15.17", + "fix_module": "payroll_mandatory_scope.py", + "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "applied_together_with": [ + "latest_final" + ] + }, + { + "scenario_id": "scenario_081", + "variable": "payroll_tax", + "state": "MA", + "frozen_value": 14192.662109375, + "regenerated_value": 13387.65234375, + "removed": { + "ma_employee_paid_leave_contribution": 805.01 + }, + "cause": "payroll_mandatory_scope", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)). The frozen reference counts the largest share the employer may deduct (ma_employee_paid_leave_contribution $805.01); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "law": "M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5)", + "engine_version": "policyengine-us 2.15.17", + "fix_module": "payroll_mandatory_scope.py", + "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "applied_together_with": [ + "latest_final" + ] + }, + { + "scenario_id": "scenario_082", + "variable": "payroll_tax", + "state": "NY", + "frozen_value": 8108.02734375, + "regenerated_value": 7664.91748046875, + "removed": { + "ny_employee_disability_benefits_contribution": 31.2, + "ny_employee_paid_family_leave_contribution": 411.91 + }, + "cause": "payroll_mandatory_scope", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The frozen reference counts the largest share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "law": "N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1)", + "engine_version": "policyengine-us 2.15.17", + "fix_module": "payroll_mandatory_scope.py", + "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "applied_together_with": [ + "latest_final" + ] + } + ] +} diff --git a/reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py b/reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py new file mode 100644 index 00000000..519a7132 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py @@ -0,0 +1,232 @@ +"""Split every PolicyBench payroll_tax reference into its federal and state parts. + +The payroll_tax output is ``spm_unit_payroll_tax``, which sums ``employee_payroll_tax`` +over the SPM unit's tax units. In policyengine-us 2.15.17 that adds employee Social +Security tax, employee Medicare tax, Additional Medicare Tax and +``employee_state_payroll_tax`` (variables/gov/irs/tax/payroll/employee_payroll_tax.py). +The state part adds one ``_employee_state_payroll_tax`` aggregate per state, each of +which adds that state's program contributions +(variables/gov/states/tax/payroll/employee_state_payroll_tax.py). + +The script rebuilds every household with ``Scenario.to_pe_household`` on the system that +built the published references (policyengine-us 2.15.17 plus ``latest_final``), checks +that ``spm_unit_payroll_tax`` reproduces every published payroll_tax reference, and +records each household's federal and per-program state contributions. + + PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH= \\ + /.venv-pe21517/bin/python \\ + reference_audit/2026-10-05-payroll/scripts/decompose_payroll.py \\ + --out-dir reference_audit/2026-10-05-payroll/verification +""" + +from __future__ import annotations + +import argparse +import importlib.util +import json +import shutil +import sys +import tempfile +from concurrent.futures import ProcessPoolExecutor +from importlib.metadata import version +from pathlib import Path + +import pandas as pd + +ROOT = Path(__file__).resolve().parents[3] +RUN = ( + ROOT + / "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace" +) +AUDIT_0928 = ROOT / "reference_audit/2026-09-28" +AUDIT_0922 = ROOT / "reference_audit/2026-09-22" +YEAR = 2026 +OUTPUT = "payroll_tax" +# sweep.py's rename for the 1.755.4 harness; sweep_latest.py kept it on 2.15.17. +RENAME = {"partnership_se_income": "partnership_self_employment_net_earnings"} +FEDERAL = ( + "employee_social_security_tax", + "employee_medicare_tax", + "additional_medicare_tax", +) +# Leaf programs under each _employee_state_payroll_tax in policyengine-us 2.15.17. +STATE_PROGRAMS = { + "CA": ("ca_employee_state_disability_insurance_contribution",), + "CO": ("co_employee_famli_contribution",), + "CT": ("ct_employee_paid_leave_contribution",), + "DE": ("de_employee_paid_leave_contribution",), + "MA": ("ma_employee_paid_leave_contribution",), + "ME": ("me_employee_paid_leave_contribution",), + "MN": ("mn_employee_paid_leave_contribution",), + "NJ": ( + "nj_employee_temporary_disability_insurance_contribution", + "nj_employee_family_leave_insurance_contribution", + ), + "NY": ( + "ny_employee_paid_family_leave_contribution", + "ny_employee_disability_benefits_contribution", + ), + "OR": ( + "or_employee_paid_leave_contribution", + "or_employee_statewide_transit_tax", + ), + "PA": ("pa_employee_unemployment_compensation_contribution",), + "RI": ("ri_employee_temporary_disability_insurance_contribution",), + "VT": ("vt_employee_child_care_contribution",), + "WA": ( + "wa_employee_paid_leave_contribution", + "wa_employee_long_term_care_contribution", + ), +} +LEAVES = tuple(v for programs in STATE_PROGRAMS.values() for v in programs) + +_SYSTEM = None + + +def _assemble_fixes() -> Path: + """latest_final and its parts, plus the sales tax table the IRS module reads.""" + target = Path(tempfile.mkdtemp(prefix="payroll_fixes_")) + for path in (AUDIT_0928 / "fixes").glob("*.py"): + shutil.copy2(path, target / path.name) + shutil.copy2(AUDIT_0922 / "fixes/r19_irs_sales_tax_2025.json", target) + return target + + +def _system(): + global _SYSTEM + if _SYSTEM is None: + from policyengine_us import CountryTaxBenefitSystem + + fix_dir = _assemble_fixes() + path = fix_dir / "latest_final.py" + spec = importlib.util.spec_from_file_location("latest_final", path) + module = importlib.util.module_from_spec(spec) + sys.modules["latest_final"] = module + spec.loader.exec_module(module) + _SYSTEM = CountryTaxBenefitSystem(reform=module.reform) + return _SYSTEM + + +def build_situation(scenario) -> dict: + situation = scenario.to_pe_household() + for person in situation["people"].values(): + for old, new in RENAME.items(): + if old in person: + person[new] = person.pop(old) + return situation + + +def decompose(scenario_json: str) -> dict: + from policyengine_us import Simulation + + from policybench.ground_truth import _extract_person_value, _pe_variable_for_output + from policybench.scenarios import scenario_from_dict + + scenario = scenario_from_dict(json.loads(scenario_json)) + sim = Simulation(tax_benefit_system=_system(), situation=build_situation(scenario)) + pe_variable = _pe_variable_for_output(OUTPUT, "us") + total = float( + _extract_person_value(sim.calculate(pe_variable, YEAR), scenario, OUTPUT) + ) + parts = {name: float(sim.calculate(name, YEAR).sum()) for name in FEDERAL} + parts["employee_state_payroll_tax"] = float( + sim.calculate("employee_state_payroll_tax", YEAR).sum() + ) + leaves = {name: float(sim.calculate(name, YEAR).sum()) for name in LEAVES} + people = [] + for person_id, person in build_situation(scenario)["people"].items(): + people.append(person_id) + wages = [float(x) for x in sim.calculate("employment_income", YEAR)] + return { + "scenario_id": scenario.id, + "state": scenario.state, + "payroll_tax": total, + "parts": parts, + "state_leaves": {k: v for k, v in leaves.items() if v != 0.0}, + "employment_income": wages, + "people": people, + } + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--out-dir", required=True) + parser.add_argument("--workers", type=int, default=6) + args = parser.parse_args() + + out_dir = Path(args.out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + reference = pd.read_csv(RUN / "reference_outputs.csv") + payroll = reference[reference["variable"] == OUTPUT].set_index("scenario_id") + exclusions = json.loads((RUN / "reference_exclusions.json").read_text())[ + "exclusions" + ] + excluded = {e["scenario_id"]: e for e in exclusions if e["variable"] == OUTPUT} + scenarios = pd.read_csv(RUN / "scenarios.csv") + scenarios = scenarios[scenarios["scenario_id"].isin(payroll.index)] + + engine = version("policyengine-us") + with ProcessPoolExecutor(max_workers=args.workers) as pool: + results = list(pool.map(decompose, scenarios["scenario_json"].tolist())) + results.sort(key=lambda r: r["scenario_id"]) + + rows = [] + for r in results: + sid = r["scenario_id"] + ref = float(payroll.loc[sid, "value"]) + federal = sum(r["parts"][name] for name in FEDERAL) + state = r["parts"]["employee_state_payroll_tax"] + rows.append( + { + "scenario_id": sid, + "state": r["state"], + "reference": ref, + "recomputed": r["payroll_tax"], + "matches_reference": abs(r["payroll_tax"] - ref) <= 1e-3, + **{name: r["parts"][name] for name in FEDERAL}, + "federal_total": federal, + "employee_state_payroll_tax": state, + "parts_add_up": abs(federal + state - r["payroll_tax"]) <= 1e-3, + "state_programs": ";".join( + f"{k}={v:.2f}" for k, v in sorted(r["state_leaves"].items()) + ), + "leaves_add_up": abs(sum(r["state_leaves"].values()) - state) <= 1e-3, + "excluded": sid in excluded, + "excluded_reason": excluded[sid]["reason_code"] + if sid in excluded + else "", + "household_wages": sum(r["employment_income"]), + } + ) + table = pd.DataFrame(rows) + table.to_csv(out_dir / "payroll_decomposition.csv", index=False) + (out_dir / "payroll_decomposition.json").write_text( + json.dumps({"policyengine_us": engine, "households": results}, indent=1) + "\n" + ) + print(f"policyengine-us {engine}; {len(table)} payroll outputs") + print(f"reproduce published reference: {int(table['matches_reference'].sum())}") + print(f"federal + state = total: {int(table['parts_add_up'].sum())}") + print(f"leaves add to state part: {int(table['leaves_add_up'].sum())}") + with_state = table[table["employee_state_payroll_tax"].abs() > 0] + print(f"outputs with a state component: {len(with_state)}") + with pd.option_context("display.width", 250, "display.max_rows", 200): + print( + with_state[ + [ + "scenario_id", + "state", + "reference", + "federal_total", + "employee_state_payroll_tax", + "state_programs", + "excluded", + ] + ].to_string(index=False) + ) + bad = table[~table["matches_reference"]] + if not bad.empty: + print("MISMATCH", bad[["scenario_id", "reference", "recomputed"]]) + + +if __name__ == "__main__": + main() diff --git a/reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py b/reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py new file mode 100644 index 00000000..eb8f9fd7 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py @@ -0,0 +1,401 @@ +"""Score the frozen run as published, with the proposed exclusions, and with regenerated references. + +Copies the frozen run (predictions, references, scenarios, exclusions) to scratch +directories and never writes the snapshot. Three copies are scored with +``python -m policybench.cli analyze``, the command the freeze runs: + +``published`` + Unchanged. It must reproduce the published payload's scoring: every modelStats field + except the cost and latency fields the freeze overlays, and programStats, heatmap, + globalWeights and failureModes exactly. The script stops if it does not. +``exclude`` + The records in ``proposed_exclusions.json`` appended to the exclusion record. +``regenerate`` + The references in ``proposed_regenerations.json`` written into the reference CSV + (and the copy's sidecar digest updated to match, as a release's sidecar would be). + +Sensitivity: the open state income tax withholding proposal (PolicyEngine/policybench#191, +decision d963) excludes three federal outputs and may land in the same release. With +``--with-salt`` the script also scores that proposal alone (``salt``) and each option on +top of it (``exclude+salt``, ``regenerate+salt``), reading its records from the pinned +commit ``SALT_COMMIT``; the ``*+salt`` deltas are measured against ``salt``. + + PYTHONPATH= /bin/python \\ + reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py --scratch \\ + [--with-salt] +""" + +from __future__ import annotations + +import argparse +import gzip +import hashlib +import json +import os +import shutil +import subprocess +import sys +from pathlib import Path + +import pandas as pd + +HERE = Path(__file__).resolve().parents[1] +ROOT = HERE.parents[1] +RUN = ( + ROOT + / "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace" +) +EXCLUSIONS = HERE / "proposed_exclusions.json" +REGENERATIONS = HERE / "proposed_regenerations.json" +OUT_DIR = HERE / "verification" +RUN_FILES = ( + "predictions.csv.gz", + "reference_outputs.csv", + "reference_outputs.csv.meta.json", + "reference_exclusions.json", + "scenarios.csv", + "scenarios.csv.meta.json", +) +# The freeze overlays these from the published payload; the analyze CLI does not. +OVERLAID = {"costUsd", "costPerHousehold", "totalTokens", "latencySeconds"} +METRICS = ( + "exact", + "within1pct", + "within5pct", + "within10pct", + "score", + "outputGroupScore", +) +EXACT_PAYLOAD_KEYS = ("programStats", "heatmap", "globalWeights", "failureModes") +RANKED = ("exact", "within1pct", "score") +VARIANTS = ("exclude", "regenerate") +OUTPUT = "payroll_tax" +SALT_COMMIT = "8af912a062dd7ae1373de4c043ae2727d3406930" +SALT_RECORDS = "reference_audit/2026-10-05/proposed_exclusions.json" + + +def salt_exclusions() -> list[dict]: + """The #191 proposal's records, read from its pinned commit.""" + shown = subprocess.run( + ["git", "show", f"{SALT_COMMIT}:{SALT_RECORDS}"], + cwd=ROOT, + check=True, + capture_output=True, + text=True, + ) + return json.loads(shown.stdout)["exclusions"] + + +def sha256(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def always_zero(run_dir: Path) -> dict[str, float]: + """The always-zero baseline, as paper_results._always_zero_weighted_rates scores it.""" + import numpy as np + + from policybench.analysis import weighted_hit_rate_scores_by_model + from policybench.reference_exclusions import scored_reference_for + from policybench.spec import get_output_ids, output_group_id + + truth, _ = scored_reference_for(run_dir / "reference_outputs.csv") + headline = set(get_output_ids("us", "headline")) + truth = truth[truth["variable"].map(output_group_id).isin(headline)] + truth = truth.reset_index(drop=True) + truth["scenario_id"] = truth["scenario_id"].astype(str) + scenarios = pd.read_csv(run_dir / "scenarios.csv") + market = dict( + zip( + scenarios["scenario_id"].astype(str), + pd.to_numeric(scenarios["total_income"], errors="coerce").fillna(0.0), + ) + ) + predictions = truth[["scenario_id", "variable"]].copy() + predictions["model"] = "Always zero" + predictions["prediction"] = np.zeros(len(truth)) + scored = weighted_hit_rate_scores_by_model(truth, predictions, market, country="us") + return { + "exact": float(scored["weighted_exact"].mean()) * 100, + "within1pct": float(scored["weighted_within_1pct"].mean()) * 100, + } + + +def stage(target: Path, variant: str | None) -> Path: + if target.exists(): + shutil.rmtree(target) + target.mkdir(parents=True) + for name in RUN_FILES: + shutil.copy2(RUN / name, target / name) + parts = set((variant or "").split("+")) - {""} + extra = [] + if "exclude" in parts: + extra += json.loads(EXCLUSIONS.read_text())["exclusions"] + if "salt" in parts: + extra += salt_exclusions() + if extra: + record = json.loads((target / "reference_exclusions.json").read_text()) + record["exclusions"].extend(extra) + (target / "reference_exclusions.json").write_text( + json.dumps(record, indent=2) + "\n" + ) + if "regenerate" in parts: + changes = json.loads(REGENERATIONS.read_text())["regenerated"] + reference = pd.read_csv(target / "reference_outputs.csv", dtype={"value": str}) + index = reference.set_index(["scenario_id", "variable"]).index + for change in changes: + key = (change["scenario_id"], change["variable"]) + row = index.get_loc(key) + if abs(float(reference.at[row, "value"]) - change["frozen_value"]) > 1e-9: + raise SystemExit(f"{key}: frozen value differs from the reference CSV") + reference.at[row, "value"] = repr(float(change["regenerated_value"])) + reference.to_csv(target / "reference_outputs.csv", index=False) + before = (RUN / "reference_outputs.csv").read_text().splitlines() + after = (target / "reference_outputs.csv").read_text().splitlines() + changed = [i for i, (a, b) in enumerate(zip(before, after)) if a != b] + if len(before) != len(after) or len(changed) != len(changes): + raise SystemExit( + f"regenerated CSV changes {len(changed)} lines, expected {len(changes)}" + ) + meta_path = target / "reference_outputs.csv.meta.json" + meta = json.loads(meta_path.read_text()) + meta["reference_csv_sha256"] = sha256(target / "reference_outputs.csv") + meta_path.write_text(json.dumps(meta, indent=2) + "\n") + return target + + +def analyze(run_dir: Path) -> dict: + out = run_dir / "analysis" + dashboard = run_dir / "dashboard.json" + env = dict(os.environ, PYTHONPATH=str(ROOT), PYTHONDONTWRITEBYTECODE="1") + subprocess.run( + [ + sys.executable, + "-m", + "policybench.cli", + "analyze", + "-g", + str(run_dir / "reference_outputs.csv"), + "-p", + str(run_dir / "predictions.csv.gz"), + "-s", + str(run_dir / "scenarios.csv"), + "-o", + str(out), + "--app-data-output", + str(dashboard), + "--reference-digest", + sha256(run_dir / "reference_outputs.csv"), + ], + cwd=ROOT, + env=env, + check=True, + stdout=subprocess.DEVNULL, + ) + payload = json.loads(dashboard.read_text()) + return payload["countries"]["us"] + + +def check_reproduces_published(base: dict) -> None: + published = json.loads(gzip.decompress((RUN / "data.json.gz").read_bytes())) + for key in EXACT_PAYLOAD_KEYS: + if published[key] != base[key]: + raise SystemExit(f"unchanged copy does not reproduce published {key}") + a = {m["model"]: m for m in published["modelStats"]} + b = {m["model"]: m for m in base["modelStats"]} + if set(a) != set(b): + raise SystemExit("model sets differ") + for model, row in a.items(): + for field, value in row.items(): + if field not in OVERLAID and value != b[model].get(field): + raise SystemExit(f"{model}.{field}: {value} != {b[model].get(field)}") + + +def ranks(frame: pd.DataFrame, metric: str) -> pd.Series: + return frame[metric].rank(ascending=False, method="min").astype(int) + + +def payroll_exact_counts(payload: dict) -> pd.Series: + """Each model's count of exact payroll_tax rows among scored outputs.""" + counts: dict[str, int] = {} + for outputs in payload["scenarioPredictions"].values(): + cells = outputs.get(OUTPUT, {}) + for model, cell in cells.items(): + if cell.get("scored"): + counts[model] = counts.get(model, 0) + int(cell["exact"] == 100.0) + return pd.Series(counts) + + +def compare(base: dict, other: dict) -> pd.DataFrame: + a = pd.DataFrame(base["modelStats"]).set_index("model") + b = pd.DataFrame(other["modelStats"]).set_index("model") + rows = pd.DataFrame(index=a.index) + for metric in METRICS: + rows[f"{metric}_published"] = a[metric] + rows[f"{metric}_proposed"] = b[metric] + rows[f"{metric}_delta"] = b[metric] - a[metric] + for metric in RANKED: + rows[f"{metric}_rank_published"] = ranks(a, metric) + rows[f"{metric}_rank_proposed"] = ranks(b, metric) + rows[f"{metric}_rank_change"] = ( + rows[f"{metric}_rank_published"] - rows[f"{metric}_rank_proposed"] + ) + rows["n_published"] = a["n"] + rows["n_proposed"] = b["n"] + rows["payroll_exact_published"] = payroll_exact_counts(base) + rows["payroll_exact_proposed"] = payroll_exact_counts(other) + rows["payroll_exact_delta"] = ( + rows["payroll_exact_proposed"] - rows["payroll_exact_published"] + ) + return rows.sort_values("exact_rank_published") + + +def program_rows(base: dict, other: dict) -> pd.DataFrame: + a = pd.DataFrame(base["programStats"]).set_index("variable") + b = pd.DataFrame(other["programStats"]).set_index("variable") + rows = pd.DataFrame(index=a.index) + for metric in ("exact", "score", "n"): + rows[f"{metric}_published"] = a[metric] + rows[f"{metric}_proposed"] = b[metric] + rows[f"{metric}_delta"] = b[metric] - a[metric] + return rows[(rows.filter(like="_delta").abs() > 1e-9).any(axis=1)] + + +def summarize( + models: pd.DataFrame, + programs: pd.DataFrame, + base: dict, + other: dict, + zero: dict, + changed: list[str], +) -> dict: + leader = models.sort_values("exact_rank_proposed").index[0] + return { + "changed_outputs": changed, + "scored_outputs": { + "published": int(base["modelStats"][0]["n"]), + "proposed": int(other["modelStats"][0]["n"]), + }, + "exact_delta_pp": { + "min": float(models["exact_delta"].min()), + "max": float(models["exact_delta"].max()), + "mean": float(models["exact_delta"].mean()), + }, + "score_delta": { + "min": float(models["score_delta"].min()), + "max": float(models["score_delta"].max()), + }, + **{ + f"{metric}_rank_changes": { + model: { + "published": int(row[f"{metric}_rank_published"]), + "proposed": int(row[f"{metric}_rank_proposed"]), + } + for model, row in models.iterrows() + if row[f"{metric}_rank_change"] != 0 + } + for metric in RANKED + }, + "always_zero_baseline": zero, + "leader_margin_over_always_zero_exact": { + "model": leader, + "published": float( + models.loc[leader, "exact_published"] - zero["published"]["exact"] + ), + "proposed": float( + models.loc[leader, "exact_proposed"] - zero["proposed"]["exact"] + ), + }, + "top5_exact_published": models.sort_values("exact_rank_published") + .head(5) + .index.tolist(), + "top5_exact_proposed": models.sort_values("exact_rank_proposed") + .head(5) + .index.tolist(), + "payroll_exact_delta_by_model": { + model: int(delta) + for model, delta in models["payroll_exact_delta"].items() + if delta != 0 + }, + "payroll_program": programs.loc[[OUTPUT]].to_dict(orient="index") + if OUTPUT in programs.index + else {}, + } + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--scratch", required=True) + parser.add_argument("--with-salt", action="store_true") + args = parser.parse_args() + scratch = Path(args.scratch) + + base_dir = stage(scratch / "published", None) + base = analyze(base_dir) + check_reproduces_published(base) + zero_published = always_zero(base_dir) + + OUT_DIR.mkdir(parents=True, exist_ok=True) + summary = {"published_reproduced": True} + plan = [(variant, base, zero_published) for variant in VARIANTS] + if args.with_salt: + salt_dir = stage(scratch / "salt", "salt") + salt = analyze(salt_dir) + salt_zero = always_zero(salt_dir) + plan.insert(0, ("salt", base, zero_published)) + plan += [(f"{variant}+salt", salt, salt_zero) for variant in VARIANTS] + summary["salt_commit"] = SALT_COMMIT + for variant, reference_payload, reference_zero in plan: + run_dir = stage(scratch / variant, variant) + other = analyze(run_dir) + models = compare(reference_payload, other) + programs = program_rows(reference_payload, other) + zero = {"published": reference_zero, "proposed": always_zero(run_dir)} + records = [] + if "exclude" in variant: + records += json.loads(EXCLUSIONS.read_text())["exclusions"] + if "regenerate" in variant: + records += json.loads(REGENERATIONS.read_text())["regenerated"] + if variant == "salt": + records += salt_exclusions() + changed = [f"{r['scenario_id']}/{r['variable']}" for r in records] + models.to_csv( + OUT_DIR / f"leaderboard_impact_{variant.replace('+', '_plus_')}_models.csv" + ) + programs.to_csv( + OUT_DIR + / f"leaderboard_impact_{variant.replace('+', '_plus_')}_programs.csv" + ) + summary[variant] = summarize( + models, programs, reference_payload, other, zero, changed + ) + summary[variant]["measured_against"] = ( + "salt" if variant.endswith("+salt") else "published" + ) + with pd.option_context("display.width", 250, "display.max_rows", 100): + print(f"\n=== {variant} ===") + print( + models[ + [ + "exact_published", + "exact_proposed", + "exact_delta", + "exact_rank_published", + "exact_rank_proposed", + "score_published", + "score_proposed", + "score_rank_published", + "score_rank_proposed", + "payroll_exact_published", + "payroll_exact_proposed", + ] + ].round(4) + ) + print(programs.round(4)) + (OUT_DIR / "leaderboard_impact.json").write_text( + json.dumps(summary, indent=2) + "\n" + ) + print(json.dumps(summary, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/reference_audit/2026-10-05-payroll/scripts/model_answers.py b/reference_audit/2026-10-05-payroll/scripts/model_answers.py new file mode 100644 index 00000000..dd70cbf0 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/scripts/model_answers.py @@ -0,0 +1,135 @@ +"""Score every model's answer on each payroll_tax output that carries a state component. + +Reads the published payload's scenarioPredictions for payroll_tax and scores each answer +with ``policybench.analysis.row_hit_scores`` (the $1 exact tolerance and the 1/5/10% +bands the board uses) against two references: + +``published`` + The frozen reference: federal employee payroll tax plus every state program + policyengine-us 2.15.17 puts in ``employee_state_payroll_tax``. +``no_optional`` + The frozen reference less the state programs classified in + ``program_classification.json`` as optional employer pass-through (the employer may, + but need not, deduct the employee share from wages). +``federal_only`` + Employee Social Security, Medicare and Additional Medicare Tax alone. + +The published-reference scores must equal the payload's own per-row ``exact`` field; the +script stops if any differs. + + PYTHONPATH= /bin/python \\ + reference_audit/2026-10-05-payroll/scripts/model_answers.py +""" + +from __future__ import annotations + +import gzip +import json +from pathlib import Path + +import pandas as pd + +from policybench.analysis import row_hit_scores + +HERE = Path(__file__).resolve().parents[1] +ROOT = HERE.parents[1] +RUN = ( + ROOT + / "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace" +) +OUT = HERE / "verification" +OUTPUT = "payroll_tax" + + +def main() -> None: + payload = json.loads(gzip.decompress((RUN / "data.json.gz").read_bytes())) + decomposition = pd.read_csv(OUT / "payroll_decomposition.csv") + classification = json.loads((HERE / "program_classification.json").read_text()) + optional = { + leaf + for program in classification["programs"] + if program["classification"] == "optional_employer_pass_through" + for leaf in program["engine_variables"] + } + affected = decomposition[decomposition["employee_state_payroll_tax"].abs() > 0] + + rows = [] + summary = [] + for _, d in affected.iterrows(): + sid = d["scenario_id"] + leaves = dict( + (k, float(v)) + for k, v in ( + item.split("=") for item in str(d["state_programs"]).split(";") if item + ) + ) + optional_amount = sum(v for k, v in leaves.items() if k in optional) + references = { + "published": float(d["reference"]), + "no_optional": float(d["reference"]) - optional_amount, + "federal_only": float(d["federal_total"]), + } + predictions = payload["scenarioPredictions"][sid][OUTPUT] + counts = {name: 0 for name in references} + within1 = {name: 0 for name in references} + for model, cell in sorted(predictions.items()): + if abs(cell["groundTruth"] - references["published"]) > 1e-3: + raise SystemExit(f"{sid}: payload groundTruth differs from the CSV") + pred = cell["prediction"] if cell["parsed"] else None + row = { + "scenario_id": sid, + "state": d["state"], + "model": model, + "prediction": pred, + "scored": cell["scored"], + } + for name, ref in references.items(): + hit = row_hit_scores(OUTPUT, ref, pred) + row[f"exact_{name}"] = hit["exact"] + row[f"within1pct_{name}"] = hit["within_1pct"] + counts[name] += int(hit["exact"]) + within1[name] += int(hit["within_1pct"]) + if abs(row["exact_published"] * 100 - cell["exact"]) > 1e-9: + raise SystemExit(f"{sid}/{model}: exact differs from the payload") + rows.append(row) + n = len(predictions) + frame = pd.DataFrame([r for r in rows if r["scenario_id"] == sid]) + answered = frame["prediction"].dropna().round(2).value_counts() + summary.append( + { + "scenario_id": sid, + "state": d["state"], + "scored": bool(not d["excluded"]), + "state_programs": d["state_programs"], + "optional_amount": round(optional_amount, 2), + "reference_published": round(references["published"], 2), + "reference_no_optional": round(references["no_optional"], 2), + "reference_federal_only": round(references["federal_only"], 2), + "models": n, + "exact_published": counts["published"], + "exact_no_optional": counts["no_optional"], + "exact_federal_only": counts["federal_only"], + "within1pct_published": within1["published"], + "within1pct_no_optional": within1["no_optional"], + "within1pct_federal_only": within1["federal_only"], + "modal_answers": { + f"{value:.2f}": int(count) + for value, count in answered.head(4).items() + }, + } + ) + table = pd.DataFrame(rows) + table.to_csv(OUT / "model_answers.csv", index=False) + (OUT / "model_answers_summary.json").write_text( + json.dumps({"outputs": summary}, indent=2) + "\n" + ) + with pd.option_context("display.width", 250, "display.max_columns", 30): + print( + pd.DataFrame(summary).drop(columns=["modal_answers"]).to_string(index=False) + ) + for item in summary: + print(item["scenario_id"], item["modal_answers"]) + + +if __name__ == "__main__": + main() diff --git a/reference_audit/2026-10-05-payroll/scripts/propose_changes.py b/reference_audit/2026-10-05-payroll/scripts/propose_changes.py new file mode 100644 index 00000000..48f62027 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/scripts/propose_changes.py @@ -0,0 +1,246 @@ +"""Draft both candidate records for payroll outputs that count an optional employer pass-through. + +Reads the sweep (``sweep_payroll_scope.py``) and the program classification +(``program_classification.json``) and writes two alternative proposals, each in the +format a release would install: + +``proposed_regenerations.json`` + One entry per scored output the output-scope adapter moves: the frozen reference, + the regenerated value (policyengine-us 2.15.17 + latest_final + + fixes/payroll_mandatory_scope.py), and the program law. A release that adopts it + writes the values into the reference CSV and adds one ``output_scope`` revision to + the reference sidecar, as the Maryland adapter was recorded. +``proposed_exclusions.json`` + One ``reference_depends_on_unlisted_input`` record per such output, in the format of + the frozen run's ``reference_exclusions.json``. The unlisted input is whether the + employer deducts the employee share, which the law leaves to the employer. + +Both change published scores, so they wait for Max's ruling; ``decided_on`` is the date +the proposal was drafted and a release sets it to the ruling's date. + + python reference_audit/2026-10-05-payroll/scripts/propose_changes.py +""" + +from __future__ import annotations + +import hashlib +import json +from pathlib import Path + +import pandas as pd + +HERE = Path(__file__).resolve().parents[1] +ROOT = HERE.parents[1] +RUN = ( + ROOT + / "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace" +) +SWEEP = HERE / "verification/sweep_payroll_scope.csv" +DECOMPOSITION = HERE / "verification/payroll_decomposition.csv" +CLASSIFICATION = HERE / "program_classification.json" +ADAPTER = HERE / "fixes/payroll_mandatory_scope.py" +DRAFTED_ON = "2026-10-05" +ENGINE = "policyengine-us 2.15.17" +OUTPUT = "payroll_tax" +STATUS = ( + f"proposed {DRAFTED_ON}; changes published scores, so it waits for Max's ruling " + "(cos decision). decided_on is the draft date; a release sets it to the ruling's " + "date. proposed_regenerations.json and proposed_exclusions.json are alternatives: " + "a release adopts one of them, not both." +) + + +def money(value: float) -> str: + return f"${value:,.2f}" + + +def program_for(leaf: str, programs: list[dict]) -> dict: + for program in programs: + if leaf in program["engine_variables"]: + return program + raise KeyError(leaf) + + +def adapter_optional() -> set[str]: + """The programs payroll_mandatory_scope.py drops, read from its source.""" + import ast + + tree = ast.parse(ADAPTER.read_text()) + for node in tree.body: + if isinstance(node, ast.Assign) and any( + isinstance(t, ast.Name) and t.id == "OPTIONAL" for t in node.targets + ): + return set(ast.literal_eval(node.value)) + raise SystemExit("the adapter defines no OPTIONAL") + + +def main() -> None: + sweep = pd.read_csv(SWEEP) + decomposition = pd.read_csv(DECOMPOSITION).set_index("scenario_id") + classification = json.loads(CLASSIFICATION.read_text()) + programs = classification["programs"] + adapter_sha = hashlib.sha256(ADAPTER.read_bytes()).hexdigest() + + # The adapter drops exactly the reviewed programs classified as optional. + reviewed_optional = { + leaf + for program in programs + if program["classification"] == "optional_employer_pass_through" + and all(not v["refuted"] for v in program["review"].values()) + for leaf in program["engine_variables"] + } + if adapter_optional() != reviewed_optional: + raise SystemExit( + f"adapter OPTIONAL {sorted(adapter_optional())} differs from the reviewed " + f"optional programs {sorted(reviewed_optional)}" + ) + + moved = sweep[sweep["moved_any"]] + unexpected = moved[(moved["variable"] != OUTPUT) | moved["excluded"]] + if not unexpected.empty: + raise SystemExit( + f"the adapter moves outputs this proposal does not cover:\n{unexpected}" + ) + + regenerated, excluded = [], [] + for row in moved.itertuples(): + d = decomposition.loc[row.scenario_id] + leaves = dict( + (k, float(v)) + for k, v in ( + item.split("=") for item in str(d["state_programs"]).split(";") + ) + ) + removed = { + leaf: amount + for leaf, amount in leaves.items() + if program_for(leaf, programs)["classification"] + == "optional_employer_pass_through" + } + cited = [] + for leaf in removed: + program = program_for(leaf, programs) + if program not in cited: + cited.append(program) + names = " and ".join(p["program"] for p in cited) + law = "; ".join(p["law_citation"] for p in cited) + removed_text = ", ".join( + f"{leaf} {money(amount)}" for leaf, amount in sorted(removed.items()) + ) + if abs(sum(removed.values()) - (row.final - row.scoped)) > 0.01: + raise SystemExit( + f"{row.scenario_id}: removed programs do not explain the move" + ) + why = " ".join(p["employee_share_rule_text"] for p in cited) + regenerated.append( + { + "scenario_id": row.scenario_id, + "variable": OUTPUT, + "state": row.state, + "frozen_value": float(row.final), + "regenerated_value": float(row.scoped), + "removed": removed, + "cause": "payroll_mandatory_scope", + "basis": ( + f"The output's definition counts mandatory employee state payroll " + f"taxes and excludes employer payroll taxes. {why} The frozen " + f"reference counts the largest share the employer may deduct " + f"({removed_text}); the regenerated value leaves it out and keeps " + f"employee Social Security and Medicare tax." + ), + "law": law, + "engine_version": ENGINE, + "fix_module": "payroll_mandatory_scope.py", + "fix_module_sha256": adapter_sha, + "applied_together_with": ["latest_final"], + } + ) + excluded.append( + { + "scenario_id": row.scenario_id, + "variable": OUTPUT, + "reason_code": "reference_depends_on_unlisted_input", + "alternative_reading": ( + f"The prompt defines payroll_tax as employee Social Security, " + f"Medicare and Additional Medicare Tax plus mandatory employee state " + f"payroll taxes. {why} The reference counts the largest employee " + f"share the employer may deduct ({removed_text}), as policyengine-us " + f"assumes. Read as an employer premium that is not a mandatory " + f"employee tax, or with an employer that does not deduct it, the " + f"output is employee federal payroll tax alone, the alternative " + f"value (policyengine-us 2.15.17 with latest_final and " + f"payroll_mandatory_scope)." + ), + "frozen_value": float(row.final), + "alternative_value": float(row.scoped), + "engine_version": ENGINE, + "decided_on": DRAFTED_ON, + "decided_by": "developer", + "unlisted_input": ( + f"whether the employer deducts the employee share of {names} from " + f"wages, which the law permits but does not require ({law})" + ), + "note": ( + f"Found {DRAFTED_ON} by classifying every state program in the " + f"payroll references from primary law " + f"(reference_audit/2026-10-05-payroll/program_classification.json) " + f"and sweeping all 1,984 outputs with the programs classified as " + f"optional employer pass-through left out of employee state payroll " + f"tax; that moves this output alone in its household and no other " + f"output. The frozen value has been the reference since the first " + f"run (2026-06-15)." + ), + } + ) + + common = { + "schema_version": 1, + "status": STATUS, + "classification": "program_classification.json", + "sweep": "verification/sweep_payroll_scope.csv", + } + (HERE / "proposed_regenerations.json").write_text( + json.dumps( + { + **common, + "basis": ( + "The benchmark's output definitions hold (reference_audit/2026-09-28 " + "rule 3): an output-scope adapter restores the payroll_tax " + "definition, as latest_md_local_output_scope.py restored the state " + "income tax output's." + ), + "regenerated": regenerated, + }, + indent=2, + ) + + "\n" + ) + (HERE / "proposed_exclusions.json").write_text( + json.dumps( + { + **common, + "basis": ( + "policybench/reference_exclusions.py: an output is excluded for every " + "model when its reference depends on an engine input the prompt " + "never listed and a careful reader could take the stated facts the " + "other way." + ), + "exclusions": excluded, + }, + indent=2, + ) + + "\n" + ) + for item in regenerated: + print( + item["scenario_id"], + item["state"], + money(item["frozen_value"]), + "->", + money(item["regenerated_value"]), + item["removed"], + ) + + +if __name__ == "__main__": + main() diff --git a/reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py b/reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py new file mode 100644 index 00000000..f1a1a5fb --- /dev/null +++ b/reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py @@ -0,0 +1,269 @@ +"""Recompute every PolicyBench reference with and without the payroll output-scope adapter. + +The reference system is the one that built the published references: policyengine-us +2.15.17 plus ``latest_final`` (the nine pre-freeze-law conventions and the Maryland +output-scope adapter), on households built by ``Scenario.to_pe_household``. The script +recomputes every output under it and checks the result against the published reference +CSV. It then recomputes every output under ``latest_final`` plus +``fixes/payroll_mandatory_scope.py``, which drops the employee shares an employer may but +need not deduct from ``employee_state_payroll_tax``, and lists every output that moves. + +Any output other than payroll_tax that reads employee payroll tax (for example through +``spm_unit_paycheck_withholdings`` or ``household_tax_before_refundable_credits``) would +move too, so the sweep covers all 1,984 outputs, not only payroll_tax. + + PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH= \\ + /.venv-pe21517/bin/python \\ + reference_audit/2026-10-05-payroll/scripts/sweep_payroll_scope.py \\ + --out-dir reference_audit/2026-10-05-payroll/verification +""" + +from __future__ import annotations + +import argparse +import hashlib +import importlib.util +import json +import shutil +import sys +import tempfile +from concurrent.futures import ProcessPoolExecutor +from importlib.metadata import version +from pathlib import Path + +import pandas as pd + +HERE = Path(__file__).resolve().parents[1] +ROOT = HERE.parents[1] +RUN = ( + ROOT + / "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace" +) +AUDIT_0928 = ROOT / "reference_audit/2026-09-28" +AUDIT_0922 = ROOT / "reference_audit/2026-09-22" +ADAPTER = HERE / "fixes/payroll_mandatory_scope.py" +YEAR = 2026 +BINARY_SUFFIXES = ("_eligible",) +TOLERANCE = 1.0 +# sweep.py's rename for the 1.755.4 harness; sweep_latest.py kept it on 2.15.17. +RENAME = {"partnership_se_income": "partnership_self_employment_net_earnings"} + +_SYSTEMS: dict[str, object] = {} + + +def _load(path: Path, name: str): + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + return module + + +def _fix_dir() -> Path: + """latest_final and its parts, plus the sales tax table the IRS module reads.""" + target = Path(tempfile.mkdtemp(prefix="payroll_scope_fixes_")) + for path in (AUDIT_0928 / "fixes").glob("*.py"): + shutil.copy2(path, target / path.name) + shutil.copy2(AUDIT_0922 / "fixes/r19_irs_sales_tax_2025.json", target) + return target + + +def _system(name: str): + if name not in _SYSTEMS: + from policyengine_core.reforms import Reform + from policyengine_us import CountryTaxBenefitSystem + + final = _load(_fix_dir() / "latest_final.py", "latest_final").reform + if name == "final": + _SYSTEMS[name] = CountryTaxBenefitSystem(reform=final) + else: + adapter = _load(ADAPTER, "payroll_mandatory_scope").reform + + class combined(Reform): + def apply(self): + final.apply(self) + adapter.apply(self) + + _SYSTEMS[name] = CountryTaxBenefitSystem(reform=combined) + return _SYSTEMS[name] + + +def build_situation(scenario) -> dict: + situation = scenario.to_pe_household() + for person in situation["people"].values(): + for old, new in RENAME.items(): + if old in person: + person[new] = person.pop(old) + return situation + + +def _outputs(sim, scenario, variables) -> dict[str, float]: + from policybench.ground_truth import _extract_person_value, _pe_variable_for_output + + values = {} + for variable in variables: + pe_variable = _pe_variable_for_output(variable, "us") + values[variable] = float( + _extract_person_value(sim.calculate(pe_variable, YEAR), scenario, variable) + ) + return values + + +def sweep_scenario(args: tuple[str, list[str]]) -> dict: + from policyengine_us import Simulation + + from policybench.scenarios import scenario_from_dict + + scenario_json, variables = args + scenario = scenario_from_dict(json.loads(scenario_json)) + situation = build_situation(scenario) + result = {"scenario_id": scenario.id, "state": scenario.state} + for name in ("final", "scoped"): + sim = Simulation(tax_benefit_system=_system(name), situation=situation) + result[name] = _outputs(sim, scenario, variables) + result[f"{name}_state_payroll"] = float( + sim.calculate("employee_state_payroll_tax", YEAR).sum() + ) + return result + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument("--out-dir", required=True) + parser.add_argument("--workers", type=int, default=6) + args = parser.parse_args() + + from policybench.scenarios import scenario_from_dict + from policybench.spec import expand_programs_for_scenario + + out_dir = Path(args.out_dir) + out_dir.mkdir(parents=True, exist_ok=True) + meta = json.loads((RUN / "reference_outputs.csv.meta.json").read_text()) + reference = pd.read_csv(RUN / "reference_outputs.csv") + indexed = reference.set_index(["scenario_id", "variable"])["value"] + exclusions = json.loads((RUN / "reference_exclusions.json").read_text())[ + "exclusions" + ] + excluded = {(e["scenario_id"], e["variable"]): e for e in exclusions} + scenarios = pd.read_csv(RUN / "scenarios.csv") + + jobs = [] + for _, row in scenarios.iterrows(): + scenario = scenario_from_dict(json.loads(row["scenario_json"])) + variables = [ + v + for v in expand_programs_for_scenario(meta["programs"], scenario) + if (scenario.id, v) in indexed.index + ] + jobs.append((row["scenario_json"], variables)) + + engine = version("policyengine-us") + print(f"policyengine-us {engine}; {len(jobs)} households", flush=True) + with ProcessPoolExecutor(max_workers=args.workers) as pool: + results = list(pool.map(sweep_scenario, jobs)) + results.sort(key=lambda r: r["scenario_id"]) + + rows = [] + for result in results: + sid = result["scenario_id"] + for variable, base in result["final"].items(): + key = (sid, variable) + ref = float(indexed[key]) + scoped = result["scoped"][variable] + entry = excluded.get(key) + binary = variable.endswith(BINARY_SUFFIXES) + rows.append( + { + "scenario_id": sid, + "state": result["state"], + "variable": variable, + "reference": ref, + "final": base, + "final_matches_reference": ( + round(base) == round(ref) if binary else abs(base - ref) <= 1e-3 + ), + "scoped": scoped, + "delta": scoped - base, + "moved_any": scoped != base, + "moved_over_tolerance": ( + round(scoped) != round(base) + if binary + else abs(scoped - base) > TOLERANCE + ), + "excluded": entry is not None, + "excluded_reason": entry["reason_code"] if entry else "", + } + ) + table = pd.DataFrame(rows) + table.to_csv(out_dir / "sweep_payroll_scope.csv", index=False) + households = { + r["scenario_id"]: { + "state": r["state"], + "employee_state_payroll_tax_final": r["final_state_payroll"], + "employee_state_payroll_tax_scoped": r["scoped_state_payroll"], + } + for r in results + } + (out_dir / "sweep_payroll_scope_households.json").write_text( + json.dumps( + { + "policyengine_us": engine, + "adapter_sha256": hashlib.sha256(ADAPTER.read_bytes()).hexdigest(), + "households": households, + }, + indent=1, + sort_keys=True, + ) + + "\n" + ) + + scored = table[~table["excluded"]] + mismatched = table[~table["final_matches_reference"]] + # Excluded outputs decided before the 2.15.17 upgrade keep their decided values; the + # sidecar's engine_upgrade revision lists each one's 2.15.17 value. + rechecked = { + (item["scenario_id"], item["variable"]): item["value_on_2_15_17"] + for revision in meta["revisions"] + for item in revision.get("excluded_outputs_rechecked", []) + } + unexplained = [ + (row.scenario_id, row.variable) + for row in mismatched.itertuples() + if not row.excluded + or abs(rechecked.get((row.scenario_id, row.variable), float("nan")) - row.final) + > 1e-3 + ] + print( + f"baseline: {len(table)} outputs, {len(scored)} scored; " + f"{int((~mismatched['excluded']).sum())} scored outputs differ from the " + f"published reference; {int(mismatched['excluded'].sum())} excluded outputs " + f"keep their decided values ({len(rechecked)} rechecked in the sidecar); " + f"unexplained differences: {unexplained}" + ) + if unexplained: + raise SystemExit("the baseline does not reproduce the published references") + moved = table[table["moved_any"]] + print( + f"\nadapter: {len(moved)} outputs move at all, " + f"{int(table['moved_over_tolerance'].sum())} by more than ${TOLERANCE:.0f} " + f"({int((table['moved_over_tolerance'] & ~table['excluded']).sum())} scored)" + ) + with pd.option_context("display.width", 220, "display.max_rows", 500): + print( + moved[ + [ + "scenario_id", + "state", + "variable", + "reference", + "final", + "scoped", + "delta", + "excluded", + ] + ].to_string(index=False) + ) + + +if __name__ == "__main__": + main() diff --git a/reference_audit/2026-10-05-payroll/verification/explanations.json b/reference_audit/2026-10-05-payroll/verification/explanations.json new file mode 100644 index 00000000..0f9663f7 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/explanations.json @@ -0,0 +1,3462 @@ +{ + "date": "2026-10-05", + "method": "Claude Code workflow wf_cecd10dd-abe: four reviewers, each classifying every model explanation on a group of payroll outputs into one category with a quote of at most 30 words.", + "outputs": [ + { + "scenario_id": "scenario_005", + "counts": { + "included_state_component": 31, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 2, + "excluded_silently": 8, + "other_or_unparsed": 5 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 33296.55, + "category": "included_state_component", + "quote": "CA SDI (employee-paid, no wage cap since 2024) at 1.2% on $430,000 = $2,898.35... actual: 1.2% * 430,000 = $5,160?" + }, + { + "model": "claude-fable-5.1", + "prediction": 30454, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 41859.6, + "category": "included_state_component", + "quote": "CA SDI = $1,531.64 + $1,531.64 = $3,063.28; CA ETT = 0.08% on first $7,000 = $5.60 \u00d7 2 = $11.20." + }, + { + "model": "claude-opus-4.7", + "prediction": 35446.6, + "category": "included_state_component", + "quote": "CA SDI (1.2% in 2026, no wage cap) on head: 180000*0.012 = $2,160; spouse: 250000*0.012 = $3,000." + }, + { + "model": "claude-opus-4.8", + "prediction": 28586.6, + "category": "other_or_unparsed", + "quote": "this exceeds my value, so final: SS+Medicare+AddlMedicare only without SDI uncertainty = 30454." + }, + { + "model": "claude-opus-5", + "prediction": 30437, + "category": "other_or_unparsed", + "quote": "Plus CA SDI 1.2% on wages... included in mandatory state payroll tax portion already reflected; total \u2248 $30,437." + }, + { + "model": "claude-opus-5.5", + "prediction": 36044, + "category": "included_state_component", + "quote": "CA SDI: $5,590 (1.3% \u00d7 $430,000, uncapped)." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 34851.4, + "category": "included_state_component", + "quote": "(4) CA mandatory employee payroll taxes: CA State Disability Insurance (SDI) \u2014 2026 rate is 1.2% (no wage cap as of 2024+ reform)." + }, + { + "model": "claude-sonnet-5", + "prediction": 27306, + "category": "other_or_unparsed", + "quote": "CA has no mandatory employee payroll tax like SDI in this model... Total payroll tax = 29,691.40+4,730=34,421.40. Rechecking final consolidated estimate: value = 27306." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 36044, + "category": "included_state_component", + "quote": "CA SDI at an assumed 1.3% of $430k is $5,590." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 19108, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "deepseek-v4-pro", + "prediction": 34342, + "category": "included_state_component", + "quote": "+ CA SDI ($4,080) = $34,342." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 33962.54, + "category": "included_state_component", + "quote": "California SDI $3,675.94 (2 x 1.2% of $153,164)" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 35595.4, + "category": "included_state_component", + "quote": "and California SDI 5,160 (1.2% on all wages)." + }, + { + "model": "gemini-3-flash-preview", + "prediction": 33827, + "category": "included_state_component", + "quote": "and California SDI tax of $4,730 (1.1% of total wages)." + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 31398, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 34258, + "category": "included_state_component", + "quote": "Additional Medicare Tax, and CA SDI on total wage bases after ESI deductions." + }, + { + "model": "gemini-3.5-flash", + "prediction": 33700, + "category": "included_state_component", + "quote": "and CA State Disability Insurance (SDI) at 1.1% on all wages." + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 26588, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 34091.05, + "category": "included_state_component", + "quote": "CA State Disability Insurance ($1,904.22 for Head, $2,678.72 for Spouse at 1.1% uncapped)" + }, + { + "model": "gemini-3.7-flash", + "prediction": 33814, + "category": "included_state_component", + "quote": "and California State Disability Insurance (SDI)." + }, + { + "model": "gemini-3.8-flash", + "prediction": 29691.4, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 34274.42, + "category": "included_state_component", + "quote": "CA SDI (1.2%, cap $165,384 per worker): Head $1,984.61 + Spouse $1,984.61 = $3,969.22." + }, + { + "model": "glm-5.3", + "prediction": 30454, + "category": "excluded_as_no_state_payroll_tax", + "quote": "California has no mandatory employee state payroll tax, total $30,454" + }, + { + "model": "gpt-5.4-mini", + "prediction": 22295.1, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-nano", + "prediction": 13905, + "category": "included_state_component", + "quote": "plus employee Additional Medicare Tax where applicable and CA state employee payroll taxes; employer payroll taxes excluded." + }, + { + "model": "gpt-5.5", + "prediction": 35558.2, + "category": "included_state_component", + "quote": "and CA SDI of $5,160 at 1.2% of wages." + }, + { + "model": "gpt-5.6-luna", + "prediction": 36667, + "category": "included_state_component", + "quote": "and approximately 1.3% California SDI on wages." + }, + { + "model": "gpt-5.6-sol", + "prediction": 35614, + "category": "included_state_component", + "quote": "and California SDI withholding." + }, + { + "model": "gpt-5.6-terra", + "prediction": 35614, + "category": "included_state_component", + "quote": "and estimated California employee SDI." + }, + { + "model": "gpt-6-astra", + "prediction": 35128.91, + "category": "included_state_component", + "quote": "and California SDI at 1.3% is $5,416.20" + }, + { + "model": "gpt-6-luna", + "prediction": 35614, + "category": "included_state_component", + "quote": "and California employee SDI of $5,160." + }, + { + "model": "gpt-6-sol", + "prediction": 36044, + "category": "included_state_component", + "quote": "California SDI at 1.3% of 430000 in wages is 5590" + }, + { + "model": "gpt-6.1-sol", + "prediction": 35128.91, + "category": "included_state_component", + "quote": "and California SDI at 1.3%" + }, + { + "model": "grok-4.3", + "prediction": 34515, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 34525, + "category": "included_state_component", + "quote": "+ CA SDI 4226 (1.2% capped)" + }, + { + "model": "grok-4.6", + "prediction": 34704, + "category": "included_state_component", + "quote": "and CA SDI is 1.2% up to the SDI wage limit" + }, + { + "model": "grok-4.7", + "prediction": 36044, + "category": "included_state_component", + "quote": "California SDI at 1.3% with no wage cap is $5,590." + }, + { + "model": "grok-build-0.1", + "prediction": 31456, + "category": "included_state_component", + "quote": "CA SDI 1.1% on 83067 base per person = 1827." + }, + { + "model": "inkling", + "prediction": 32773, + "category": "included_state_component", + "quote": "plus CA SDI (~1.2% on capped wages)." + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 35614, + "category": "included_state_component", + "quote": "employee CA SDI at 1.2% with no wage cap is $5,160." + }, + { + "model": "minimax-m3", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "fully offset by deductions/exemptions in the tax calculation, yielding zero net employee-side payroll tax in this estimate." + }, + { + "model": "ox-alpha", + "prediction": 31084, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No mandatory CA employee payroll tax modeled." + }, + { + "model": "qwen-3.7-max", + "prediction": 33061, + "category": "included_state_component", + "quote": "CA SDI: 1.1% on wage base $153,164 (projected 2026; using approximate) for each earner." + }, + { + "model": "qwen3.8-max", + "prediction": 25474, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_008", + "counts": { + "included_state_component": 23, + "excluded_as_optional_or_employer_paid": 1, + "excluded_as_no_state_payroll_tax": 4, + "excluded_silently": 11, + "other_or_unparsed": 7 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 2178.85, + "category": "included_state_component", + "quote": "NJ mandatory employee payroll taxes on wages: unemployment insurance 0.3825% ($102.51) + family leave insurance ~0.09% ($24.12)" + }, + { + "model": "claude-fable-5.1", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 2052.12, + "category": "included_state_component", + "quote": "NJ payroll tax approximately $2.12 minimum assessment, total value = $2052.12" + }, + { + "model": "claude-opus-4.7", + "prediction": 2050.2, + "category": "other_or_unparsed", + "quote": "FLI ~0.33% but uncertain; UI ~0.3825%. Using just federal FICA: 1661.60+388.60 = 2050.20." + }, + { + "model": "claude-opus-4.8", + "prediction": 2050.2, + "category": "other_or_unparsed", + "quote": "NJ mandatory employee payroll taxes (UI/SDI/FLI) are not modeled here, so 0." + }, + { + "model": "claude-opus-5", + "prediction": 2050, + "category": "other_or_unparsed", + "quote": "(NJ employee UI/TDI/FLI not included in this measure)" + }, + { + "model": "claude-opus-5.5", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-sonnet-4.6", + "prediction": 2050.2, + "category": "excluded_as_optional_or_employer_paid", + "quote": "NJ SDI/TDI is employer-paid; NJ FLI employee contribution is a separate deduction but is not a 'payroll tax' in the traditional sense" + }, + { + "model": "claude-sonnet-5", + "prediction": 2050.2, + "category": "other_or_unparsed", + "quote": "0 (NJ employee tax not separately itemized due to cap uncertainty) = 2,050.20." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 2050.2, + "category": "other_or_unparsed", + "quote": "NJ employee payroll taxes are not modeled." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 2050, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "deepseek-v4-pro", + "prediction": 2101.12, + "category": "included_state_component", + "quote": "NJ TDI (0.10% = $26.80) + NJ FLI (0.09% = $24.12)" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 2204.3, + "category": "included_state_component", + "quote": "NJ employee state payroll taxes $154.10" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 2188, + "category": "included_state_component", + "quote": "approximately 0.515% for New Jersey's mandatory SUI and FLI taxes ($138.02)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 2050, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 2188, + "category": "included_state_component", + "quote": "plus applicable New Jersey state payroll taxes (SUI/FLI/TDI) of roughly $138." + }, + { + "model": "gemini-3.5-flash", + "prediction": 2174, + "category": "included_state_component", + "quote": "plus estimated New Jersey employee state payroll taxes (UI/FLI) of $123.80." + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 2188.22, + "category": "included_state_component", + "quote": "plus NJ mandatory employee SUI/FLI taxes of $138.02" + }, + { + "model": "gemini-3.7-flash", + "prediction": 2189.52, + "category": "included_state_component", + "quote": "plus New Jersey state payroll taxes (FLI/SUI/WF) total $2,189.52" + }, + { + "model": "gemini-3.8-flash", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "glm-5.3", + "prediction": 2050.2, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or NJ employee state payroll tax modeled" + }, + { + "model": "gpt-5.4-mini", + "prediction": 4097, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no additional Medicare tax or mandatory state employee payroll tax is indicated" + }, + { + "model": "gpt-5.4-nano", + "prediction": 1525, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 2302.79, + "category": "included_state_component", + "quote": "NJ employee UI/WF/SWF, TDI, and FLI contributions at 0.9425%*26800=252.59" + }, + { + "model": "gpt-5.6-luna", + "prediction": 2303.46, + "category": "included_state_component", + "quote": "$253.26 of NJ employee payroll contributions" + }, + { + "model": "gpt-5.6-sol", + "prediction": 2314.18, + "category": "included_state_component", + "quote": "estimated mandatory New Jersey employee payroll contributions are 263.98." + }, + { + "model": "gpt-5.6-terra", + "prediction": 2252.54, + "category": "included_state_component", + "quote": "estimated mandatory NJ employee payroll contributions are $202.34" + }, + { + "model": "gpt-6-astra", + "prediction": 2276.66, + "category": "included_state_component", + "quote": "estimated mandatory NJ unemployment, workforce, disability and family-leave contributions total 226.46" + }, + { + "model": "gpt-6-luna", + "prediction": 2314.18, + "category": "included_state_component", + "quote": "estimated mandatory NJ employee payroll contributions are $263.98." + }, + { + "model": "gpt-6-sol", + "prediction": 2276.66, + "category": "included_state_component", + "quote": "approximately 226.46 of mandatory New Jersey employee contributions" + }, + { + "model": "gpt-6.1-sol", + "prediction": 2265.27, + "category": "included_state_component", + "quote": "estimated NJ employee unemployment/workforce, disability, and family-leave contributions total 215.07" + }, + { + "model": "grok-4.3", + "prediction": 2050, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 2098, + "category": "included_state_component", + "quote": "Plus NJ mandatory employee TDI + FLI at combined 0.18% = $48.24." + }, + { + "model": "grok-4.6", + "prediction": 2228, + "category": "included_state_component", + "quote": "NJ mandatory employee UI (0.3825%) plus TDI/FLI add about $178." + }, + { + "model": "grok-4.7", + "prediction": 2314.18, + "category": "included_state_component", + "quote": "New Jersey mandatory employee UI/workforce 0.425%, TDI 0.23%, and FLI 0.33% total 0.985% of wages, or $263.98." + }, + { + "model": "grok-build-0.1", + "prediction": 2050, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no additional Medicare or state payroll taxes." + }, + { + "model": "inkling", + "prediction": 2177, + "category": "included_state_component", + "quote": "NJ mandatory employee taxes (UI ~0.3825% and FLI ~0.09%) add about $127." + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 2308.15, + "category": "included_state_component", + "quote": "New Jersey employee UI/workforce, temporary disability, and family-leave contributions add 257.95" + }, + { + "model": "minimax-m3", + "prediction": 2050, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 2050.2, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory employee NJ payroll tax modeled." + }, + { + "model": "qwen-3.7-max", + "prediction": 2152.71, + "category": "included_state_component", + "quote": "So NJ employee payroll = 0.3825% \u00d7 $26,800 = $102.51." + }, + { + "model": "qwen3.8-max", + "prediction": 2050.2, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_022", + "counts": { + "included_state_component": 29, + "excluded_as_optional_or_employer_paid": 2, + "excluded_as_no_state_payroll_tax": 4, + "excluded_silently": 9, + "other_or_unparsed": 2 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 4557.61, + "category": "included_state_component", + "quote": "CA SDI employee tax at 1.2% (no wage cap in 2026) = $631.62" + }, + { + "model": "claude-fable-5.1", + "prediction": 4026.58, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 4028.27, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No mandatory California state payroll taxes apply." + }, + { + "model": "claude-opus-4.7", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "CA SDI 2026 rate 1.2% = $631.62 (no wage cap in CA SDI as of 2024+)." + }, + { + "model": "claude-opus-4.8", + "prediction": 4026.58, + "category": "excluded_as_optional_or_employer_paid", + "quote": "CA has no mandatory employee payroll tax (SDI applies but is excluded as not required here per instructions to count Social Security and Medicare)." + }, + { + "model": "claude-opus-5", + "prediction": 4026, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CA has no mandatory employee income-based payroll tax modeled here." + }, + { + "model": "claude-opus-5.5", + "prediction": 4711, + "category": "included_state_component", + "quote": "CA SDI is about 1.3% \u00d7 52,635 = 684." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 4605.57, + "category": "included_state_component", + "quote": "California mandatory employee-side payroll tax: SDI (State Disability Insurance) rate for 2026 is 1.1% on all wages (CA removed the wage base cap)" + }, + { + "model": "claude-sonnet-5", + "prediction": 3264.37, + "category": "excluded_as_optional_or_employer_paid", + "quote": "CA has no mandatory employee-side state payroll tax (SDI is voluntary/paid by employer in aggregate but here treated as not applicable as a separate mandatory employee tax" + }, + { + "model": "claude-sonnet-5.5", + "prediction": 4710.84, + "category": "included_state_component", + "quote": "CA SDI at an assumed 1.3% = 684.26." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 4026.61, + "category": "other_or_unparsed", + "quote": "Wages below Social Security wage base and below Additional Medicare threshold, no state payroll tax included." + }, + { + "model": "deepseek-v4-pro", + "prediction": 4605, + "category": "included_state_component", + "quote": "CA SDI tax: $579." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "CA SDI 1.2% $631.62" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "CA SDI 1.2% * 52,635 = 631.62" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 4605.57, + "category": "included_state_component", + "quote": "California SDI tax (1.1% of $52,635 = $578.99, as the SDI cap is removed)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 4027, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 4606, + "category": "included_state_component", + "quote": "and 1.1% CA SDI ($579) on $52,635 of wages." + }, + { + "model": "gemini-3.5-flash", + "prediction": 4606, + "category": "included_state_component", + "quote": "and CA SDI tax of 1.1% ($579) on gross wages of $52,635." + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 4027, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 4500.3, + "category": "included_state_component", + "quote": "CA State Disability Insurance tax (0.9% of $52,635 = $473.72)" + }, + { + "model": "gemini-3.7-flash", + "prediction": 4500.29, + "category": "included_state_component", + "quote": "and California SDI on wages of $52,635." + }, + { + "model": "gemini-3.8-flash", + "prediction": 4026.58, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "CA SDI (mandatory state payroll tax): 1.2% \u00d7 $52,635 = $631.62." + }, + { + "model": "glm-5.3", + "prediction": 4026, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-mini", + "prediction": 4030, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax at this income level and no state employee payroll tax in CA." + }, + { + "model": "gpt-5.4-nano", + "prediction": 3308, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "plus estimated California SDI at 1.2%" + }, + { + "model": "gpt-5.6-luna", + "prediction": 4710.84, + "category": "included_state_component", + "quote": "and California SDI of 1.3% on $52,635 of wages" + }, + { + "model": "gpt-5.6-sol", + "prediction": 4711, + "category": "included_state_component", + "quote": "estimated California SDI withholding is about $684." + }, + { + "model": "gpt-5.6-terra", + "prediction": 4027, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 4710.83, + "category": "included_state_component", + "quote": "1.3% California SDI for 2026." + }, + { + "model": "gpt-6-luna", + "prediction": 4658, + "category": "included_state_component", + "quote": "plus estimated California SDI at 1.2%" + }, + { + "model": "gpt-6-sol", + "prediction": 4711, + "category": "included_state_component", + "quote": "estimated California employee SDI at 1.3% adds 684" + }, + { + "model": "gpt-6.1-sol", + "prediction": 4710.83, + "category": "included_state_component", + "quote": "and 1.3% California SDI." + }, + { + "model": "grok-4.3", + "prediction": 4027, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 4658, + "category": "included_state_component", + "quote": "CA SDI at 1.2%*52635=632" + }, + { + "model": "grok-4.6", + "prediction": 4658, + "category": "included_state_component", + "quote": "Employee-side FICA and mandatory CA SDI on wages of 52,635" + }, + { + "model": "grok-4.7", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "mandatory CA SDI at 1.2% is $631.62." + }, + { + "model": "grok-build-0.1", + "prediction": 4658, + "category": "included_state_component", + "quote": "CA SDI at projected 1.2% rate 52635*0.012=632" + }, + { + "model": "inkling", + "prediction": 4658, + "category": "included_state_component", + "quote": "California SDI at about 1.2% = $632" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 4658.2, + "category": "included_state_component", + "quote": "plus mandatory CA employee SDI at 1.2% with no wage cap = 631.62" + }, + { + "model": "minimax-m3", + "prediction": 4026, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 4658, + "category": "included_state_component", + "quote": "plus mandatory California employee SDI at 1.2% = $632." + }, + { + "model": "qwen-3.7-max", + "prediction": 4605.57, + "category": "included_state_component", + "quote": "CA SDI 2026 rate ~1.1% on wages up to ~$153,164 = $52,635 * 0.011 = $578.99." + }, + { + "model": "qwen3.8-max", + "prediction": 4026.58, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory employee state payroll taxes apply." + } + ] + }, + { + "scenario_id": "scenario_023", + "counts": { + "included_state_component": 30, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 5, + "excluded_silently": 7, + "other_or_unparsed": 4 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 1531.4, + "category": "included_state_component", + "quote": "CA employee SDI 1.13% (2026 rate, no wage cap) = $197.11." + }, + { + "model": "claude-fable-5.1", + "prediction": 1334.39, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 1335.85, + "category": "excluded_as_no_state_payroll_tax", + "quote": "California does not have a mandatory state payroll income tax on wages." + }, + { + "model": "claude-opus-4.7", + "prediction": 1543.71, + "category": "included_state_component", + "quote": "CA SDI 1.2% (2026) = $209.32." + }, + { + "model": "claude-opus-4.8", + "prediction": 1467.32, + "category": "included_state_component", + "quote": "CA SDI 0.9% = $156.99." + }, + { + "model": "claude-opus-5", + "prediction": 1546, + "category": "included_state_component", + "quote": "plus CA SDI 1.2% = $209.32." + }, + { + "model": "claude-opus-5.5", + "prediction": 1561.15, + "category": "included_state_component", + "quote": "California SDI at about 1.3% of $17,443 is $226.76." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 1526.26, + "category": "included_state_component", + "quote": "CA SDI: $17,443 \u00d7 1.1% = $191.87." + }, + { + "model": "claude-sonnet-5", + "prediction": 1334.39, + "category": "excluded_as_no_state_payroll_tax", + "quote": "California has no mandatory employee-side state payroll tax (CA SDI was repealed as a separate wage-base tax structure change is not assumed here" + }, + { + "model": "claude-sonnet-5.5", + "prediction": 1334.39, + "category": "other_or_unparsed", + "quote": "CA SDI is not included." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 716, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No Additional Medicare Tax and no state payroll tax." + }, + { + "model": "deepseek-v4-pro", + "prediction": 827.83, + "category": "included_state_component", + "quote": "CA SDI tax = 1.2% * $9,354 = $112.25." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 1526.26, + "category": "included_state_component", + "quote": "California SDI 1.1% x 17,443 = 191.87" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 1543.71, + "category": "included_state_component", + "quote": "6.2% Social Security + 1.45% Medicare + 1.2% CA SDI" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 1526.26, + "category": "included_state_component", + "quote": "California State Disability Insurance (estimated at 1.1% of $17,443 = $191.87)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 1334.44, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 818, + "category": "included_state_component", + "quote": "FICA (Social Security and Medicare) and California State Disability Insurance (SDI) are calculated on taxable gross wages" + }, + { + "model": "gemini-3.5-flash", + "prediction": 818, + "category": "included_state_component", + "quote": "1.1% CA SDI on FICA-taxable wages of $9,354" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 1595, + "category": "included_state_component", + "quote": "plus California SDI, totaling $1,595." + }, + { + "model": "gemini-3.6-flash", + "prediction": 1526.26, + "category": "included_state_component", + "quote": "1.1% CA SDI tax ($191.87)" + }, + { + "model": "gemini-3.7-flash", + "prediction": 809.12, + "category": "included_state_component", + "quote": "and California State Disability Insurance (SDI) on taxable wages total $809.12." + }, + { + "model": "gemini-3.8-flash", + "prediction": 1526.26, + "category": "included_state_component", + "quote": "plus California SDI at 1.1% ($191.87)" + }, + { + "model": "glm-5.2", + "prediction": 1543.71, + "category": "included_state_component", + "quote": "CA SDI (mandatory employee state payroll tax): 1.2% \u00d7 $17,443 = $209.32." + }, + { + "model": "glm-5.3", + "prediction": 1329, + "category": "excluded_as_no_state_payroll_tax", + "quote": "California has no mandatory employee state payroll tax." + }, + { + "model": "gpt-5.4-mini", + "prediction": 1375.966, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory state payroll tax in California." + }, + { + "model": "gpt-5.4-nano", + "prediction": 1208, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 1543.71, + "category": "included_state_component", + "quote": "plus estimated California SDI at 1.2%" + }, + { + "model": "gpt-5.6-luna", + "prediction": 1561.15, + "category": "included_state_component", + "quote": "estimated California SDI withholding at 1.3% is $226.76" + }, + { + "model": "gpt-5.6-sol", + "prediction": 1561.15, + "category": "included_state_component", + "quote": "estimated 2026 California SDI at 1.3% is $226.76" + }, + { + "model": "gpt-5.6-terra", + "prediction": 1334.39, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 1561.15, + "category": "included_state_component", + "quote": "California SDI adds 1.3%." + }, + { + "model": "gpt-6-luna", + "prediction": 1544, + "category": "included_state_component", + "quote": "California employee SDI at an estimated 1.2% adds about $209" + }, + { + "model": "gpt-6-sol", + "prediction": 1561.15, + "category": "included_state_component", + "quote": "plus California SDI at 1.3%" + }, + { + "model": "gpt-6.1-sol", + "prediction": 837.18, + "category": "included_state_component", + "quote": "plus California SDI at 1.3%, produce the annual payroll tax." + }, + { + "model": "grok-4.3", + "prediction": 1334, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 1544, + "category": "included_state_component", + "quote": "plus CA SDI mandatory employee payroll tax at projected 1.2% = 209.32" + }, + { + "model": "grok-4.6", + "prediction": 1544, + "category": "included_state_component", + "quote": "plus mandatory California SDI at the 1.2% rate ($209.32)" + }, + { + "model": "grok-4.7", + "prediction": 1543.71, + "category": "included_state_component", + "quote": "California SDI at the carried 1.2% rate is 209.32." + }, + { + "model": "grok-build-0.1", + "prediction": 1283, + "category": "included_state_component", + "quote": "CA mandatory employee SDI tax 1.1% = $161." + }, + { + "model": "inkling", + "prediction": 828, + "category": "included_state_component", + "quote": "CA SDI at 1.2% adds $112.25" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "minimax-m3", + "prediction": 1334, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 1544, + "category": "included_state_component", + "quote": "mandatory CA employee SDI 1.2% \u00d7 $17,443 = $209." + }, + { + "model": "qwen-3.7-max", + "prediction": 1334.39, + "category": "other_or_unparsed", + "quote": "California SDI is a mandatory state payroll tax: 1.1% \u00d7 $17,443 = $191.87. Grand total = $1,334.39 + $191.87 = $1,526.26. value = 1334.39" + }, + { + "model": "qwen3.8-max", + "prediction": 261.65, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_028", + "counts": { + "included_state_component": 12, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 17, + "excluded_silently": 15, + "other_or_unparsed": 2 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 5730, + "category": "included_state_component", + "quote": "Total FICA $4,590 plus PA employee SUI $42 = $4,632." + }, + { + "model": "claude-fable-5.1", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA state payroll tax: $0 (PA has no employee-side income tax withheld from wages for state purposes" + }, + { + "model": "claude-opus-4.7", + "prediction": 4632, + "category": "included_state_component", + "quote": "Wait - PA has employee unemployment tax 0.07%. Total = 3720+870+42=4632." + }, + { + "model": "claude-opus-4.8", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee payroll tax counted here." + }, + { + "model": "claude-opus-5", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-opus-5.5", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-sonnet-4.6", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Pennsylvania has no mandatory employee-side state payroll tax (PA has a flat state income tax but no separate employee payroll tax)." + }, + { + "model": "claude-sonnet-5", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Pennsylvania has no mandatory employee-side state payroll tax (no state disability/UI employee contribution)." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No PA employee state payroll tax is modeled." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare (below 200,000) and no PA state employee payroll tax" + }, + { + "model": "deepseek-v4-pro", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "with no Additional Medicare Tax or state payroll tax = $4,590" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 4632, + "category": "included_state_component", + "quote": "the Pennsylvania employee unemployment compensation tax ($60,000 x 0.07% = $42)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 4632, + "category": "included_state_component", + "quote": "Pennsylvania's employee-side SUI tax of 0.07% ($42) on the $60,000 gross wages" + }, + { + "model": "gemini-3.5-flash", + "prediction": 4632, + "category": "included_state_component", + "quote": "Pennsylvania imposes an employee-paid state unemployment compensation tax of 0.07% ($42)" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 4632, + "category": "included_state_component", + "quote": "plus PA employee unemployment compensation tax of 0.07% ($42), totaling $4,632" + }, + { + "model": "gemini-3.7-flash", + "prediction": 4632, + "category": "included_state_component", + "quote": "and 0.07% PA SUI ($42), totaling $4,632" + }, + { + "model": "gemini-3.8-flash", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "glm-5.3", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee state payroll tax" + }, + { + "model": "gpt-5.4-mini", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory state payroll tax applies in PA" + }, + { + "model": "gpt-5.4-nano", + "prediction": 4600, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory PA employee payroll tax included" + }, + { + "model": "gpt-5.6-luna", + "prediction": 4597, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment tax of 7" + }, + { + "model": "gpt-5.6-sol", + "prediction": 4632, + "category": "included_state_component", + "quote": "plus an estimated $42 Pennsylvania employee unemployment contribution" + }, + { + "model": "gpt-5.6-terra", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 4632, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment contributions at 0.07% are $42" + }, + { + "model": "gpt-6-luna", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory employee state payroll tax is specified." + }, + { + "model": "gpt-6-sol", + "prediction": 4632, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment contributions are 0.07%, or 42" + }, + { + "model": "gpt-6.1-sol", + "prediction": 4632, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment contributions are $42" + }, + { + "model": "grok-4.3", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax and no PA employee payroll tax." + }, + { + "model": "grok-4.6", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee state payroll tax." + }, + { + "model": "grok-4.7", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Pennsylvania has no mandatory employee state payroll tax." + }, + { + "model": "grok-build-0.1", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "inkling", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no PolicyEngine-modeled employee state payroll tax applies in PA." + }, + { + "model": "minimax-m3", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee state payroll tax." + }, + { + "model": "ox-alpha", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "qwen-3.7-max", + "prediction": 4590, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee-side state payroll tax (no state disability/unemployment deduction from employees beyond income tax withholding" + }, + { + "model": "qwen3.8-max", + "prediction": 4590, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_032", + "counts": { + "included_state_component": 7, + "excluded_as_optional_or_employer_paid": 1, + "excluded_as_no_state_payroll_tax": 19, + "excluded_silently": 16, + "other_or_unparsed": 3 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Minnesota has no mandatory employee state payroll tax in 2026" + }, + { + "model": "claude-fable-5.1", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax and no modeled mandatory MN employee payroll tax" + }, + { + "model": "claude-haiku-4.5", + "prediction": 2175.9, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Minnesota has no mandatory state payroll tax on employees." + }, + { + "model": "claude-opus-4.7", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "MN has no mandatory employee state payroll tax." + }, + { + "model": "claude-opus-4.8", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "MN has no mandatory employee payroll tax." + }, + { + "model": "claude-opus-5", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax and no mandatory MN employee payroll tax" + }, + { + "model": "claude-opus-5.5", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-sonnet-4.6", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Minnesota has no mandatory employee-side state payroll tax (MN does not impose a separate employee payroll tax beyond federal)" + }, + { + "model": "claude-sonnet-5", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Minnesota has no mandatory state payroll tax (no state disability or paid family leave employee contribution in 2026 for this scenario)" + }, + { + "model": "claude-sonnet-5.5", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "I assumed no other mandatory employee state payroll tax is modeled for MN." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No Additional Medicare Tax and no employee state payroll tax." + }, + { + "model": "deepseek-v4-pro", + "prediction": 2320, + "category": "included_state_component", + "quote": "MN PFML employee share 0.35%=$101.50" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 2320, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no employee state payroll tax applies, for a total of $2,218.50. value = 2218.5" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "MN Paid Leave employee share 0.44% * $29,000 = $127.60" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 2219, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.5-flash", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.7-flash", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.8-flash", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "glm-5.3", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-mini", + "prediction": 4428.6, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no additional Medicare tax or mandatory state employee payroll tax in Minnesota" + }, + { + "model": "gpt-5.4-nano", + "prediction": 1520, + "category": "other_or_unparsed", + "quote": "mandatory MN employee payroll tax is included only if applicable as state payroll taxes are not specified, so using standard FICA only" + }, + { + "model": "gpt-5.5", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "Minnesota paid-leave employee premium 127.6" + }, + { + "model": "gpt-5.6-luna", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "with no additional or state payroll tax" + }, + { + "model": "gpt-5.6-sol", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "the estimated employee half of Minnesota's 0.88% Paid Leave premium is $127.60" + }, + { + "model": "gpt-5.6-terra", + "prediction": 2219, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "Minnesota employee paid-leave premiums at 0.44% add $127.60" + }, + { + "model": "gpt-6-luna", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no other employee-side payroll tax is indicated" + }, + { + "model": "gpt-6-sol", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6.1-sol", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "plus $127.60 for Minnesota's 0.44% employee paid-leave contribution" + }, + { + "model": "grok-4.3", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory Minnesota employee payroll tax applies" + }, + { + "model": "grok-4.6", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No Additional Medicare Tax and no Minnesota employee-side payroll tax." + }, + { + "model": "grok-4.7", + "prediction": 2218.5, + "category": "excluded_as_optional_or_employer_paid", + "quote": "Minnesota has no mandatory employee state payroll tax (paid-leave withholding is optional)" + }, + { + "model": "grok-build-0.1", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "inkling", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or state employee payroll taxes apply" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 2346.1, + "category": "included_state_component", + "quote": "The 50% employee share of Minnesota's 0.88% 2026 paid-leave premium adds $127.60" + }, + { + "model": "minimax-m3", + "prediction": 2219, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 2218.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Minnesota levies no mandatory employee payroll tax" + }, + { + "model": "qwen-3.7-max", + "prediction": 2219, + "category": "excluded_as_no_state_payroll_tax", + "quote": "MN has no mandatory employee state payroll tax (no state disability insurance tax like CA/NY)" + }, + { + "model": "qwen3.8-max", + "prediction": 2218.5, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_043", + "counts": { + "included_state_component": 18, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 14, + "excluded_silently": 11, + "other_or_unparsed": 3 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 315.14, + "category": "included_state_component", + "quote": "Including the mandatory CO FAMLI employee premium of 0.45%: total = 331.14. Submitting FICA plus FAMLI \u2248 331.14... value = 315.14" + }, + { + "model": "claude-fable-5.1", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no modeled mandatory Colorado employee payroll tax" + }, + { + "model": "claude-haiku-4.5", + "prediction": 313, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No mandatory state payroll taxes apply in Colorado." + }, + { + "model": "claude-opus-4.7", + "prediction": 312.73, + "category": "other_or_unparsed", + "quote": "Actually CO FAMLI employee = 0.45%. But the standard payroll calc here: SS 253.46 + Medicare 59.28 = 312.74. Excluding FAMLI as state UI-like." + }, + { + "model": "claude-opus-4.8", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado has no mandatory employee payroll tax applicable here" + }, + { + "model": "claude-opus-5", + "prediction": 313, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-opus-5.5", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-sonnet-4.6", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado has no mandatory employee-side state payroll tax." + }, + { + "model": "claude-sonnet-5", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado has no mandatory employee-side state payroll tax (no state disability/UI employee contribution) applicable here." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 330.72, + "category": "included_state_component", + "quote": "Colorado FAMLI employee share 0.44% = 17.99" + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado has no employee state payroll tax" + }, + { + "model": "deepseek-v4-pro", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado FAMLI employee share 0.45% = $18.40" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 337.26, + "category": "included_state_component", + "quote": "Colorado's mandatory employee FAMLI payroll premium ($24.53)" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 330.72, + "category": "included_state_component", + "quote": "Colorado FAMLI employee share 0.44% = $17.99" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "the taxable wage base for FICA and CO FAMLI taxes is $0, resulting in no payroll tax liability" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado FAMLI premiums (0.45% of $4,088 = $18.40)" + }, + { + "model": "gemini-3.5-flash", + "prediction": 312.74, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 331.13, + "category": "included_state_component", + "quote": "0.45% Colorado FAMLI employee tax ($18.40)" + }, + { + "model": "gemini-3.7-flash", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado FAMLI employee contributions (0.45% of $4,088 = $18.40)" + }, + { + "model": "gemini-3.8-flash", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado FAMLI employee premium = $4,088 \u00d7 0.45% = $18.396." + }, + { + "model": "glm-5.3", + "prediction": 313, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-mini", + "prediction": 312, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or state employee payroll tax applies in Colorado" + }, + { + "model": "gpt-5.4-nano", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "the provided wages are treated as not subject/insufficient for the modeled employee-side components in this output setup" + }, + { + "model": "gpt-5.5", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax and no mandatory Colorado employee payroll tax included" + }, + { + "model": "gpt-5.6-luna", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado employee FAMLI payroll premiums at 0.45% are $18.40" + }, + { + "model": "gpt-5.6-sol", + "prediction": 331.13, + "category": "included_state_component", + "quote": "the estimated Colorado FAMLI employee premium at 0.45% is $18.40" + }, + { + "model": "gpt-5.6-terra", + "prediction": 312.732, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 330.72, + "category": "included_state_component", + "quote": "plus Colorado FAMLI employee contributions at 0.44% of wages" + }, + { + "model": "gpt-6-luna", + "prediction": 330.72, + "category": "included_state_component", + "quote": "including Colorado's estimated 0.44% employee FAMLI premium gives $330.72" + }, + { + "model": "gpt-6-sol", + "prediction": 330.72, + "category": "included_state_component", + "quote": "Colorado FAMLI contributions at 0.44% are $17.99" + }, + { + "model": "gpt-6.1-sol", + "prediction": 330.72, + "category": "included_state_component", + "quote": "Colorado's 2026 employee FAMLI contribution of 0.44% adds $17.99" + }, + { + "model": "grok-4.3", + "prediction": 313, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or Colorado employee payroll tax" + }, + { + "model": "grok-4.6", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No Additional Medicare Tax and no Colorado employee payroll tax." + }, + { + "model": "grok-4.7", + "prediction": 330.72, + "category": "included_state_component", + "quote": "Colorado FAMLI employee share is 0.44% (half of the 0.88% 2026 premium)" + }, + { + "model": "grok-build-0.1", + "prediction": 313, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No Additional Medicare Tax or mandatory state payroll taxes apply." + }, + { + "model": "inkling", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "kimi-k2.6", + "prediction": 331.13, + "category": "included_state_component", + "quote": "Colorado Family and Medical Leave Insurance employee tax is 0.45% of $4,088 ($18.396)" + }, + { + "model": "kimi-k3", + "prediction": 331.13, + "category": "included_state_component", + "quote": "the Colorado FAMLI employee share at 0.45% adds $18.40" + }, + { + "model": "minimax-m3", + "prediction": 313, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado has no mandatory employee state payroll tax." + }, + { + "model": "ox-alpha", + "prediction": 312.73, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "qwen-3.7-max", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado does not have a mandatory employee state payroll tax." + }, + { + "model": "qwen3.8-max", + "prediction": 312.73, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Colorado does not impose a mandatory employee payroll income tax." + } + ] + }, + { + "scenario_id": "scenario_081", + "counts": { + "included_state_component": 22, + "excluded_as_optional_or_employer_paid": 1, + "excluded_as_no_state_payroll_tax": 7, + "excluded_silently": 9, + "other_or_unparsed": 7 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 13268.7, + "category": "other_or_unparsed", + "quote": "PolicyEngine-style employee payroll tax typically counts SS + Medicare only when state payroll tax not modeled" + }, + { + "model": "claude-fable-5.1", + "prediction": 13387.65, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 13383.41, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Massachusetts does not have a separate mandatory employee payroll tax beyond federal taxes." + }, + { + "model": "claude-opus-4.7", + "prediction": 12321.73, + "category": "other_or_unparsed", + "quote": "employee pays ~0.18% family + 0% medical for small employers. Estimating MA PFML employee contribution ~$0." + }, + { + "model": "claude-opus-4.8", + "prediction": 12717.62, + "category": "other_or_unparsed", + "quote": "MA has mandatory employee Paid Family and Medical Leave contributions, but those are not modeled here as 0 absent rate." + }, + { + "model": "claude-opus-5", + "prediction": 13385, + "category": "excluded_as_no_state_payroll_tax", + "quote": "MA has no mandatory employee payroll tax counted here beyond PFML: ~0." + }, + { + "model": "claude-opus-5.5", + "prediction": 13387.65, + "category": "other_or_unparsed", + "quote": "The MA Paid Family and Medical Leave (PFML) employee contribution (about $805 at 0.46%) is excluded on the assumption the model does not include it." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 14444.66, + "category": "included_state_component", + "quote": "MA PFML employee share = $175,002 \u00d7 0.00604 = $1,057.01; Total = $14,444.66." + }, + { + "model": "claude-sonnet-5", + "prediction": 12318.65, + "category": "excluded_as_optional_or_employer_paid", + "quote": "Massachusetts has no mandatory employee-side state payroll tax (no state disability/paid leave employee contribution assumed here beyond employer-side PFML, which is excluded)." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 13387.65, + "category": "other_or_unparsed", + "quote": "I did not include any MA paid family and medical leave contribution." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 13387.65, + "category": "excluded_as_no_state_payroll_tax", + "quote": "MA has no employee state payroll tax" + }, + { + "model": "deepseek-v4-pro", + "prediction": 14053, + "category": "included_state_component", + "quote": "full SS (6.2%), Medicare (1.45%), and MA PFML employee share (0.38%) apply" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 13593.41, + "category": "included_state_component", + "quote": "MA PFML employee contribution is $656.26 (0.375%), for a total of $14,043.91" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 14157.66, + "category": "included_state_component", + "quote": "MA PFML employee share = 175002 * 0.0044 = 770.01; total = 14157.66" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 13944.16, + "category": "included_state_component", + "quote": "the Massachusetts Paid Family and Medical Leave employee contribution ($556.51, estimated at 0.318% of wages)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 11843, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 13402, + "category": "included_state_component", + "quote": "plus estimated MA PFML tax" + }, + { + "model": "gemini-3.5-flash", + "prediction": 13224, + "category": "included_state_component", + "quote": "Massachusetts Paid Family and Medical Leave contribution of approximately 0.24% ($402)" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 14133, + "category": "included_state_component", + "quote": "plus MA PFML tax of $765.01" + }, + { + "model": "gemini-3.6-flash", + "prediction": 12822.4, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.7-flash", + "prediction": 13444, + "category": "included_state_component", + "quote": "plus MA PFML employee contributions" + }, + { + "model": "gemini-3.8-flash", + "prediction": 13387.65, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 14319, + "category": "included_state_component", + "quote": "MA PFML employee contribution \u2248 0.532% \u00d7 $175,002 = $931" + }, + { + "model": "glm-5.3", + "prediction": 13387.65, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Massachusetts mandatory employee payroll tax" + }, + { + "model": "gpt-5.4-mini", + "prediction": 13389.37, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-nano", + "prediction": 5420, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 14192.66, + "category": "included_state_component", + "quote": "plus estimated MA PFML employee contributions at 0.46% of wages" + }, + { + "model": "gpt-5.6-luna", + "prediction": 14193, + "category": "included_state_component", + "quote": "Massachusetts PFML employee payroll tax approximately 805" + }, + { + "model": "gpt-5.6-sol", + "prediction": 14192.66, + "category": "included_state_component", + "quote": "Massachusetts PFML employee contributions of 0.46%" + }, + { + "model": "gpt-5.6-terra", + "prediction": 14350.16, + "category": "included_state_component", + "quote": "plus estimated Massachusetts employee PFML withholding of $962.51" + }, + { + "model": "gpt-6-astra", + "prediction": 14192.66, + "category": "included_state_component", + "quote": "Massachusetts employee paid-leave contributions at 0.46% are $805.0092" + }, + { + "model": "gpt-6-luna", + "prediction": 13388, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-sol", + "prediction": 14192.6622, + "category": "included_state_component", + "quote": "Massachusetts employee paid-family-and-medical-leave contributions at 0.46% add 805.0092" + }, + { + "model": "gpt-6.1-sol", + "prediction": 14192.66, + "category": "included_state_component", + "quote": "$805.01 Massachusetts employee paid-family-and-medical-leave contributions" + }, + { + "model": "grok-4.3", + "prediction": 15926, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 13944, + "category": "included_state_component", + "quote": "MA PFML mandatory employee contribution at 0.318% is 556" + }, + { + "model": "grok-4.6", + "prediction": 14193, + "category": "included_state_component", + "quote": "Mandatory Massachusetts PFML employee tax is 0.0046 * 175002 = 805.01" + }, + { + "model": "grok-4.7", + "prediction": 14192.66, + "category": "included_state_component", + "quote": "Mandatory MA PFML employee contribution at 0.46% is $805.01" + }, + { + "model": "grok-build-0.1", + "prediction": 13388, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no state payroll tax" + }, + { + "model": "inkling", + "prediction": 14193, + "category": "included_state_component", + "quote": "Massachusetts mandatory PFML employee contribution is about 0.46% (~$805)" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 13835.66, + "category": "included_state_component", + "quote": "the 2026 Massachusetts employee PFML rate of 0.256% adds 448.01" + }, + { + "model": "minimax-m3", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "value=0" + }, + { + "model": "ox-alpha", + "prediction": 13387.65, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Massachusetts levies no mandatory employee payroll tax modeled here." + }, + { + "model": "qwen-3.7-max", + "prediction": 14525.24, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No mandatory state payroll taxes apply in Massachusetts beyond standard income tax withholding which is not a payroll tax." + }, + { + "model": "qwen3.8-max", + "prediction": 13922, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_082", + "counts": { + "included_state_component": 22, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 6, + "excluded_silently": 12, + "other_or_unparsed": 6 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 7664.92, + "category": "other_or_unparsed", + "quote": "NY has no mandatory employee state payroll tax modeled beyond negligible PFL/SDI, which is excluded here." + }, + { + "model": "claude-fable-5.1", + "prediction": 7664.92, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 8304.92, + "category": "included_state_component", + "quote": "NY state mandatory payroll tax (Paid Family Leave Insurance) is approximately 0.62% of wages up to the state wage base: $100,195 \u00d7 0.0062 = $621.21." + }, + { + "model": "claude-opus-4.7", + "prediction": 7745.43, + "category": "included_state_component", + "quote": "Using SDI $31.20 + PFL ~$49.31 \u2248 $80.51." + }, + { + "model": "claude-opus-4.8", + "prediction": 7664.92, + "category": "excluded_as_no_state_payroll_tax", + "quote": "NY has no mandatory employee payroll tax." + }, + { + "model": "claude-opus-5", + "prediction": 7665, + "category": "excluded_as_no_state_payroll_tax", + "quote": "NY does not impose a mandatory employee payroll tax counted here." + }, + { + "model": "claude-opus-5.5", + "prediction": 7664.92, + "category": "other_or_unparsed", + "quote": "no mandatory NY employee payroll tax is modeled." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 8042.77, + "category": "included_state_component", + "quote": "Total: $6,212.09 + $1,452.83 + $31.20 + $346.65 = $8,042.77." + }, + { + "model": "claude-sonnet-5", + "prediction": 7794.31, + "category": "included_state_component", + "quote": "Adding NY SDI/PFL employee contribution estimate (~$129.39, capped small amount per NY 2026 rates) brings total to approximately $7,794.31." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 7664.92, + "category": "other_or_unparsed", + "quote": "No NY payroll tax is modeled." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 7651, + "category": "excluded_as_no_state_payroll_tax", + "quote": "there are no mandatory employee state payroll taxes in NY" + }, + { + "model": "deepseek-v4-pro", + "prediction": 7696.12, + "category": "included_state_component", + "quote": "NY SDI contribution $31.20. Total $7,696.12." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 8034.6, + "category": "included_state_component", + "quote": "New York Paid Family Leave maximum annual employee contribution is $369.68" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 8128.96, + "category": "included_state_component", + "quote": "NY PFL $432.84 + NY DBL $31.20 = $8,128.96" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 8029, + "category": "included_state_component", + "quote": "approximately $333 for New York Paid Family Leave, and $31 for New York Disability Insurance" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 7665, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 6368.33, + "category": "included_state_component", + "quote": "plus NY Paid Family Leave ($294.62) and NY SDI ($31.20)" + }, + { + "model": "gemini-3.5-flash", + "prediction": 6436.5, + "category": "included_state_component", + "quote": "plus approximately $394 for NY State Paid Family Leave" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 7664.92, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 6313, + "category": "included_state_component", + "quote": "plus state disability and paid family leave contributions" + }, + { + "model": "gemini-3.7-flash", + "prediction": 6376, + "category": "included_state_component", + "quote": "plus NY state mandatory payroll deductions (SDI/PFL)" + }, + { + "model": "gemini-3.8-flash", + "prediction": 7664.92, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 8069.38, + "category": "included_state_component", + "quote": "Total = $6,212.09 + $1,452.83 + $31.20 + $373.26 = $8,069.38." + }, + { + "model": "glm-5.3", + "prediction": 7765, + "category": "excluded_as_no_state_payroll_tax", + "quote": "NY has no employee-side state payroll tax" + }, + { + "model": "gpt-5.4-mini", + "prediction": 7660, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory employee state payroll tax in NY" + }, + { + "model": "gpt-5.4-nano", + "prediction": 7659, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 8050.65, + "category": "included_state_component", + "quote": "adding estimated NY employee PFL/DBL contributions of $385.73" + }, + { + "model": "gpt-5.6-luna", + "prediction": 7696.12, + "category": "included_state_component", + "quote": "New York employee disability payroll tax is approximately $31.20" + }, + { + "model": "gpt-5.6-sol", + "prediction": 7664.92, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.6-terra", + "prediction": 7654, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 8128.96, + "category": "included_state_component", + "quote": "estimated New York paid-family-leave contributions are $432.84 and disability contributions are $31.20" + }, + { + "model": "gpt-6-luna", + "prediction": 8108.03, + "category": "included_state_component", + "quote": "adding estimated 2026 NY paid-family-leave contributions capped at $411.91 and NY disability contributions of $31.20" + }, + { + "model": "gpt-6-sol", + "prediction": 8108.03, + "category": "included_state_component", + "quote": "estimated mandatory NY paid-family-leave and disability contributions add 411.91 and 31.20" + }, + { + "model": "gpt-6.1-sol", + "prediction": 8108.03, + "category": "included_state_component", + "quote": "$411.91 New York paid-family-leave contributions and $31.20 disability contributions" + }, + { + "model": "grok-4.3", + "prediction": 7665, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 6042, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no NY employee payroll tax" + }, + { + "model": "grok-4.6", + "prediction": 8032, + "category": "included_state_component", + "quote": "NY Paid Family Leave is a mandatory employee payroll tax of about 367 on the 2026 PFL wage base." + }, + { + "model": "grok-4.7", + "prediction": 7665, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-build-0.1", + "prediction": 6029, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "inkling", + "prediction": 8061, + "category": "included_state_component", + "quote": "NY mandatory employee payroll taxes (SDI capped ~$31 and PFL ~$365) add ~$396." + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 8107.98, + "category": "included_state_component", + "quote": "New York Paid Family Leave is capped at $411.86 and employee disability insurance is $0.60 for 52 weeks, or $31.20." + }, + { + "model": "minimax-m3", + "prediction": 7665, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 7665, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "qwen-3.7-max", + "prediction": 7464.33, + "category": "other_or_unparsed", + "quote": "Using final calculation: $6,212.09 + $1,452.83 + NY mandatory ~$-200 = value = 7464.33" + }, + { + "model": "qwen3.8-max", + "prediction": 7752.42, + "category": "other_or_unparsed", + "quote": "No mandatory employee state payroll tax is included." + } + ] + }, + { + "scenario_id": "scenario_099", + "counts": { + "included_state_component": 29, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 2, + "excluded_silently": 10, + "other_or_unparsed": 5 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 14118, + "category": "included_state_component", + "quote": "CA SDI employee tax 1.2% (no wage cap in 2026): $85,000*0.012 = $1,020 plus $80,000*0.012 = $960, total $1,980." + }, + { + "model": "claude-fable-5.1", + "prediction": 12622.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 12622.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "California has no state-mandated employee payroll tax for income tax withholding (payroll taxes exclude state income tax withholding as per definition)." + }, + { + "model": "claude-opus-4.7", + "prediction": 13197.5, + "category": "included_state_component", + "quote": "CA SDI 1.2% on $165,000 = $1,980." + }, + { + "model": "claude-opus-4.8", + "prediction": 12622.5, + "category": "other_or_unparsed", + "quote": "However payroll_tax here focuses on federal FICA per typical benchmark; combining SS+Medicare = 5270+1232.50+4960+1160 = 12622.50." + }, + { + "model": "claude-opus-5", + "prediction": 12622, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-opus-5.5", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "CA SDI of $2,145 assumes a 1.3% rate on $165,000." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 14437.5, + "category": "included_state_component", + "quote": "CA SDI applies to all wages with no wage base cap at 1.1% rate" + }, + { + "model": "claude-sonnet-5", + "prediction": 12903.5, + "category": "other_or_unparsed", + "quote": "CA has no mandatory employee-side state payroll tax (SDI is voluntary/not modeled here as mandatory in this calc)... Using standard SS/Medicare only: 12622.50. Final combined value = 12903.50" + }, + { + "model": "claude-sonnet-5.5", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "CA SDI at an assumed 1.3% of $165,000 = $2,145." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 14438, + "category": "included_state_component", + "quote": "plus CA SDI ~1.1% on $165,000 = $1,815" + }, + { + "model": "deepseek-v4-pro", + "prediction": 12285, + "category": "included_state_component", + "quote": "CA SDI = 1.1% \u00d7 $140,403 = $1,544." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 12285.26, + "category": "included_state_component", + "quote": "CA SDI 1.1% ($1,544.43)" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "CA SDI 1.2% on $165,000 = $1,980." + }, + { + "model": "gemini-3-flash-preview", + "prediction": 14437.5, + "category": "included_state_component", + "quote": "California State Disability Insurance ($1,815, which is 1.1% of $165,000)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 12622.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 12285.26, + "category": "included_state_component", + "quote": "California SDI is 1.1% of $140,403, which is $1,544.43." + }, + { + "model": "gemini-3.5-flash", + "prediction": 14437.5, + "category": "included_state_component", + "quote": "1.1% CA State Disability Insurance ($1,815)" + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 12963, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 13280, + "category": "included_state_component", + "quote": "Employee payroll taxes (Social Security, Medicare, and California SDI) total $13,280." + }, + { + "model": "gemini-3.7-flash", + "prediction": 12425.67, + "category": "included_state_component", + "quote": "Medicare, and California State Disability Insurance (SDI) taxes on taxable wages." + }, + { + "model": "gemini-3.8-flash", + "prediction": 14103, + "category": "included_state_component", + "quote": "comprising Social Security, Medicare, and California SDI equal 14103" + }, + { + "model": "glm-5.2", + "prediction": 12632.4, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.3", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "plus mandatory CA SDI at 1.2% with no wage cap = $1,980" + }, + { + "model": "gpt-5.4-mini", + "prediction": 12622, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.4-nano", + "prediction": 13577, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "plus estimated California SDI at 1.2% of wages ($1,980)" + }, + { + "model": "gpt-5.6-luna", + "prediction": 14312.4, + "category": "included_state_component", + "quote": "plus approximately 1.2% California SDI" + }, + { + "model": "gpt-5.6-sol", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "estimated California SDI is 1980" + }, + { + "model": "gpt-5.6-terra", + "prediction": 14603, + "category": "included_state_component", + "quote": "California SDI is about $1,980." + }, + { + "model": "gpt-6-astra", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "California SDI equals $165000 \u00d7 1.3%." + }, + { + "model": "gpt-6-luna", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "estimated California SDI withholding at 1.3% adds $2,145" + }, + { + "model": "gpt-6-sol", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "estimated California SDI withholding at 1.3% adds 2145" + }, + { + "model": "gpt-6.1-sol", + "prediction": 14767.5, + "category": "included_state_component", + "quote": "California SDI at 1.3% adds $2,145." + }, + { + "model": "grok-4.3", + "prediction": 12623, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 14438, + "category": "included_state_component", + "quote": "plus CA SDI 1.1% on 165000" + }, + { + "model": "grok-4.6", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "Mandatory CA SDI at 1.2% on $165,000 adds $1,980." + }, + { + "model": "grok-4.7", + "prediction": 14602.5, + "category": "included_state_component", + "quote": "California SDI at 1.2% with no wage cap adds $1,980." + }, + { + "model": "grok-build-0.1", + "prediction": 10741, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "inkling", + "prediction": 14603, + "category": "included_state_component", + "quote": "CA SDI 1.2% ($1,980)" + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "minimax-m3", + "prediction": 25290, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "ox-alpha", + "prediction": 12622.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no modeled employee-side state payroll tax applies." + }, + { + "model": "qwen-3.7-max", + "prediction": 12622.5, + "category": "other_or_unparsed", + "quote": "Total = $12,622.50 + $1,815 = $14,437.50. Wait, let me recalculate without SDI first and then add. Actually the base calculation gives $12,622.50. value = 12622.5" + }, + { + "model": "qwen3.8-max", + "prediction": 13094.7, + "category": "included_state_component", + "quote": "California State Disability Insurance is $472.20 at 1.2% of $85,000 for the head." + } + ] + }, + { + "scenario_id": "scenario_120", + "counts": { + "included_state_component": 20, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 11, + "excluded_silently": 9, + "other_or_unparsed": 6 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 13736.63, + "category": "included_state_component", + "quote": "CT mandatory employee payroll tax: Paid Family and Medical Leave = 0.5% x 165,597 = 827.99 (below SS wage cap)." + }, + { + "model": "claude-fable-5.1", + "prediction": 12668.17, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 12619.41, + "category": "other_or_unparsed", + "quote": "CT payroll tax (approximated at -$48.79 net from itemizations) = $12,619.41." + }, + { + "model": "claude-opus-4.7", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "CT mandatory employee payroll tax: CT Paid Family and Medical Leave 0.5% on wages up to SS base: 165597 \u00d7 0.5% = $827.99." + }, + { + "model": "claude-opus-4.8", + "prediction": 12668.17, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CT has no mandatory employee payroll tax included here besides standard." + }, + { + "model": "claude-opus-5", + "prediction": 10680, + "category": "other_or_unparsed", + "quote": "CT has a mandatory employee paid-leave payroll tax of 0.5% (828). Total \u2248 10,680 after accounting for PolicyEngine's employee payroll tax computation." + }, + { + "model": "claude-opus-5.5", + "prediction": 12668, + "category": "other_or_unparsed", + "quote": "The CT paid-leave contribution is not included, on the assumption that PolicyEngine does not model it." + }, + { + "model": "claude-sonnet-4.6", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "CT PFMLA employee contribution = $165,597 \u00d7 0.5% = $827.99." + }, + { + "model": "claude-sonnet-5", + "prediction": 12668.17, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Connecticut has no mandatory employee-side state payroll tax (no state disability/UI employee withholding)." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 12668.17, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No state payroll tax was counted." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 12054, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Connecticut has no employee payroll tax." + }, + { + "model": "deepseek-v4-pro", + "prediction": 12891, + "category": "included_state_component", + "quote": "CT Paid Family Leave (0.5%) is $790.84." + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 13496, + "category": "included_state_component", + "quote": "Connecticut Paid Family and Medical Leave $827.99 on $165,597 of wages" + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 12668.17, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 13496, + "category": "included_state_component", + "quote": "$828 for the mandatory Connecticut Paid Leave premium (0.5% of $165,597)." + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 14197, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 12891, + "category": "included_state_component", + "quote": "plus Connecticut's 0.5% Paid Family and Medical Leave (PFML) tax." + }, + { + "model": "gemini-3.5-flash", + "prediction": 12890.7, + "category": "included_state_component", + "quote": "and CT PFML is 0.5% ($790.84), totaling $12,890.70." + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 14352.19, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 12890.7, + "category": "included_state_component", + "quote": "and 0.5% CT Paid Family Leave ($790.84) on FICA wages of $158,168." + }, + { + "model": "gemini-3.7-flash", + "prediction": 12891, + "category": "included_state_component", + "quote": "Connecticut paid family leave payroll tax on covered wages" + }, + { + "model": "gemini-3.8-flash", + "prediction": 12891.1, + "category": "included_state_component", + "quote": "plus Connecticut Paid Family and Medical Leave (0.5% up to the Social Security taxable maximum)" + }, + { + "model": "glm-5.2", + "prediction": 12044.31, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.3", + "prediction": 12668, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CT has no employee payroll tax." + }, + { + "model": "gpt-5.4-mini", + "prediction": 10087, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Connecticut has no mandatory employee state payroll tax" + }, + { + "model": "gpt-5.4-nano", + "prediction": 6180, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-5.5", + "prediction": 13496, + "category": "included_state_component", + "quote": "and Connecticut paid-leave tax of 0.5% on $165,597 of wages" + }, + { + "model": "gpt-5.6-luna", + "prediction": 13590.67, + "category": "included_state_component", + "quote": "Connecticut paid-family-leave withholding of $922.50" + }, + { + "model": "gpt-5.6-sol", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "Connecticut paid-leave contributions estimated at 0.5%" + }, + { + "model": "gpt-5.6-terra", + "prediction": 12621, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 12890.69, + "category": "included_state_component", + "quote": "and 0.5% Connecticut paid-leave contributions." + }, + { + "model": "gpt-6-luna", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "Connecticut paid-leave contribution of 0.5% on $165,597 wages" + }, + { + "model": "gpt-6-sol", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "Connecticut paid-leave contributions are approximately 827.99" + }, + { + "model": "gpt-6.1-sol", + "prediction": 12890.69, + "category": "included_state_component", + "quote": "Connecticut paid-leave contributions are $790.84" + }, + { + "model": "grok-4.3", + "prediction": 12668, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 12668, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no employee state payroll tax in CT." + }, + { + "model": "grok-4.6", + "prediction": 13496, + "category": "included_state_component", + "quote": "CT PFML 0.5% adds 828." + }, + { + "model": "grok-4.7", + "prediction": 13496.16, + "category": "included_state_component", + "quote": "Connecticut Paid Leave employee tax is $827.99 (0.5%)." + }, + { + "model": "grok-build-0.1", + "prediction": 12668, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no other employee payroll taxes." + }, + { + "model": "inkling", + "prediction": 12668, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CT has no mandatory employee state payroll taxes." + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "minimax-m3", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "value = 0" + }, + { + "model": "ox-alpha", + "prediction": 12668, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CT has no mandatory employee payroll tax." + }, + { + "model": "qwen-3.7-max", + "prediction": 12668.18, + "category": "excluded_as_no_state_payroll_tax", + "quote": "CT has no mandatory employee state payroll tax." + }, + { + "model": "qwen3.8-max", + "prediction": 9655, + "category": "excluded_silently", + "quote": "" + } + ] + }, + { + "scenario_id": "scenario_123", + "counts": { + "included_state_component": 15, + "excluded_as_optional_or_employer_paid": 0, + "excluded_as_no_state_payroll_tax": 16, + "excluded_silently": 11, + "other_or_unparsed": 4 + }, + "per_model": [ + { + "model": "claude-fable-5", + "prediction": 11663.35, + "category": "included_state_component", + "quote": "PA SUI employee withholding 0.07% of wages = $70 + $31.50 = $101.50" + }, + { + "model": "claude-fable-5.1", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-haiku-4.5", + "prediction": 11741.39, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Pennsylvania has no state income tax and no mandatory state payroll tax." + }, + { + "model": "claude-opus-4.7", + "prediction": 11092.5, + "category": "other_or_unparsed", + "quote": "PA has no employee-side mandatory state payroll tax beyond minimal SUI; using $11,092.50." + }, + { + "model": "claude-opus-4.8", + "prediction": 11097.5, + "category": "included_state_component", + "quote": "Employee payroll tax = Social Security (6.2%) + Medicare (1.45%) + PA SUI employee tax (0.07% in 2026)." + }, + { + "model": "claude-opus-5", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-opus-5.5", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "claude-sonnet-4.6", + "prediction": 11106.5, + "category": "included_state_component", + "quote": "PA employees do pay into the UC fund at 0.07% on first $10,000. Head: $7, Child 1: $7 = $14 total." + }, + { + "model": "claude-sonnet-5", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee-side state payroll tax (no state disability/UI employee withholding)." + }, + { + "model": "claude-sonnet-5.5", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No employee state payroll tax is modeled." + }, + { + "model": "deepseek-v4-flash-0731", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "deepseek-v4-pro", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "deepseek-v4-pro-0813", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or mandatory state payroll tax applies." + }, + { + "model": "deepseek-v4.1-flash", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3-flash-preview", + "prediction": 11194, + "category": "included_state_component", + "quote": "plus Pennsylvania's mandatory employee unemployment compensation tax estimated at 0.07% of gross wages ($145,000 * 0.0007 = $101.50)" + }, + { + "model": "gemini-3.1-flash-lite-preview", + "prediction": 11108, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.1-pro-preview", + "prediction": 11194, + "category": "included_state_component", + "quote": "plus Pennsylvania's 0.07% employee state unemployment tax ($101.50)." + }, + { + "model": "gemini-3.5-flash", + "prediction": 11194, + "category": "included_state_component", + "quote": "PA employee unemployment compensation tax is approximately 0.07% of wages ($101.50)." + }, + { + "model": "gemini-3.5-flash-lite", + "prediction": 11138, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gemini-3.6-flash", + "prediction": 11194, + "category": "included_state_component", + "quote": "and PA employee UC tax of 0.07% ($101.50) on $145,000 gross wages" + }, + { + "model": "gemini-3.7-flash", + "prediction": 11194, + "category": "included_state_component", + "quote": "PA employee SUI tax (0.07% of $145,000 = $101.50)" + }, + { + "model": "gemini-3.8-flash", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "glm-5.2", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee-side state payroll tax." + }, + { + "model": "glm-5.3", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee payroll tax." + }, + { + "model": "gpt-5.4-mini", + "prediction": 12366, + "category": "excluded_as_no_state_payroll_tax", + "quote": "No mandatory employee state payroll tax is assumed in PA" + }, + { + "model": "gpt-5.4-nano", + "prediction": 15399, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no other mandatory employee state payroll taxes provided separately" + }, + { + "model": "gpt-5.5", + "prediction": 11194, + "category": "included_state_component", + "quote": "plus Pennsylvania employee unemployment contribution of 0.07% or $101.50." + }, + { + "model": "gpt-5.6-luna", + "prediction": 11194, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment contributions add $101.50" + }, + { + "model": "gpt-5.6-sol", + "prediction": 11194, + "category": "included_state_component", + "quote": "Pennsylvania's 0.07% employee unemployment contribution adds 101.5" + }, + { + "model": "gpt-5.6-terra", + "prediction": 11092.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "gpt-6-astra", + "prediction": 11179.5, + "category": "included_state_component", + "quote": "and $87 Pennsylvania employee unemployment contributions" + }, + { + "model": "gpt-6-luna", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare Tax or Pennsylvania employee payroll tax applies." + }, + { + "model": "gpt-6-sol", + "prediction": 11194, + "category": "included_state_component", + "quote": "Pennsylvania employee unemployment contributions at 0.07% add 101.50" + }, + { + "model": "gpt-6.1-sol", + "prediction": 11194, + "category": "included_state_component", + "quote": "$101.50 Pennsylvania employee unemployment contributions at 0.07%." + }, + { + "model": "grok-4.3", + "prediction": 11107.5, + "category": "excluded_silently", + "quote": "" + }, + { + "model": "grok-4.5", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no mandatory PA employee state payroll tax." + }, + { + "model": "grok-4.6", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no mandatory employee-side state payroll tax." + }, + { + "model": "grok-4.7", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "Pennsylvania has no mandatory employee state payroll tax." + }, + { + "model": "grok-build-0.1", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "no Additional Medicare (earned income $145000 < $250000 MFJ threshold) or mandatory state employee payroll tax." + }, + { + "model": "inkling", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "State mandatory employee payroll taxes are zero." + }, + { + "model": "kimi-k2.6", + "prediction": null, + "category": "other_or_unparsed", + "quote": "" + }, + { + "model": "kimi-k3", + "prediction": 11194, + "category": "included_state_component", + "quote": "PA employee UC withholding is 101.50 at 0.07%." + }, + { + "model": "minimax-m3", + "prediction": 0, + "category": "other_or_unparsed", + "quote": "however this benchmark output reports 0 for this scenario per the structured household inputs." + }, + { + "model": "ox-alpha", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA levies no mandatory employee payroll tax." + }, + { + "model": "qwen-3.7-max", + "prediction": 11092.5, + "category": "excluded_as_no_state_payroll_tax", + "quote": "PA has no state payroll taxes (employee-side income tax is separate)." + }, + { + "model": "qwen3.8-max", + "prediction": 10851.95, + "category": "other_or_unparsed", + "quote": "rounded to $10,851.95 to match PolicyEngine's state payroll-tax rounding." + } + ] + } + ] +} \ No newline at end of file diff --git a/reference_audit/2026-10-05-payroll/verification/law_research.json b/reference_audit/2026-10-05-payroll/verification/law_research.json new file mode 100644 index 00000000..623fa33a --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/law_research.json @@ -0,0 +1,4794 @@ +{ + "date": "2026-10-05", + "method": "Claude Code workflow wf_cecd10dd-abe: one researcher per program, then two independent reviewers per program (statute lens, administration lens), each trying to refute the classification. Agents used WebFetch and WebSearch and read the installed policyengine-us 2.15.17 source.", + "programs": [ + { + "key": "MN", + "research": { + "program": "Minnesota Paid Leave (family and medical benefit insurance premiums, Minn. Stat. ch. 268B)", + "state": "MN", + "engine_variables": [ + "mn_employee_paid_leave_contribution", + "mn_paid_leave_taxable_wages", + "mn_employee_state_payroll_tax", + "employee_state_payroll_tax (aggregator)", + "parameter gov.states.mn.tax.payroll.paid_leave.employee_rate", + "parameter gov.irs.payroll.social_security.cap" + ], + "premium_liability": "The employer owes the premium. Under 268B.14 subd. 1, premiums \"accrue and become payable by each employer,\" and each employer pays them to the state every quarter. The commissioner assesses the premium against the employer's premium account (268B.13). Under Minn. R. 3317.5000 subp. 5, the employer \"remains liable to the commissioner for premiums.\" If the employer fails to pay, the employee keeps the right to benefits (268B.14 subd. 9). Employees have no direct liability to the state. Their part is a \"charge back\": the employer may deduct it from wages (268B.14 subd. 3).", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "Minn. Stat. 268B.14, subd. 1(a) (2025 Minnesota Statutes)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "Family and medical leave premiums accrue and become payable by each employer, except for an employer with an approved private plan under section 268B.10, for each calendar year on the taxable wages", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 3 (Employee charge back; as amended by Laws 2024 c 127 art 73 s 38, eff. 2026-01-01)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "employers must pay a minimum of 50 percent of the annual premiums paid under this section. Employees, through a deduction in their wages to the employer, must pay the remaining portion, if any, of the premium not paid by the employer.", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 3 (minimum-wage floor on deductions)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "Deductions under this section must not cause an employee's wage, after the deduction, to fall below the rate required to be paid to the employee by any applicable statute, regulation, rule, ordinance, or government resolution or policy", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 5a(b)(1) (small employer rate)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "employers must pay a minimum of 25 percent of the rate calculated in subdivisions 6 and 7. Employers shall not deduct from any employees' pay to fund the employer portion of the premium", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 5a(b)(2) (small employer rate)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "employees must pay the remaining portion due under this subdivision, if any, of the premium not paid by the employer. The employer must make wage deductions as necessary under this subdivision to fund the employee portion of the premium.", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 4 (wages subject to premium)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The maximum wages subject to premium in a calendar year is equal to the maximum earnings in that year subject to the FICA Old-Age, Survivors, and Disability Insurance tax.", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.01, subd. 43 (taxable wages)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.01", + "quote": "equal to the maximum earnings in that year subject to the FICA Old-Age, Survivors, and Disability Insurance tax rounded to the nearest $1,000.", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 6 (statutory employer premium rate)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The employer premium rates beginning January 1, 2026, shall be as follows: (1) for an employer participating in both family and medical benefit programs, 0.7 percent;", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.14, subd. 7 (commissioner's rate adjustment authority; the source of the 0.88% rate)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The commissioner may adjust the annual premium rates pursuant to this section prior to January 1, 2026.", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.10, subd. 2(5) (private plan: cap on employee cost; same clause appears for the family-benefit private plan)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B.10", + "quote": "no greater amount is required to be paid by employees toward the cost of benefits under the employer plan than by this chapter", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.26(b)(2) (employer notice to each employee)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "(2) the amount of premium deductions made by the employer under this chapter;", + "fetched_and_read": true + }, + { + "citation": "Minn. Stat. 268B.27, subd. 2(3) (collective bargaining construction)", + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "limit the parties to a collective bargaining agreement from bargaining and agreeing with respect to leave benefits and related policies and employee protections that meet or exceed, and do not otherwise conflict with, the minimum standards", + "fetched_and_read": true + }, + { + "citation": "Minn. R. 3317.5000, subp. 5 (DEED Paid Leave rules)", + "url": "https://www.revisor.mn.gov/rules/3317/full", + "quote": "An employer remains liable to the commissioner for premiums on wages paid until the effective date of the approved private plan.", + "fetched_and_read": true + }, + { + "citation": "Laws 2024, ch. 127, art. 73, sec. 38 (amendment to 268B.14 subd. 3 adding 'and subject to subdivision 6', effective January 1, 2026)", + "url": "https://www.revisor.mn.gov/laws/2024/0/127/", + "quote": "Notwithstanding section 177.24, subdivision 4, or 181.06, subdivision 1, and subject to subdivision 6, employers must pay a minimum of 50 percent of the annual premiums paid under this section.", + "fetched_and_read": true + }, + { + "citation": "SSA, Cost-of-Living Increase and Other Determinations for 2026, Federal Register 2025-19763 (published 2025-11-03)", + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "quote": "Employers can deduct up to 0.44% of wages from employees", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "quote": "small employers can deduct up to 0.44% of wages from employees", + "fetched_and_read": true + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "quote": "Employers may choose to pay more than their required portion", + "fetched_and_read": true + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "quote": "in 2026, the premium rate will be .88%.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-union-workers-presentation_tcm1045-711054.pdf", + "quote": "Employers can cover more than 0.44% if they chose.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-union-workers-presentation_tcm1045-711054.pdf", + "quote": "cannot be charged more than you would be under the state plan", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-in-person-employers_tcm1045-706805.pdf", + "quote": "Action 1: Decide how to split premiums.", + "fetched_and_read": true + }, + { + "url": "https://content.govdelivery.com/accounts/MNDEED/bulletins/3d3828d", + "quote": "the premium rate will be 0.88 percent", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Yes. Minn. Stat. 268B.14 subd. 3 sets only a floor on the employer's part: \"a minimum of 50 percent.\" Employees pay \"the remaining portion, if any, of the premium not paid by the employer,\" so an employer that pays 100% leaves the employee nothing to pay. For small employers, subd. 5a(b) works the same way, with an employer minimum of 25% of the full rate. DEED says this explicitly in a presentation it gave the Senate Jobs committee on 2025-04-02 (archived by the Legislative Reference Library): \"Employers may choose to pay more than their required portion for some or all employees.\" Its October 2025 union-worker deck says the same (\"Employers can cover more than 0.44% if they chose\"). DEED's September 2025 employer deck tells employers to \"Decide how to split premiums.\" Under 268B.26(b)(2), each employer must tell its employees the amount of premium deductions it makes, which confirms that the amount differs from employer to employer.", + "rate_2026": { + "total_rate": "0.88% of taxable wages (medical 0.61% + family 0.27%). DEED's commissioner set this under 268B.14 subd. 7 ('may adjust the annual premium rates ... prior to January 1, 2026'); it replaces the statutory 0.7% in subd. 6. Small employers (30 or fewer employees and an average wage at or below 150% of the statewide average) pay 75% of that rate, 0.66%, under subd. 5a.", + "max_employee_share": "0.44% of taxable wages: the 0.88% total minus the employer's minimum 50% (subd. 3). For small employers it is also 0.44%: the 0.66% total minus the employer's minimum of 25% x 0.88% = 0.22% (subd. 5a(b)). DEED confirms 'small employers can deduct up to 0.44%'. The minimum employee share is 0%, because the employer may pay everything. Under an approved private plan, employees cannot be charged more than they would pay under the state plan (268B.10 subd. 2(5)).", + "wage_base_or_cap": "About $185,000 for 2026. 268B.01 subd. 43 sets taxable wages up to the Social Security OASDI maximum rounded to the nearest $1,000. SSA's 2026 base is $184,500 (Federal Register, 2025-11-03), and DEED's December 2025 document states the cap as $185,000. 268B.14 subd. 4 states the cap without the rounding, so it would read $184,500. The cap does not bind for any benchmark household under about $184,500.", + "sources": [ + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "published": "2025 Minnesota Statutes edition, published electronically 10/19/2025. Subd. 3 (50% employer minimum) enacted by Laws 2023 c 59 and amended by Laws 2024 c 127 art 73 s 38. Subd. 5a added by Laws 2024 c 127 art 73 s 39. Subd. 7 amended by Laws 2024 c 127 and 1Sp2025 c 6 art 4 s 27. The revisor's Table 2 shows no 2026 Regular Session amendments anywhere in ch. 268B." + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.01", + "published": "2025 Minnesota Statutes. Subd. 43 (taxable wages, rounded to the nearest $1,000) from Laws 2023 c 59 and 2024 c 127." + }, + { + "url": "https://content.govdelivery.com/accounts/MNDEED/bulletins/3d3828d", + "published": "DEED bulletin announcing the 0.88% rate. The page has no dateline. The matching DEED press release (mn.gov/deed/newscenter/press-releases/?id=1045-670064) could not be read because of a bot challenge. KSTP's article on the announcement carries datePublished 2025-02-21 in its metadata (a secondary source, used only for the date)." + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "published": "DEED presentation for the 2025-04-02 Senate Jobs hearing (PDF created 2025-04-01). States the 0.88% rate, the 0.61%/0.27% split, the deduction of up to 50%, and that employers may pay more." + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "published": "DEED, PDF created and modified 2025-12-19. States 0.88% for most employers, 0.66% for small employers, the deduction of up to 0.44% for both, and the $185,000 rounded cap." + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "published": "2025-11-03 (SSA 2026 OASDI contribution and benefit base, $184,500)." + } + ], + "published_before_2026_07_03": "Yes. The statutory split rule (the 50% employer minimum and the employee charge-back) dates from 2023-2024 session laws, published in the 2025 Statutes on 2025-10-19. DEED announced the 0.88% rate around 2025-02-21 and restated it in dated DEED documents from 2025-04-01, 2025-09-12, 2025-10-28 and 2025-12-19. The 2026 OASDI base was published 2025-11-03. No 2026 session change to ch. 268B appears in the revisor's Table 2." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_032", + "engine_amount": 127.6, + "law_amount_at_max_employee_share": 127.6, + "matches_within_1_dollar": true, + "note": "Wages are $29,000, below the cap of about $185,000, so taxable wages are $29,000. Large employer: total premium 0.88% x 29,000 = $255.20; employer minimum 50% = $127.60; maximum employee share 0.44% x 29,000 = $127.60. Small employer: total 0.66% x 29,000 = $191.40; employer minimum 0.22% x 29,000 = $63.80; maximum employee share 191.40 - 63.80 = $127.60. Engine: 0.0044 x 29,000 = $127.60. I ran policyengine-us 2.15.17 this session and it returned mn_employee_paid_leave_contribution = 127.6 and employee_state_payroll_tax = 127.6. The difference from the legal maximum is $0.00. The legal minimum employee share is $0, if the employer elects to pay the whole premium, so the legally possible range for this household is $0 to $127.60. The deduction may also not push pay below the applicable minimum wage; at $29,000 that would bind only at extreme hours, and the prompt gives no hours." + } + ], + "engine_documentation": "Read this session in policyengine_us 2.15.17. In variables/gov/states/mn/tax/payroll/paid_leave/mn_employee_paid_leave_contribution.py, the label is \"Minnesota employee paid leave contribution\", and the documentation reads \"Employee-side Minnesota Paid Leave contribution, assuming the employer withholds the maximum permitted employee share.\" Its formula is the employee_rate parameter times mn_paid_leave_taxable_wages, defined_for StateCode.MN. In mn_paid_leave_taxable_wages.py, the documentation reads \"Minnesota Paid Leave covered wages capped at the Social Security contribution and benefit base, rounded to the nearest $1,000.\" The reference is revisor 268B.01, and the formula is min(max(0, employment_income), 1000*floor(ss_cap/1000 + 0.5)). The parameter parameters/gov/states/mn/tax/payroll/paid_leave/employee_rate.yaml is described as \"Employee-side Minnesota Paid Leave contribution rate\". Its values are 0 from 0000-01-01 and 0.0044 from 2026-01-01. It cites the DEED press release id=1045-670064 and Minn. Stat. 268B.14. In mn_employee_state_payroll_tax.py, the documentation reads \"Employee-side Minnesota Paid Leave contributions.\" It adds mn_employee_paid_leave_contribution. The aggregator employee_state_payroll_tax includes mn_employee_state_payroll_tax, and its documentation reads \"Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules.\" The parameter gov.irs.payroll.social_security.cap is 184,500 for 2026, commented \"Official SSA determination.\" The engine has no input for the employer's chosen split, small-employer status, a private plan, or the minimum-wage floor on deductions. It always applies the 0.44% maximum.", + "caveats": [ + "The aggregator's documentation calls its components 'mandatory' employee payroll taxes. For Minnesota, the component is the largest deduction the law permits, not a mandatory amount. Under a strict reading of the benchmark's 'mandatory employee state payroll taxes', the mandatory Minnesota employee amount is $0, and $127.60 is the legal upper bound.", + "The statute's wording is mixed. Subd. 3 says employees 'must pay the remaining portion, if any', and subd. 5a(b)(2) says the employer 'must make wage deductions as necessary'. But the size of that remainder is set by how much the employer chooses to pay above its minimum. DEED reads it as optional: 'Employers can deduct up to 0.44%' and 'may choose to pay more'.", + "Small employers (30 or fewer employees and an average wage at or below 150% of the statewide average; DEED's 2026 quarterly threshold is $27,745.88) pay 0.66% in total. The maximum employee share is still 0.44%, so employer size does not change the engine's maximum. It changes only the employer's minimum, from 0.44% to 0.22%.", + "Approved private (equivalent) plans under 268B.10: the employer pays no state premium for the program the plan covers. Employees cannot be charged more than under the state plan (at most 0.44% in 2026), and the employer may charge less or nothing. Employers with a private plan for only one program pay the state premium for the other program. Per the statute's structure and DEED's 0.61%/0.27% split, that makes the employee maximum 0.305% (medical) or 0.135% (family). The engine does not model this.", + "Collective bargaining: 268B.27 subd. 2(3) lets bargaining parties agree to benefits that meet or exceed the chapter. Nothing in ch. 268B fixes the employee share for union workers, so a CBA can allocate the 0-0.44% employee share, including employer-paid. The construction-industry CBA waivers (268B.09) cover job protections and health-insurance continuation, not premiums.", + "Not covered: 268B.01 subd. 17 excludes federal employees, self-employed people, independent contractors and seasonal employees as defined there. Self-employed people may opt in under 268B.11, which is a separate premium base.", + "Under subd. 3, deductions may not push an employee's wage below the applicable minimum wage. The prompt gives no hours, but at $29,000 a year this would bind only at implausible hours.", + "Rate provenance: 268B.14 subd. 6 still says 0.7%. The 0.88% comes from the commissioner's adjustment under subd. 7, as DEED announced it. I could not read DEED's own press release page or pl.mn.gov (bot challenge or HTTP 403). The announcement date of about 2025-02-21 rests on KSTP's metadata, corroborated by the dated DEED legislative presentation of 2025-04-01. I did not check whether the rate was also published in the State Register.", + "Wage cap: 268B.14 subd. 4 says the OASDI maximum ($184,500), while 268B.01 subd. 43 and DEED say it is rounded to the nearest $1,000 ($185,000). The engine uses $185,000. This is immaterial for the $29,000 household.", + "The 2027 rate (reported as unchanged at 0.88%) was set under subd. 7 by July 31, 2026. It is not relevant to tax year 2026, and I did not read its primary page (pl.mn.gov returned 403)." + ], + "confidence": "high", + "summary": "Minnesota Paid Leave premiums are owed by the employer (Minn. Stat. 268B.14 subd. 1: premiums 'accrue and become payable by each employer'; Minn. R. 3317.5000 subp. 5). The employee part is a statutory 'employee charge back' (subd. 3). The employer must pay at least 50% of the premium (at least 25% of the full rate for small employers under subd. 5a), and employees pay through wage deductions only 'the remaining portion, if any, of the premium not paid by the employer.' The employer may choose to pay more, up to the whole premium, so the employee share runs from 0% to 0.44% of wages. DEED's guidance says the same: employers 'can deduct up to 0.44%' and 'may choose to pay more than their required portion'. Each employer must tell employees how much it deducts (268B.26(b)(2)), and private plans cannot charge employees more than the state plan would (268B.10). For 2026 the total rate is 0.88%, set by the DEED commissioner under subd. 7 and announced around 2025-02-21; small employers pay 0.66%. The maximum employee share is 0.44% for both sizes, on wages up to the Social Security base rounded to the nearest $1,000 (about $185,000). All of this was published well before 2026-07-03, and the revisor shows no 2026 change to ch. 268B. The engine applies 0.44% unconditionally, and documents that it assumes 'the employer withholds the maximum permitted employee share.' For scenario_032, 0.0044 x $29,000 = $127.60, which equals the legal maximum exactly; I confirmed the figure by running the engine. That amount is an upper bound, not a mandatory one; the legal floor is $0. Classification: optional employer pass-through." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "Family and medical leave premiums accrue and become payable by each employer, except for an employer with an approved private plan under section 268B.10, for each calendar year on the taxable wages that the employer paid to employees in covered employment.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "employers must pay a minimum of 50 percent of the annual premiums paid under this section. Employees, through a deduction in their wages to the employer, must pay the remaining portion, if any, of the premium not paid by the employer.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "(2) employees must pay the remaining portion due under this subdivision, if any, of the premium not paid by the employer. The employer must make wage deductions as necessary under this subdivision to fund the employee portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The failure of an employer to pay premiums does not impact the right of an employee to benefits, or any other right, under this chapter.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "The commissioner must maintain a premium account for each employer. The commissioner must assess the premium account for all the premiums due under section 268B.14", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "the employer must determine the amount of any overpayment attributable to a deduction from employee wages under section 268B.14, subdivision 3, and return any amount erroneously deducted to each affected employee.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "(2) the amount of premium deductions made by the employer under this chapter;", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.10", + "quote": "(5) no greater amount is required to be paid by employees toward the cost of benefits under the employer plan than by this chapter;", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.01", + "quote": "equal to the maximum earnings in that year subject to the FICA Old-Age, Survivors, and Disability Insurance tax rounded to the nearest $1,000.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/rules/3317/full", + "quote": "An employer remains liable to the commissioner for premiums on wages paid until the effective date of the approved private plan.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/laws/2024/0/127/", + "quote": "Notwithstanding section 177.24, subdivision 4, or 181.06, subdivision 1, and subject to subdivision 6, employers must pay a minimum of 50 percent of the annual premiums paid under this section.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/177.27", + "quote": "268B.09, subdivisions 1 to 6, and 268B.14, subdivision 3, with any rule promulgated under section 177.28", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/table2?search_term=268B&year=2026&page_size=2000", + "quote": "Chapter 268B Actions during the 2026 Regular Session ... No Records Found", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14/versions", + "quote": "10/19/2025 * 2025", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "quote": "Employers can deduct up to 0.44% of wages from employees for their portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "quote": "small employers can deduct up to 0.44% of wages from employees for their portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "quote": "Employers may choose to pay more than their required portion for some or all employees.", + "fetched_and_read": true + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "quote": "in 2026, the premium rate will be .88%.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-in-person-employers_tcm1045-706805.pdf", + "quote": "If an employer pays more than the minimum share of the premium, they can deduct this additional contribution as an ordinary and necessary business expense.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-in-person-employers_tcm1045-706805.pdf", + "quote": "Action 1: Decide how to split premiums.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-union-workers-presentation_tcm1045-711054.pdf", + "quote": "Employers can cover more than 0.44% if they chose.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-union-workers-presentation_tcm1045-711054.pdf", + "quote": "You cannot be charged more than you would be under the state plan (up to 0.44% of your wages in 2026).", + "fetched_and_read": true + }, + { + "url": "https://content.govdelivery.com/accounts/MNDEED/bulletins/3d3828d", + "quote": "Employers may deduct the employee portion of Paid Leave premiums from paychecks starting January 1, 2026, when benefits become available.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None that change the scenario_032 amount. I re-ran policyengine-us 2.15.17 from the triage venv for a single MN earner with $29,000 wages in 2026. It returned mn_paid_leave_taxable_wages = 29000, mn_employee_paid_leave_contribution = 127.6 and employee_state_payroll_tax = 127.6, which equals 0.0044 x 29,000, the statutory maximum employee share. The legal floor is $0, because the employer may pay the whole premium.", + "The 0.88% total rate is not in statute: 268B.14 subd. 6 still reads 0.7%, which would cap the employee share at 0.35%. 0.88% rests on the commissioner's subd. 7 adjustment authority ('may adjust ... prior to January 1, 2026'). I found it only in DEED materials: the undated govdelivery bulletin, the Senate deck (PDF created 2025-04-01), the employer deck (created 2025-09-12), the union deck (created 2025-10-28) and the small-employer PDF (created 2025-12-19). I did not find a State Register notice. All of these predate 2026-07-03.", + "Wage cap: 268B.14 subd. 4 ties the cap to the unrounded OASDI maximum ($184,500 for 2026), while 268B.01 subd. 43 and DEED round it to $185,000. The engine uses the rounded $185,000. This is immaterial below about $184,500 of wages, so it does not affect scenario_032.", + "Small employers: subd. 5a sets 75% of the 0.88% rate (0.66%) with an employer minimum of 25% of the full rate (0.22%), so the maximum employee share is again 0.44%. The engine's maximum does not depend on employer size, but the employer can still elect to pay more." + ], + "citation_problems": [ + "No quote in the research was found to be wrong. I checked every operative_text and agency_guidance quote verbatim against the fetched source. Several span line breaks or alt-text in the PDFs, but the words match.", + "The Laws 2024 c 127 art 73 s 38 quote appears in the session law only as original text plus the inserted words 'and subject to subdivision 6,' marked as new text. The rendered combined text matches the research verbatim.", + "The engine parameter cites DEED press release mn.gov/deed/newscenter/press-releases/?id=1045-670064. I could not read it: curl hit a Radware captcha and WebFetch got only the headline. I did not bypass the challenge. The govdelivery bulletin has no dateline. So the announcement date of about 2025-02-21 (from KSTP metadata) remains unconfirmed by a primary source. The earliest dated primary document I read that states 0.88% is the LRL-archived Senate deck (PDF created 2025-04-01).", + "The research's 'Table 2 shows no 2026 amendments' claim is correct, but only with the right query. revisor.mn.gov/statutes/table2?full=268B returns 'No Records Found' even for all years, so that form is useless. The working query is ?search_term=268B&year=2026. It returns no records for 268B, 268B.14, 268B.01 or 268B.10, while the 2026 table is populated for other chapters (e.g. 290, 268, 181)." + ], + "reasoning": "Statute lens. I fetched the full 268B.14 section, the full ch. 268B text, 268B.01, 268B.10, Minn. R. ch. 3317, Laws 2024 c 127, and 177.27. I also searched the whole chapter for deduct, withhold, charge, contribut, 'employee shall/must' and liability language.\n\n1. Liability for the premium is entirely the employer's. Under 268B.14 subd. 1(a), premiums are 'payable by each employer', and each employer pays them quarterly. Under 268B.13, the commissioner assesses each employer's premium account. Under subd. 9, an employer's nonpayment does not affect the employee's rights. Under 268B.15 subd. 7(d), an employer that receives a refund must return any 'erroneously deducted' amounts to employees. Minn. R. 3317.5000 subp. 5 says the employer 'remains liable to the commissioner for premiums'. Nothing in ch. 268B makes the employee liable to the state, and the state has no collection path against employees for premiums.\n\n2. The only employee-share text is the 'Employee charge back' in subd. 3 and the parallel subd. 5a(b)(2). Both use 'must' language: employees 'must pay the remaining portion, if any, of the premium not paid by the employer', and the employer 'must make wage deductions as necessary'. This is the strongest argument for refutation. But the size of the remainder is set by the employer's own election above a floor: 'a minimum of 50 percent' (25% of the full rate for small employers). 'If any' expressly contemplates a zero remainder. So the 'must' only stops an employer from deducting more than the remainder, or from taking the employer's own portion out of pay (subd. 5a(b)(1): 'Employers shall not deduct ... to fund the employer portion'). It does not require any positive employee amount.\n\n3. The surrounding provisions support this reading:\n- 268B.26(b)(2) requires employers to tell employees 'the amount of premium deductions made by the employer', so the amount varies by employer.\n- 268B.10 subd. 2(5) and 3(5) only cap what private-plan employees can be charged.\n- 177.27 subd. 4 lets DLI order compliance with 268B.14 subd. 3. That is wage-deduction protection for employees, not a withholding mandate.\n\n4. DEED's own primary materials consistently describe the share as optional:\n- 'Employers can deduct up to 0.44%' (December 2025).\n- 'Employers may deduct the employee portion' (govdelivery bulletin).\n- 'Employers may choose to pay more than their required portion for some or all employees' (Senate deck, April 2025, and the September 2025 employer deck).\n- 'Employers can cover more than 0.44% if they chose' (union deck, October 2025).\n- 'Action 1: Decide how to split premiums.'\n- The employer deck also treats any amount above the minimum as an 'employer pickup' that is deductible as a business expense.\n\nOne DEED line says the reduced small-employer premium 'does not change the amount owed by employees'. In context that refers to the 0.44% maximum, not a mandatory amount.\n\nThis fits category (b) exactly: the premium is the employer's liability, and the employer may, but need not, deduct up to 0.44% of taxable wages. Calling it mixed or unclear would overweight the conditional 'must', which binds only after the employer has chosen its own share.\n\nEngine. I read the engine source this session. The mn_employee_paid_leave_contribution documentation says it assumes 'the employer withholds the maximum permitted employee share'. The formula is employee_rate (0.0044 from 2026-01-01) times mn_paid_leave_taxable_wages, capped at the SS base rounded to $1,000. The aggregator employee_state_payroll_tax includes mn_employee_state_payroll_tax and is documented as 'mandatory'. The engine figure of $127.60 for scenario_032 is therefore the legal upper bound, not a mandatory amount.\n\nTiming. The statutory split rule (2023 and 2024 session laws) appears in the 2025 Statutes, published electronically on 10/19/2025, and the revisor's Table 2 shows no 2026 change to ch. 268B. The rate documents and the SSA base (Federal Register, 2025-11-03) all predate 2026-07-03. The research's quotes, citations and engine description all checked out; its only soft spot is the unconfirmed press-release date, which it disclosed. The classification stands." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "Family and medical leave premiums accrue and become payable by each employer, except for an employer with an approved private plan under section 268B.10, for each calendar year on the taxable wages that the employer paid to employees in covered employment.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "Employees, through a deduction in their wages to the employer, must pay the remaining portion, if any, of the premium not paid by the employer.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "employees must pay the remaining portion due under this subdivision, if any, of the premium not paid by the employer. The employer must make wage deductions as necessary under this subdivision to fund the employee portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The failure of an employer to pay premiums does not impact the right of an employee to benefits, or any other right, under this chapter.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B.14", + "quote": "The commissioner may adjust the annual premium rates pursuant to this section prior to January 1, 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "The commissioner must maintain a premium account for each employer. The commissioner must assess the premium account for all the premiums due under section 268B.14", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "the employer must determine the amount of any overpayment attributable to a deduction from employee wages under section 268B.14, subdivision 3, and return any amount erroneously deducted to each affected employee.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/cite/268B/full", + "quote": "equal to the maximum earnings in that year subject to the FICA Old-Age, Survivors, and Disability Insurance tax rounded to the nearest $1,000.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/rules/3317/full", + "quote": "An employer remains liable to the commissioner for premiums on wages paid until the effective date of the approved private plan.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/laws/2024/0/127/", + "quote": "Sec. 38. ... and subject to subdivision 6, ... EFFECTIVE DATE. This section is effective January 1, 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.revisor.mn.gov/statutes/table2?search_term=268B&year=2026", + "quote": "Chapter 268B Actions during the 2026 Regular Session ... No Records Found", + "fetched_and_read": true + }, + { + "url": "https://pl.mn.gov/employers/taxes-and-paid-leave", + "quote": "If an employer pays more than the required minimum share of the premium, this is additional compensation to the employee and is included in the employee's federal gross income as wages.", + "fetched_and_read": true + }, + { + "url": "https://pl.mn.gov/employers/roles-and-responsibilities", + "quote": "Regardless of the size of your organization, you can deduct a maximum of .44% of an employee's wages to fund their portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://pl.mn.gov/resources/common-questions", + "quote": "Employers can collect up to 0.44 percent from employees. Employers may choose to pay more than their required share.", + "fetched_and_read": true + }, + { + "url": "https://pl.mn.gov/resources/calculators/premium-rate-and-contributions", + "quote": "Employers are required to contribute 50% of premiums (33% for small employers), but can optionally choose to cover a higher percentage of the premium.", + "fetched_and_read": true + }, + { + "url": "https://pl.mn.gov/employers/small-employers", + "quote": "Employers can collect up to 0.44% of the premium from employees. This means small employers may pay as little as .22% of employee wages for their share of the premium.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-small-employer-premium-rate-designation-acc_tcm1045-716947.pdf", + "quote": "rounded to the nearest thousand dollars (currently $185,000). Employers can deduct up to 0.44% of wages from employees for their portion of the premium.", + "fetched_and_read": true + }, + { + "url": "https://www.lrl.mn.gov/archive/minutes/senate/2025/jobs/20250402/Jobs_20250402_PFML-Update-Hearing-2025-Paid-Leave-Update-Senate-presentation.pdf", + "quote": "Starting January 1, 2026, employers can deduct up to 50% of premiums from employee paychecks. Employers may choose to pay more than their required portion for some or all employees.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-in-person-employers_tcm1045-706805.pdf", + "quote": "Action 1: Decide how to split premiums.", + "fetched_and_read": true + }, + { + "url": "https://mn.gov/deed/assets/paid-leave-union-workers-presentation_tcm1045-711054.pdf", + "quote": "Employers can cover more than 0.44% if they chose.", + "fetched_and_read": true + }, + { + "url": "https://content.govdelivery.com/accounts/MNDEED/bulletins/3d3828d", + "quote": "Employers may deduct the employee portion of Paid Leave premiums from paychecks starting January 1, 2026, when benefits become available.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/mn/tax/payroll/paid_leave/mn_employee_paid_leave_contribution.py", + "quote": "Employee-side Minnesota Paid Leave contribution, assuming the employer withholds the maximum permitted employee share.", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None that change scenario_032. I ran policyengine-us 2.15.17 this session for a single Minnesota earner with $29,000 wages in 2026. It returned mn_paid_leave_taxable_wages = 29000, mn_employee_paid_leave_contribution = 127.6, mn_employee_state_payroll_tax = 127.6 and employee_state_payroll_tax = 127.6. By hand, 0.0044 x 29,000 = $127.60. Large employer: 0.88% x 29,000 = $255.20 total, with an employer minimum of 50%, which is $127.60. Small employer: 0.66% x 29,000 = $191.40 total, with an employer minimum of 25% x 0.88% x 29,000 = $63.80, leaving at most $127.60 for the employee. The engine's figure is the legal maximum. The legal range is $0 to $127.60, and the mandatory employee amount is $0.", + "The 0.88% rate for 2026 appears in no statute or rule. 268B.14 subd. 6 still reads 0.7%. The 0.88% comes from the commissioner's adjustment authority in subd. 7 and is published only in DEED communications. I found no State Register notice. The GovDelivery bulletin has no date. The earliest dated primary document I read that states 0.88% is DEED's Senate Jobs deck, a PDF created 2025-04-01.", + "There is a wage-cap conflict. 268B.14 subd. 4 sets the cap at the OASDI maximum without rounding, which is $184,500. 268B.01 subd. 43 and DEED round it to the nearest $1,000, giving $185,000. The engine computes 1000*floor(184500/1000+0.5) = $185,000. This does not matter for a $29,000 household.", + "Several older DEED decks show earlier caps: $176,100 in the 2025-04-01 Senate deck and $176,000 in the September and October 2025 decks. Only the 2025-12-19 PDF and the current FAQ say $185,000. This is not material here.", + "DEED's current web pages were modified after the 2026-07-03 cutoff. The common-questions page was modified 2026-09-30, the premium-rate page 2026-08-31, the roles page 2026-08-19 and the small-employers page 2026-09-30. Their present text therefore does not by itself prove publication before the cutoff. The same may-deduct-up-to-0.44% content does appear in DEED documents dated before the cutoff: PDFs from 2025-04-01, 2025-09-12, 2025-10-28 and 2025-12-19, and the taxes page modified 2026-05-08. The Wayback CDX returned HTTP 429, so I could not check archived captures." + ], + "citation_problems": [ + "I could not read the DEED press release mn.gov/deed/newscenter/press-releases/?id=1045-670064, which the engine's employee_rate.yaml cites. The page is behind a ShieldSquare/perfdrive bot challenge, and WebFetch returned only the headline. A search-result summary says it described the premium as 'split evenly between the employee and the employer'. I could not confirm that wording, and I did not rely on it.", + "The GovDelivery bulletin (content.govdelivery.com/accounts/MNDEED/bulletins/3d3828d) has an empty dateline. The announcement date of about 2025-02-21 rests on secondary sources: the researcher's KSTP metadata, and a Minnesota Reformer URL dated 2025/02/21 that I saw only in search results and did not read.", + "Every quote in the research that I checked matches the primary text verbatim. That covers 268B.14 subds. 1, 3, 4, 5a(b)(1)-(2), 6 and 7; 268B.01 subd. 43; 268B.10 subd. 2(5); 268B.26(b)(2); the 268B.27 collective-bargaining clause; Minn. R. 3317.5000 subp. 5; Laws 2024 c 127 art 73 s 38; the SSA Federal Register notice; and the four DEED PDFs and the bulletin. The union-deck quote comes from a PDF whose layout scrambles the text, but the sentence appears intact.", + "The research also says the 268B.27 clause sits in subdivision 2. I confirmed the clause text in the full chapter, but not its subdivision number." + ], + "reasoning": "Through the administration lens the sources support the classification.\n\n1) Liability. The statute places the premium on the employer. 268B.14 subd. 1 says premiums \"accrue and become payable by each employer.\" Under 268B.13 the commissioner assesses each employer's premium account. Interest, liens and levies under 268B.15 and 268B.23 run against the employer. Minn. R. 3317.5000 subp. 5 says the employer \"remains liable to the commissioner for premiums.\" Nothing in ch. 268B or Minn. R. ch. 3317 makes an employee liable to the state for premiums that were not deducted. An applicant owes the state only for benefit overpayments under 268B.185. Subd. 9 keeps the employee's rights intact if the employer fails to pay. 268B.15 subd. 7(d) requires the employer to refund wrongly deducted amounts to employees, which treats the deduction as the employer's charge-back mechanism.\n\n2) May or must. The administrative rules (Minn. R. ch. 3317, published 2025-07-09 at 49 SR 1368) say nothing about how the premium is split. The only related text is a private-plan notice item describing \"the process to calculate and collect employee contributions.\" Every DEED source I read uses permissive language. The bulletin says \"Employers may deduct the employee portion.\" The 2025-12-19 PDF says \"Employers can deduct up to 0.44%.\" The roles page says \"you can deduct a maximum of .44%.\" The FAQ says \"can collect up to 0.44 percent... may choose to pay more.\" The calculator page says employers \"can optionally choose to cover a higher percentage.\" The September 2025 employer deck tells employers to \"Decide how to split premiums.\"\n\n3) Can the employer choose to pay it? Yes. Subd. 3 fixes only an employer floor of 50%, or 25% of the full rate for small employers under subd. 5a. The employee pays \"the remaining portion, if any, of the premium not paid by the employer.\" The \"must\" in subd. 3 and in subd. 5a(b)(2) (\"must make wage deductions as necessary\") applies only to whatever the employer has chosen not to pay. It does not set a fixed employee amount. The employer must notify each employee of \"the amount of premium deductions made by the employer\" (268B.26(b)(2)), which assumes the amount varies by employer. This matches the benchmark's option (b): the premium is the employer's liability, and the employer may, but need not, deduct up to a share.\n\n4) Rate, share and cap. The total is 0.88%, split 0.61% medical and 0.27% family, set under the subd. 7 adjustment authority. Small employers pay 0.66%. The maximum employee share is 0.44% for both sizes, which I recomputed from the statute and confirmed in DEED documents. The cap is the OASDI base of $184,500 (Federal Register, 2025-11-03), rounded to $185,000 under 268B.01 subd. 43. The statutory split rule took effect 2026-01-01 (Laws 2024 c 127 art 73 s 38) and appears in the 2025 Statutes, whose PDF was created 2025-10-19. The revisor's Table 2 lists no 2026 Regular Session actions for ch. 268B, though it does list 2026 actions for ch. 268. All of this was published before 2026-07-03.\n\n5) Engine. I read the source and ran it. It applies a flat 0.0044 rate to wages capped at $185,000, and its documentation says it assumes \"the employer withholds the maximum permitted employee share.\" The aggregator's documentation calls its components \"mandatory\" employee payroll taxes, but for Minnesota the component is the legal maximum, not a mandatory amount. For scenario_032 the engine's $127.60 equals the legal maximum exactly. The legally possible range is $0 to $127.60. The deduction also cannot push pay below the applicable minimum wage; that limit binds only if the hourly wage is within about 0.44% of the minimum wage, and the prompt gives no hours.\n\nNuance that does not change the result. DEED's taxes page (modified 2026-05-08) says an employer that pays more than its minimum share is giving the employee additional compensation, taxable as wages. It also says the employee may deduct the full amount \"that the employer pays on their behalf (up to .44% of wages)\" as a state income tax. So for federal tax purposes, an employer pickup is still treated as the employee's 0.44%, paid through extra wages. That is how the federal government characterizes a choice the employer makes. It does not create any state-law duty to deduct, so the share remains an optional employer pass-through.\n\nAll key quotes were confirmed. The one gap in the rate's provenance is that I could not read the bot-blocked press release, and the dated DEED PDFs cover that gap. Not refuted." + } + ] + }, + { + "key": "CO", + "research": { + "program": "Colorado Paid Family and Medical Leave Insurance (FAMLI) premiums", + "state": "CO", + "engine_variables": [ + "co_employee_famli_contribution", + "co_famli_taxable_wages", + "co_employee_state_payroll_tax", + "employee_state_payroll_tax", + "co_employer_famli_contribution (employer side, for contrast)", + "parameter gov.states.co.tax.payroll.famli.employee_rate", + "parameter gov.states.co.tax.payroll.famli.employer_rate", + "parameter gov.states.co.tax.payroll.famli.employer_headcount_threshold", + "parameter gov.irs.payroll.social_security.cap" + ], + "premium_liability": "The employer. C.R.S. 8-13.3-507(2) says the employer \"shall remit\" premiums to the fund for each employee. Under 507(5), an employer with 10 or more employees must remit 100% of the premium, and one with fewer than 10 must remit 50%. The employee has no statutory duty to pay the state on covered employment. The statute's only employee payment duty is for people who opt in under 8-13.3-514: self-employed individuals and employees of local governments that opted out (507(4), 507(9)(b)). The purposes clause, 8-13.3-502(4)(a), speaks of collecting premiums \"from employers and employees\", but it is a findings clause, not an operative rule. Under 507(7), premiums are \"fees and not taxes\".", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "C.R.S. 8-13.3-507(2) (Colorado Revised Statutes 2025, OLLS uncertified printout, Title 8 p. 221)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "beginning on January 1, 2023, for each employee, an employer shall remit to the fund established under section 8-13.3-518 premiums in the form and manner determined by the division.", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(5), first sentence (employers with 10 or more employees)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with 10 or more employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 100 percent of the premium", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(5), second sentence (employers with fewer than 10 employees)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with fewer than 10 employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 50 percent of the premium", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(3)(b), as amended by SB 25-144 (the PDF text breaks 'eighty-eight' across a line)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "From January 1, 2026, through December 31, 2026, the premium amount is eighty-eight hundredths of one percent of wages per employee.", + "fetched_and_read": true + }, + { + "citation": "SB 25-144, Session Laws 2025 ch. 293, section 2 (signed act; Approved: May 30, 2025)", + "url": "https://leg.colorado.gov/laws/session-laws/SB25-144/293/download", + "quote": "FROM JANUARY 1, 2026, THROUGH DECEMBER 31, 2026, THE PREMIUM AMOUNT IS EIGHTY -EIGHT HUNDREDTHS OF ONE PERCENT OF WAGES PER EMPLOYEE .", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(6) (wage cap)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Premiums shall not be required for employees' wages above the contribution and benefit base limit established annually by the federal social security administration", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(7)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Premiums established under this section are fees and not taxes.", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(8) (approved private plans)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with an approved private plan under section 8-13.3-521 shall not be required to remit premiums under this section to the fund.", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-507(9)(b) (local government opt-out)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employee of the local government is not required to pay the premiums imposed in this section unless the employee has elected coverage pursuant to section 8-13.3-514.", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-521(1)(k) (private plan cost to employees)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "the cost to employees covered by a private plan shall not be greater than the cost charged to employees under the state plan under section 8-13.3-507.", + "fetched_and_read": true + }, + { + "citation": "C.R.S. 8-13.3-503(8)(c)", + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "\"Employer\" does not include the federal government.", + "fetched_and_read": true + }, + { + "citation": "7 CCR 1107-1 section 1.4(6) (current version effective 01/01/2026; adopted 10/31/2025; Colorado Register 12/10/2025)", + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "An employer required to remit premiums pursuant to C.R.S. \u00a7 8-13.3-507 may not deduct more than the maximum allowable employee share of the premium from wages paid for a pay period.", + "fetched_and_read": true + }, + { + "citation": "7 CCR 1107-1 section 1.4(6)(A) (effective 01/01/2026)", + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay that portion of the employee share", + "fetched_and_read": true + }, + { + "citation": "7 CCR 1107-1 section 1.4 (original rule effective 01/01/2022; defines the shares)", + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=9914&fileName=7+CCR+1107-1", + "quote": "\"Employee share\" is defined as 50 percent of the premium required for an employee by section 8-13.3-507 (3) C.R.S.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://famli.colorado.gov/employers", + "quote": "The premiums are set to 0.88% of the employee's wage, with 0.44% paid by the employer and 0.44% paid by the employee. Some employers may choose not to deduct any premiums contributions from their employees' wages.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Employers cannot collect missed premiums from employees in later pay periods. It is important to know employees are never required to pay more than 50% of the total premium.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Businesses with nine or fewer employees should submit wage reports and send in 0.44% of their employees wages each quarter.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If you have 10 or more employees (nationwide), you may deduct up to 50% of the 0.88% premium as a standard payroll deduction.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If you have less than 10 employees (nationwide), you are not required to pay the employer share.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/update-your-employee-headcount-for-2026-premiums", + "quote": "Option to deduct up to 0.44% from employee wages toward your total premium amount", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/another-national-first-paid-leave-for-neonatal-intensive-care-unit-parents", + "quote": "The measure also lowers premiums for all Colorado workers and employers, from 0.9% of wages to 0.88% of wages, starting in 2026.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Yes. C.R.S. 8-13.3-507(5) says the employer \"may deduct up to 50 percent\" of the premium from wages, while the employer stays liable to remit the full premium (or 50% for employers with fewer than 10 employees). Under 7 CCR 1107-1 \u00a71.4(6)(A), in force since 01/01/2026, an employer that does not deduct the maximum \"is considered to have elected to pay\" that part of the employee share. It cannot recover the amount from later paychecks, except when the employee's wages were too low to cover it. The FAMLI Division's employer page agrees: \"Some employers may choose not to deduct any premiums contributions.\"", + "rate_2026": { + "total_rate": "0.88% of wages per employee, for January 1 through December 31, 2026. Source: C.R.S. 8-13.3-507(3)(b), as amended by SB 25-144 (Session Laws 2025, ch. 293).", + "max_employee_share": "0.44% of wages, which is 50% of the 0.88% premium (C.R.S. 8-13.3-507(5); 'employee share' is defined as 50% in 7 CCR 1107-1). The maximum is the same for employers of any size. The employer may deduct less, down to $0.", + "wage_base_or_cap": "Premiums apply only up to the Social Security contribution and benefit base (C.R.S. 8-13.3-507(6); 7 CCR 1107-1 \u00a71.4(10)). For 2026 that base is $184,500, per the SSA notice in the Federal Register, 90 FR, doc. 2025-19763, published 2025-11-03. The maximum employee share is therefore $811.80 (0.0044 x 184,500).", + "sources": [ + { + "url": "https://leg.colorado.gov/laws/session-laws/SB25-144/293/download", + "published": "SB 25-144 signed act, Approved May 30, 2025; effective August 6, 2025 (bill page https://leg.colorado.gov/bills/sb25-144: Governor signed 2025-05-30, Session Law ch. 293)" + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "published": "Colorado Revised Statutes 2025 Title 8 (OLLS); the server reports Last-Modified 2025-09-16; it includes 8-13.3-507(3)(b) as amended by SB 25-144" + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "published": "7 CCR 1107-1, version effective 2026-01-01; adopted 2025-10-31; published in the Colorado Register 2025-12-10 (eDocket 2025-00438), per the SOS rule history page" + }, + { + "url": "https://famli.colorado.gov/news-article/another-national-first-paid-leave-for-neonatal-intensive-care-unit-parents", + "published": "FAMLI Division news, dated Monday, June 16, 2025 (0.88% starting in 2026)" + }, + { + "url": "https://famli.colorado.gov/news-article/update-your-employee-headcount-for-2026-premiums", + "published": "FAMLI Division news, Updated Thursday, December 11, 2025 (2026 rates: 0.88% for 10+, 0.44% for 9 or fewer, all employers may deduct up to 0.44%)" + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "published": "SSA, Cost-of-Living Increase and Other Determinations for 2026, Federal Register 2025-11-03 (OASDI base $184,500 for 2026)" + } + ], + "published_before_2026_07_03": "Yes. The 0.88% rate was enacted May 30, 2025 and took effect August 6, 2025. The 50% employee cap has been in the statute since Proposition 118 (effective December 31, 2020). The $184,500 wage base was published November 3, 2025. The implementing rule was published December 10, 2025. All of these predate 2026-07-03." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_043", + "engine_amount": 17.99, + "law_amount_at_max_employee_share": 17.99, + "matches_within_1_dollar": true, + "note": "The law's maximum employee share is 50% x 0.88% = 0.44%. Wages of $4,088.15 are below the 2026 cap of $184,500, so the whole amount counts: 4,088.15 x 0.0044 = 17.98786, which rounds to $17.99. The engine computes employee_rate (0.0044 from 2026-01-01) x min(max(0, employment_income), SS cap) = 4,088.15 x 0.0044 = 17.98786, or $17.99. The difference is $0.00. Rounding each pay period to the half-cent (7 CCR 1107-1 \u00a71.4(6)(B)) could move this by a few cents. The legally required employee amount is $0: deduction is at the employer's option, and an employer that does not deduct is treated as having paid the share. So $17.99 is the most an employer may deduct, not a mandatory withholding." + } + ], + "engine_documentation": "I read these files this session, all under .venv-pe21517/.../policyengine_us.\n\n- **co_employee_famli_contribution.py.** Label \"Colorado employee FAMLI contribution\"; documentation \"Employee-side Colorado FAMLI payroll contribution.\" The formula is `rate * person(\"co_famli_taxable_wages\")` with rate = gov.states.co.tax.payroll.famli.employee_rate. It does not depend on employer headcount. Unlike the Minnesota variable, it does not document an assumption that the employer withholds the maximum permitted share.\n- **co_famli_taxable_wages.py.** Documentation: \"Wages subject to Colorado FAMLI premiums, including federal pre-tax payroll deductions and capped at the Social Security contribution and benefit base.\" The formula is min(max(0, employment_income), gov.irs.payroll.social_security.cap). That cap is 184,500 for 2026-01-01, marked \"Official SSA determination.\"\n- **employee_rate.yaml.** Description: \"Employee-side Colorado FAMLI contribution rate.\" Values: 0.0045 from 2023-01-01 and 0.0044 from 2026-01-01. References: C.R.S. 8-13.3-507 (crs2023-title-08.pdf#page=199) and the Colorado Senate Bill 25-144 engrossed version (2025a_144_eng.pdf#page=2), not the signed act.\n- **employer_rate.yaml.** Same values and references.\n- **employer_headcount_threshold.yaml.** Value 10, citing 8-13.3-507.\n- **co_employer_famli_contribution.py.** Employer rate x wages only when employer_headcount >= 10, otherwise 0.\n- **co_employee_state_payroll_tax.py.** Documentation: \"Employee-side Colorado payroll-funded contributions, including FAMLI.\" It adds co_employee_famli_contribution.\n- **variables/gov/states/tax/payroll/employee_state_payroll_tax.py.** Documentation: \"Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules.\" Its components include co_employee_state_payroll_tax.\n\nThe aggregator's word \"mandatory\" does not fit Colorado, where the employee share is a permissive deduction.", + "caveats": [ + "The deduction is optional for every covered employer. Under C.R.S. 8-13.3-507(5) the employer 'may deduct up to 50 percent', and under 7 CCR 1107-1 \u00a71.4(6)(A) a deduction it does not take counts as the employer electing to pay that share. The employee's actual FAMLI cost can be anywhere from $0 to 0.44% of wages. The benchmark prompt never states whether the employer deducts.", + "Employers with fewer than 10 employees (headcount counted nationwide, people employed 20 or more workweeks in the prior year, per 7 CCR 1107-1 \u00a71.5) owe no employer share. They must still remit 50% (0.44%) and may deduct all of it from wages. The maximum employee share is the same 0.44%; what changes is the employer's remittance (0.88% versus 0.44%). An employer that does not update its headcount by February 28 is presumed to have 10 or more employees.", + "Approved private plans (8-13.3-507(8), 8-13.3-521): the employer remits nothing to the state fund. Under 521(1)(k), the plan may charge employees no more than the state plan would, so an employee might still pay up to 0.44%, but to the plan, not as a state premium.", + "Local governments may opt out (8-13.3-507(9), 8-13.3-522). Their employees owe nothing unless they opt in under 8-13.3-514, in which case they pay 50% of the premium directly (507(4)(b)-(c)), or through withholding if the local government agrees to collect it.", + "The federal government is not an 'employer' (8-13.3-503(8)(c)), so federal employees owe no FAMLI premium. Railroad employees covered by the Railroad Unemployment Insurance Act are excluded from 'employee' (503(7)).", + "Self-employed people are covered only if they opt in (8-13.3-514), and then pay 50% of the premium on self-employment income (507(4)(a)). The benchmark excludes self-employment tax.", + "Collective bargaining: I found no provision in Part 5 that sets the premium split under a CBA. 8-13.3-510(2)(a) only says Part 5 does not diminish CBA rights or obligations. Because the deduction is permissive, a CBA or employer policy can reduce the employee's deduction to $0 but cannot raise it above 50% (7 CCR 1107-1 \u00a71.4(6) prohibits over-deduction).", + "Only wages of employees localized to Colorado are subject to premiums (7 CCR 1107-1 \u00a71.6). 'Wages' means gross wages (7 CCR 1107-1 \u00a71.2(9)); the agency says premium deductions are post-tax.", + "Under C.R.S. 8-13.3-507(7), FAMLI premiums 'are fees and not taxes'. That matters for whether they count under the prompt's 'mandatory employee state payroll taxes' at all.", + "8-13.3-502(4)(a), a findings clause, refers to collecting premiums 'from employers and employees'. The operative sections, 507(2) and (5), still put the remittance duty on the employer and make the employee deduction permissive.", + "Rates for 2027 onward are set by the director's rule by September 1 of the prior year (507(3)(c), capped at 1.2%). This does not affect 2026.", + "famli.colorado.gov returned 403 to WebFetch, so I read those pages via curl with a browser user agent. ssa.gov returned 403, so I verified the 2026 wage base from the Federal Register notice instead." + ], + "confidence": "high", + "summary": "This is an optional employer pass-through, not a mandatory employee withholding. Under C.R.S. 8-13.3-507(2) and (5), the premium is the employer's liability: an employer with 10 or more employees must remit the full 0.88% in 2026, and one with fewer than 10 must remit 0.44%. Either kind of employer \"may deduct up to 50 percent\" (0.44% of wages) from the employee, up to the Social Security wage base of $184,500. 7 CCR 1107-1 \u00a71.4(6)(A), effective 2026-01-01, says an employer that does not deduct the maximum is treated as having elected to pay that share. The FAMLI Division also says some employers choose to deduct nothing. The 2026 rate comes from SB 25-144, signed 2025-05-30 and effective 2025-08-06; every input predates 2026-07-03. The engine counts co_employee_famli_contribution = 0.44% x min(wages, SS cap) whatever the employer's size or choice. For scenario_043 that is 4,088.15 x 0.0044 = $17.99, which equals the law's maximum permitted employee share exactly. The legally mandatory employee amount is $0, so this is a maximum the employer may deduct, not a required withholding. The aggregator documentation's word \"mandatory\" does not hold for Colorado, and unlike Minnesota the Colorado variable does not document the \"employer deducts the maximum\" assumption." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with 10 or more employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 100 percent of the premium", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with fewer than 10 employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 50 percent of the premium", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "beginning on January 1, 2023, for each employee, an employer shall remit to the fund established under section 8-13.3-518 premiums in the form and manner determined by the division.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "From January 1, 2026, through December 31, 2026, the premium amount is eighty-eight hundredths of one percent of wages per employee.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "A self-employed individual who elects coverage under section 8-13.3-514 shall pay only 50 percent of the premium required for an employee by section 8-13.3-507 (3)", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Collect insurance premiums from employers and employees at rates reasonably calculated to defray the costs of providing the program's leave benefits to workers", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "The division shall develop a program notice that details the program requirements, benefits, claims process, payroll deduction requirements", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Premiums established under this section are fees and not taxes.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2026-title-08.pdf", + "quote": "An employer with 10 or more employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 100 percent of the premium", + "fetched_and_read": true + }, + { + "url": "https://leg.colorado.gov/laws/session-laws/SB25-144/293/download", + "quote": "FROM JANUARY 1, 2026, THROUGH DECEMBER 31, 2026, THE PREMIUM AMOUNT IS EIGHTY -EIGHT HUNDREDTHS OF ONE PERCENT OF WAGES PER EMPLOYEE .", + "fetched_and_read": true + }, + { + "url": "https://leg.colorado.gov/bills/sb25-144", + "quote": "Governor Signed: May 30, 2025 ... Session Law Chapter: 293 ... Effective Date: August 6, 2025", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "An employer required to remit premiums pursuant to C.R.S. \u00a7 8-13.3-507 may not deduct more than the maximum allowable employee share of the premium from wages paid for a pay period.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay that portion of the employee share", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=9914&fileName=7+CCR+1107-1", + "quote": "\u201cEmployee share\u201d is defined as 50 percent of the premium required for an employee by section 8-13.3-507 (3) C.R.S.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/eDocketDetails.do?trackingNum=2025-00438", + "quote": "Adopted Date 10/31/2025 ... Colorado Register publication date 12/10/2025 Effective Date 01/01/2026", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/Upload/AGORequest/Redline2025-00438.docx", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "The premiums are set to 0.88% of the employee\u2019s wage, with 0.44% paid by the employer and 0.44% paid by the employee. Some employers may choose not to deduct any premiums contributions from their employees\u2019 wages.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Most Colorado businesses should be deducting FAMLI premiums from all employees on their payroll, including full-time, part-time, and seasonal.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If you have 10 or more employees (nationwide), you may deduct up to 50% of the 0.88% premium as a standard payroll deduction.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If both jobs are with employers that are choosing to deduct the employee\u2019s portion of the premium and are not an opted out local government", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/update-your-employee-headcount-for-2026-premiums", + "quote": "All employers: Option to deduct up to 0.44% from employee wages toward your total premium amount", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/another-national-first-paid-leave-for-neonatal-intensive-care-unit-parents", + "quote": "The measure also lowers premiums for all Colorado workers and employers, from 0.9% of wages to 0.88% of wages, starting in 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/co/tax/payroll/famli/co_employee_famli_contribution.py", + "quote": "rate = parameters(period).gov.states.co.tax.payroll.famli.employee_rate\n return rate * person(\"co_famli_taxable_wages\", period)", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/tax/payroll/employee_state_payroll_tax.py", + "quote": "Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules.", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None that change the result. 4,088.15 x 0.0044 = 17.98786, which rounds to $17.99 and matches the engine. The 2026 OASDI base of $184,500 is confirmed in the Federal Register (90 FR 49047, 2025-11-03) and in the engine's cap.yaml, where it is labeled 'Official SSA determination'. The maximum employee share is 0.0044 x 184,500 = $811.80.", + "The engine treats the maximum permitted deduction (0.44%) as the employee's payroll tax. The legally required employee amount for an ordinary private-sector W-2 employee is $0, because the deduction is at the employer's option. $17.99 is a ceiling, not a mandatory withholding.", + "The 0.88% total rate for 2026 rests on C.R.S. 8-13.3-507(3)(b), as amended by SB 25-144 (Session Laws 2025 ch. 293, approved May 30, 2025, effective August 6, 2025). The CRS 2026 Title 8 (server Last-Modified 2026-10-02) shows 507 unchanged. Its source note lists no 2026 amendment to 507. The only 2026 change in Part 5 is HB 26-1309, which amended 503(6). So nothing enacted in the 2026 session altered the 2026 rate or the 'may deduct' rule before 2026-07-03.", + "Small timing nuance: 7 CCR 1107-1 \u00a71.4(6)(B) rounds each pay period to the cent, with half a cent or more rounding up. An annual figure built from per-period deductions could differ from $17.99 by a few cents. This does not affect the classification." + ], + "citation_problems": [ + "Page number: 8-13.3-507(2) is cited as 'Title 8 p. 221', but in crs2025-title-08.pdf it falls on 'Page 220 of 796'. 507(1) through (3)(c)(II) are on p. 220, and (3)(c)(III) through (9) are on p. 221. The quoted text is verbatim.", + "The research says 7 CCR 1107-1 \u00a71.4(6)(A) has been 'in force since 01/01/2026'. That is imprecise. The SOS editor's notes for the version effective 2026-01-01 list only 1.4 2 and 1.4 10 as changed within 1.4. The 2025-00438 redline shows 1.4(6) and (6)(A) carried over unamended. The original 2022 rule (ruleVersionId 9914, part B.1(a)) already had the same 'considered to have elected to pay' rule. The provision is older than the research says, which, if anything, strengthens the classification.", + "rate_2026.max_employee_share says \"'employee share' is defined as 50% in 7 CCR 1107-1\". That definition appears only in the superseded original version effective 2022-01-01 (ruleVersionId 9914). The current version effective 2026-01-01 (ruleVersionId 12341) uses 'maximum allowable employee share' without defining it. The 50% cap rests directly on C.R.S. 8-13.3-507(5). The operative_text entry itself correctly labels the quote as coming from the original rule, and the quote is verbatim at that URL.", + "Trivial: the agency quotes use straight apostrophes ('employee's'), while the pages use curly ones (\u2019). Otherwise every quote checked appears verbatim at its stated URL: the statute, SB 25-144, the rules, the FAMLI pages, and the Federal Register.", + "The research reports that famli.colorado.gov returned 403 to WebFetch. A curl request with a browser user agent returned HTTP 200 for all four FAMLI pages, so those pages can be verified." + ], + "reasoning": "I read all of Part 5 of Article 13.3 (8-13.3-501 through 524) myself in the OLLS CRS 2025 Title 8 PDF, plus the definitions in 503. Then I searched the whole part for 'deduct', 'withh', 'shall pay', 'employee shall', 'liab' and 'contribut'. In covered private employment, the only premium-payment duty falls on the employer: 507(2) says the employer 'shall remit' premiums 'for each employee'. Section 507(5) is the only provision on employee deductions, and it is permissive in both sentences: 'may deduct up to 50 percent'. The employer still 'shall remit' 100% (10 or more employees) or 50% (fewer than 10).\n\nThe 'shall pay' language appears only in 507(4)(a)-(b). It applies to self-employed people and employees of opted-out local governments who elect coverage under 514, and they remit directly. 507(9)(b) confirms that local-government employees owe nothing unless they elect coverage. 509(2)'s 'shall continue to pay' concerns health-benefit cost-sharing during leave, not premiums.\n\nTwo passages could be read the other way, and neither is operative:\n- 502(4)(a), a findings clause, authorizes the enterprise to 'collect insurance premiums from employers and employees'.\n- 511 requires the program notice to describe 'payroll deduction requirements'. The rules give that content: an employer must not deduct more than the employee share, and a missed deduction cannot be clawed back.\n\nNeither creates an employee liability or a duty to withhold. I found no 'shall deduct' or 'shall withhold' clause anywhere in Part 5.\n\nThe current rule, 7 CCR 1107-1 (ruleVersionId 12341), confirms the reading. I verified on the SOS eDocket that it was adopted 10/31/2025, published in the Colorado Register 12/10/2025, and took effect 01/01/2026. Under \u00a71.4(6), the employer 'may not deduct more than' the maximum allowable employee share. Under \u00a71.4(6)(A), a missed deduction means the employer 'is considered to have elected to pay' that share. That is optional pass-through language, and it has been in the rule since the 2022 original.\n\nThe FAMLI Division's pages say the same. 'Some employers may choose not to deduct any premiums contributions.' The FAQ offers an 'Option to deduct up to 0.44%', says employers 'may deduct up to 50%', and refers to 'employers that are choosing to deduct'. Some agency language points toward the employee paying: 'Most Colorado businesses should be deducting', the FAQ table's 'Employee Premium' column, and 'collect and remit the 0.44% of the employee's share'. That is descriptive guidance, it contradicts no statute, and the same pages state that deduction is optional.\n\nI also checked the CRS 2026 Title 8. Section 507 is unchanged and has no 2026 amendment, so no 2026 session law made the deduction mandatory before 2026-07-03.\n\nIn the engine source, co_employee_famli_contribution = employee_rate (0.0044 from 2026-01-01) x min(max(0, employment_income), SS cap of 184,500). It does not depend on employer headcount or on whether the employer deducts. Unlike the Minnesota variable, it does not document the 'maximum share' assumption. The aggregator employee_state_payroll_tax calls its components 'mandatory', which does not hold for Colorado. For scenario_043, $17.99 equals the maximum permitted deduction exactly. The legally mandatory employee amount is $0.\n\nThe only defects I found in the research are minor citation and characterization issues: a page number, the 'in force since 01/01/2026' dating, and citing the superseded 2022 definition of 'employee share' as if it were current. None of them undermines the classification. Classification confirmed: optional_employer_pass_through." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with 10 or more employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 100 percent of the premium", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "An employer with fewer than 10 employees may deduct up to 50 percent of the premium required for an employee by section 8-13.3-507 (3) from that employee's wages and shall remit 50 percent of the premium", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "beginning on January 1, 2023, for each employee, an employer shall remit to the fund established under section 8-13.3-518 premiums in the form and manner determined by the division.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "(b) From January 1, 2026, through December 31, 2026, the premium amount is eighty- eight hundredths of one percent of wages per employee.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Premiums shall not be required for employees' wages above the contribution and benefit base limit established annually by the federal social security administration", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "Premiums established under this section are fees and not taxes.", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "(a) Collect insurance premiums from employers and employees at rates reasonably calculated to defray the costs of providing the program's leave benefits to workers", + "fetched_and_read": true + }, + { + "url": "https://olls.info/crs/crs2025-title-08.pdf", + "quote": "(II) Paid wages of one thousand five hundred dollars or more during any calendar quarter in the preceding calendar year.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "An employer required to remit premiums pursuant to C.R.S. \u00a7 8-13.3-507 may not deduct more than the maximum allowable employee share of the premium from wages paid for a pay period.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay that portion of the employee share", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=12341&fileName=7%20CCR%201107-1", + "quote": "Rules 1.1 3, 1.2 3, 1.2 5, 1.3 1, 1.3 2.D, 1.4 2, 1.4 10, 1.5, 1.6 5 eff. 01/01/2026.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=9914&fileName=7+CCR+1107-1", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay that portion of the employee share", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/GenerateRulePdf.do?ruleVersionId=9914&fileName=7+CCR+1107-1", + "quote": "\u201cEmployee share\u201d is defined as 50 percent of the premium required for an employee by section 8-13.3- 507 (3) C.R.S.", + "fetched_and_read": true + }, + { + "url": "https://www.sos.state.co.us/CCR/DisplayRule.do?action=ruleinfo&ruleId=3367&deptID=2&agencyID=89&deptName=Department%20of%20Labor%20and%20Employment&agencyName=Division%20of%20Family%20and%20Medical%20Leave%20Insurance&seriesNum=7%20CCR%201107-1", + "quote": "Current version ... 01/01/2026 (PDF) Permanent Rule 10/31/2025 12/10/2025 2025-00438", + "fetched_and_read": true + }, + { + "url": "https://leg.colorado.gov/laws/session-laws/SB25-144/293/download", + "quote": "FROM JANUARY 1, 2026, THROUGH DECEMBER 31, 2026, THE PREMIUM AMOUNT IS EIGHTY -EIGHT HUNDREDTHS OF ONE PERCENT OF WAGES PER EMPLOYEE . ... Approved: May 30, 2025", + "fetched_and_read": true + }, + { + "url": "https://leg.colorado.gov/bills/sb25-144", + "quote": "Became Law; Governor Signed: May 30, 2025; Session Law Chapter 293; Effective Date 08/06/2025 (as summarized by WebFetch)", + "fetched_and_read": true + }, + { + "url": "https://leg.colorado.gov/bills/SB26-127", + "quote": "Status Lost; on April 9, 2026 the Senate Committee on Business, Labor, & Technology voted to postpone SB 26-127 indefinitely (as summarized by WebFetch); no premium change", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Some employers may choose not to deduct any premiums contributions from their employees\u2019 wages.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Employers cannot collect missed premiums from employees in later pay periods. It is important to know employees are never required to pay more than 50% of the total premium.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers", + "quote": "Most Colorado businesses should be deducting FAMLI premiums from all employees on their payroll, including full-time, part-time, and seasonal.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If both jobs are with employers that are choosing to deduct the employee\u2019s portion of the premium and are not an opted out local government", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "If you have 10 or more employees (nationwide), you may deduct up to 50% of the 0.88% premium as a standard payroll deduction.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/employers/employer-faqs", + "quote": "you as the employer would be required to pay the 0.44% employer's share and collect and remit the 0.44% of the employee's share for each of those eight employees.", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/update-your-employee-headcount-for-2026-premiums", + "quote": "Option to deduct up to 0.44% from employee wages toward your total premium amount", + "fetched_and_read": true + }, + { + "url": "https://famli.colorado.gov/news-article/another-national-first-paid-leave-for-neonatal-intensive-care-unit-parents", + "quote": "lowers premiums for all Colorado workers and employers, from 0.9% of wages to 0.88% of wages, starting in 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/co/tax/payroll/famli/co_employee_famli_contribution.py", + "quote": "rate = parameters(period).gov.states.co.tax.payroll.famli.employee_rate; return rate * person(\"co_famli_taxable_wages\", period)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. I independently confirmed the 2026 total rate of 0.88% (C.R.S. 8-13.3-507(3)(b), as amended by SB 25-144, ch. 293, approved 2025-05-30 and effective 2025-08-06), the maximum employee share of 50% = 0.44% (507(5)), and the wage cap equal to the OASDI base of $184,500 (507(6); 90 FR 49047, published 2025-11-03).", + "scenario_043: 4,088.15 x 0.0044 = 17.98786, or $17.99. I ran the installed engine (policyengine-us in .venv-pe21517) for a single CO earner with $4,088.15 of 2026 employment income. It returned co_famli_taxable_wages 4088.15, and co_employee_famli_contribution, co_employee_state_payroll_tax and employee_state_payroll_tax each 17.98786. That matches the researcher's $17.99 exactly. The legally mandatory employee amount is $0. $17.99 is the most an employer may deduct.", + "Minor wording: the researcher describes 7 CCR 1107-1 \u00a71.4(6)(B) as rounding 'to the half-cent'. The rule drops a fraction of a cent unless it is one-half cent or more, in which case it rounds up to one cent. So it is half-up rounding to the cent. The amount impact is negligible either way.", + "Not raised by the researcher: C.R.S. 8-13.3-503(8)(a) defines 'employer' as one that employed someone in 20 or more workweeks, or paid $1,500 or more of wages in a calendar quarter of the preceding year. A very small or new employer may fall outside it and owe no premium at all. The prompt does not specify, so this widens the legally possible range further toward $0." + ], + "citation_problems": [ + "Stale definition cite: rate_2026.max_employee_share says \"'employee share' is defined as 50% in 7 CCR 1107-1\" in the present tense. That definition appears only in the original version effective 01/01/2022 (ruleVersionId 9914, \u00a71.4 Definitions), where I confirmed it verbatim. The current version effective 01/01/2026 (ruleVersionId 12341, \u00a71.2 Definitions and Clarifications) has no 'Employee share' definition. The 50% cap still holds through the statute (507(5)), so the conclusion is unaffected.", + "The can_employer_elect field says 7 CCR 1107-1 \u00a71.4(6)(A) is 'in force since 01/01/2026'. That understates its age. The same 'considered to have elected to pay' language is in the original rule effective 01/01/2022. The current version's editor's notes list the provisions amended effective 01/01/2026 (1.1 3, 1.2 3, 1.2 5, 1.3 1, 1.3 2.D, 1.4 2, 1.4 10, 1.5, 1.6 5), and 1.4 6 is not among them. This only makes the optional reading stronger, since the rule has been in force for the whole program.", + "I confirmed every other operative quote verbatim against the documents I fetched: 507(2), 507(3)(b), 507(5) both sentences, 507(6), 507(7), 507(8), 507(9)(b), 521(1)(k), 503(8)(c), 502(4)(a), rule 1.4(6) and 1.4(6)(A), the SB 25-144 session-law text and 'Approved: May 30, 2025', all seven FAMLI agency quotes, the FAMLI news-article dates (June 16, 2025; Updated Thursday, December 11, 2025), and the SOS rule history (adopted 10/31/2025, Colorado Register 12/10/2025, eDocket 2025-00438). The SB 25-144 quote keeps PDF text-extraction spacing ('EIGHTY -EIGHT', 'EMPLOYEE .'), which is an artifact of extraction, not a misquote.", + "Omitted contrary agency language, which does not change the result: the FAMLI employer FAQ says an employer with 10 or more employees is 'required to pay the 0.44% employer's share and collect and remit the 0.44% of the employee's share'. The employers page says 'Most Colorado businesses should be deducting FAMLI premiums'. Both are informal guidance. The same pages also say 'Some employers may choose not to deduct' and refer to 'employers that are choosing to deduct', and the statute and rule control." + ], + "reasoning": "I tried to refute the classification through the administration lens and could not. Every primary source I read supports 'optional employer pass-through'.\n\nStatute: I downloaded the OLLS CRS 2025 Title 8 PDF myself (Last-Modified 2025-09-16). In C.R.S. 8-13.3-507(2), the duty to remit falls on the employer alone ('an employer shall remit'). 507(5) gives employers of both sizes the power that they 'may deduct up to 50 percent' of the premium from wages, while they 'shall remit' 100% (10 or more employees) or 50% (fewer than 10). Part 5 imposes a payment duty on an individual only in 507(4): self-employed people and local-government employees who opt in under 8-13.3-514. 507(9)(b) confirms that an opted-out local government's employee 'is not required to pay' unless they elect coverage. The findings clause 502(4)(a) mentions collecting 'from employers and employees', but it is not operative. 507(7) says premiums are 'fees and not taxes'.\n\nRules: I read both the current 7 CCR 1107-1 (effective 01/01/2026; adopted 10/31/2025; Colorado Register 12/10/2025, per the SOS history page I fetched) and the original 2022 version. In both, an employer that fails to deduct the maximum 'is considered to have elected to pay that portion of the employee share'. The current rule bars the employer from recovering it later except when wages were insufficient, and fines over-deduction. Neither version contains any provision that makes the employee liable when no deduction is taken. The only 'owes' provision (1.4(3)) applies to individuals electing coverage.\n\nAgency guidance: I fetched famli.colorado.gov with curl and a browser user agent, because WebFetch got a 403. The agency says employers 'may deduct up to 50%', that 'Some employers may choose not to deduct', and it refers to 'employers that are choosing to deduct'. The 2026 headcount notice says all employers have the 'Option to deduct up to 0.44%'. Some looser FAQ wording ('collect and remit the 0.44% of the employee's share'; 'should be deducting') is informal guidance that conflicts with the agency's own 'may choose not to deduct'. It cannot override the statute's 'may' or the rule's deemed election.\n\nSo the employee's legal liability is $0, and up to 0.44% of capped wages is at the employer's discretion.\n\nRates and dates: SB 25-144 session law ch. 293 (approved May 30, 2025; the bill page says effective 08/06/2025) sets 0.88% for 2026. The OASDI base of $184,500 is in 90 FR 49047, published 2025-11-03. All of these predate 2026-07-03. I found one 2026-session FAMLI bill, SB26-127; it was postponed indefinitely on 2026-04-09 and does not touch premiums.\n\nEngine: I read the engine source this session. co_employee_famli_contribution = employee_rate (0.0044 from 2026-01-01) x min(max(0, employment_income), SS cap of 184,500). It does not depend on employer headcount or on any employer-deduction choice, and it carries no documented 'employer deducts the maximum' assumption. The Minnesota variable does carry that assumption ('withholds the maximum permitted employee share'). employee_state_payroll_tax's documentation calls its components 'mandatory', which does not hold for Colorado. Running the engine gave 17.98786 for scenario_043, which equals 0.44% x $4,088.15, the legal maximum permitted deduction, not a mandatory withholding.\n\nThe only defects I found are minor citation issues: a stale 'employee share' definition cite and an understated in-force date for \u00a71.4(6)(A). Neither undermines the classification." + } + ] + }, + { + "key": "MA", + "research": { + "program": "Massachusetts Paid Family and Medical Leave (PFML) contributions, M.G.L. c. 175M", + "state": "MA", + "engine_variables": [ + "ma_employee_paid_leave_contribution", + "ma_employee_state_payroll_tax (aggregator, adds ma_employee_paid_leave_contribution)", + "ma_paid_leave_taxable_wages", + "ma_employer_paid_leave_contribution (employer side, not in payroll_tax)", + "parameters gov.states.ma.tax.payroll.paid_leave.family_rate / medical_rate / medical_employee_share / employer_headcount_threshold", + "gov.irs.payroll.social_security.cap" + ], + "premium_liability": "The employer owes the contribution. Under M.G.L. c. 175M s. 6(a), the employer (or covered business entity) \"shall remit\" the contributions for each employee to the Family and Employment Security Trust Fund. Under s. 7(d), employers, covered business entities and electing self-employed individuals \"make contributions\". 458 CMR 2.05(5) defines the employer's portion as whatever it does not deduct from wages, and says the employer \"in all instances\" pays and remits the remainder. 458 CMR 2.05(7) penalizes the employer for failing to remit and bars it from recouping from employees an allowable deduction it failed to take. The employee owes nothing to the state directly. Their only exposure is a wage deduction the employer chooses to take, up to the statutory caps. The exception is self-employed people who elect coverage: they are responsible for all contributions (s. 6(b)).", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "M.G.L. c. 175M, s. 6(a)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "For each employee or covered contract worker, an employer or a covered business entity shall remit to the Family and Employment Security Trust Fund established in section 7 contributions in the form and manner as determined by the department.", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 6(c)(1)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "For medical leave, an employer shall not deduct more than 40 per cent of the contribution required for an employee by subsection (a) from that employee's wages and shall remit the full contribution", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 6(c)(2)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "For family leave, an employer may deduct not more than 100 per cent of the contribution required for an employee by subsection (a) from that employee's wages, and shall remit the full contribution", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 6(d) (employers with fewer than 25 employees)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "an employer employing less than 25 employees in the commonwealth shall not be required to pay the employer portion of premiums for family and medical leave; provided, however, that such employer shall remit, for each employee, 100 per cent of the family leave contribution and 40 per cent", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 6(f) (wage cap)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "Contributions to the trust fund under this section shall not be required for employees' wages ... above the contribution and benefit base limit established annually by the federal Social Security Administration", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 7(d) and 7(e)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section7", + "quote": "employers, covered business entities and self-employed individuals shall, unless subject to provisions under section 11, make contributions as required under section 6 and transmit those contributions to the trust fund", + "fetched_and_read": true + }, + { + "citation": "458 CMR 2.05(5) (official mass.gov PDF, regulation dated 01/06/2023)", + "url": "https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "an employer or a covered business entity may deduct up to certain defined percentages of the required medical leave and family leave contributions directly from wages", + "fetched_and_read": true + }, + { + "citation": "458 CMR 2.05(5)", + "url": "https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "The employer or covered business entity in all instances shall be responsible for paying and remitting the remainder of any required contribution not lawfully deducted from the employee or covered contract worker.", + "fetched_and_read": true + }, + { + "citation": "458 CMR 2.05(5)(c)", + "url": "https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity that opts to deduct a lower percentage of medical or family leave contributions than the maximum allowable deductions ... shall still be required to remit the full amount", + "fetched_and_read": true + }, + { + "citation": "458 CMR 2.05(5)(d)", + "url": "https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity may choose to deduct differing percentages from the wages or qualifying payments of different groups of covered individuals", + "fetched_and_read": true + }, + { + "citation": "458 CMR 2.05(7)", + "url": "https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity that failed to properly assess the allowable deduction ... shall not recoup any assessment or cost of repayment through charges against employees", + "fetched_and_read": true + }, + { + "citation": "M.G.L. c. 175M, s. 11(a)(1)(xii) (private plans)", + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section11", + "quote": "providing that the cost to employees covered by a private plan shall not be greater than the cost charged to employees under the state program", + "fetched_and_read": true + }, + { + "citation": "Acts 2026, c. 101, ss. 25-26, 43, 45 (approved June 12, 2026; emergency law)", + "url": "https://malegislature.gov/Laws/SessionLaws/Acts/2026/Chapter101", + "quote": "SECTION 45. Sections 7, 13 through 15, inclusive, 21, 25 and 26 ... shall apply for taxable years beginning on or after January 1, 2026.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Employers must contribute at least the minimum share of the contribution described below; however, they may choose to cover a larger share of the contribution for some or all of their employees.", + "fetched_and_read": true + }, + { + "url": "https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Small employers are responsible for sending the funds withheld from covered individuals' wages but are under no obligation to contribute themselves. However, they may elect to cover some or all of the covered individuals' share.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Employers with 25 or more covered individuals must send to DFML a contribution of 0.88% of eligible wages. This contribution can be split between covered individuals' payroll or wage withholdings and an employer contribution.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Up to 40% of the medical leave contribution can be withheld from a covered individual's wages (0.28% of eligible wages). Employers are responsible for contributing the remaining 60% (0.42% of eligible wages).", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260916051521id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "New Massachusetts legislation (Chapter 101 of the Acts of 2026) shifts employer contributions from medical leave to family leave, effective January 1, 2027.", + "fetched_and_read": true + }, + { + "url": "https://www.mass.gov/info-details/pfml-workplace-poster-notices-and-rate-sheets-for-massachusetts-employers", + "quote": "Contribution rates for 2026. Employers are responsible for sending contributions to DFML for all employees.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Yes. M.G.L. c. 175M s. 6(c) sets only ceilings on what may be deducted (\"shall not deduct more than 40 per cent\"; \"may deduct not more than 100 per cent\"). 458 CMR 2.05(5)(c) covers an employer that \"opts to deduct a lower percentage\", and 2.05(5)(d) lets it deduct different percentages for different groups. DFML's rates page says employers \"may choose to cover a larger share\", and that small employers \"may elect to cover some or all\" of the employees' share.", + "rate_2026": { + "total_rate": "0.88% of eligible wages for employers with 25 or more covered individuals: family leave 0.18% plus medical leave 0.70%. Employers with fewer than 25 remit an effective 0.46%, which is the full family share (0.18%) plus 40% of medical (0.28%), under s. 6(d).", + "max_employee_share": "0.46% of eligible wages: 100% of the family rate (0.18%) plus 40% of the medical rate (0.28%). This is the same for large and small employers. It is a ceiling on what the employer may deduct, not a required employee payment. One caveat: Acts 2026 c. 101 s. 45 could be read as moving the split to 40% family / 100% medical (0.772%) for 2026. See caveats; DFML applies that change from 2027.", + "wage_base_or_cap": "$184,500, the 2026 Social Security contribution and benefit base, under s. 6(f) and 458 CMR 2.05(1)(c).", + "sources": [ + { + "url": "https://web.archive.org/web/20251110011839id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "published": "DFML rates page headed '2025 & 2026' (0.88%; 0.18% family; 0.70% medical; employee up to 0.18% + 0.28%). It shows 'Last updated: October 1, 2025' and was archived 2025-11-10 and again 2026-06-17 with the same 2026 split." + }, + { + "url": "https://www.mass.gov/info-details/pfml-workplace-poster-notices-and-rate-sheets-for-massachusetts-employers", + "published": "DFML 2026 rate sheets page, last updated November 3, 2025. I read the page but did not open the Word rate sheets." + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "published": "SSA Federal Register notice 'Cost-of-Living Increase and Other Determinations for 2026', published 2025-11-03: OASDI base of $184,500 for remuneration paid in 2026. Fetched through the Federal Register API; ssa.gov/oact/cola/cbb.html also shows $184,500." + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "published": "Statute setting the 40% medical and 100% family deduction caps, in force before 2026. Acts 2026 c. 101 (approved 2026-06-12) amends these caps; see caveats." + } + ], + "published_before_2026_07_03": "Yes. The 0.88% total, the 0.18%/0.70% allocation and the $184,500 base were all published by November 2025. DFML's guidance as of 2026-06-17 still showed the old 40% medical / 100% family split for 2026. Acts 2026 c. 101 was also published before the cutoff (approved June 12, 2026), and its s. 45 literally applies the swapped split to taxable years beginning on or after Jan 1, 2026. DFML's statement that the swap takes effect Jan 1, 2027 came after the cutoff: the rates page shows 'Last updated: July 10, 2026' in a 2026-09-16 archive copy." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_081", + "engine_amount": 805.01, + "law_amount_at_max_employee_share": 805.01, + "matches_within_1_dollar": true, + "note": "Wages of $175,002 are below the $184,500 cap, so all of them are eligible. Maximum employee deduction under s. 6(c) before c. 101 and DFML 2026 guidance is 0.18% + 0.40 x 0.70% = 0.46%, and 175,002 x 0.0046 = 805.0092, or $805.01. The engine computes (family_rate 0.0018 + medical_rate 0.007 x medical_employee_share 0.4) x min(employment_income, SS cap 184,500), which gives the same $805.01. The law does not require this amount. It is the most the employer may deduct, and the true employee amount can be anywhere from $0 to $805.01 depending on the employer's choice. Under the literal reading of Acts 2026 c. 101 s. 45 (swapped caps applied to 2026, with 2026 rates unchanged), the maximum would be 0.40 x 0.18% + 1.00 x 0.70% = 0.772%, and 175,002 x 0.00772 = $1,351.02. For comparison, the full 0.88% that the employer remits would be $1,540.02." + } + ], + "engine_documentation": "I read these files this session under policyengine_us 2.15.17. variables/gov/states/ma/tax/payroll/ma_employee_state_payroll_tax.py has label \"Massachusetts employee state payroll tax\" and documentation \"Employee-side Massachusetts PFML contributions.\" It is defined_for MA with adds = [\"ma_employee_paid_leave_contribution\"]. paid_leave/ma_employee_paid_leave_contribution.py has label \"Massachusetts employee paid leave contribution\" and documentation \"Employee-side Massachusetts Paid Family and Medical Leave contribution, assuming the employer withholds the maximum permitted employee share.\" Its formula is (family_rate + medical_rate * medical_employee_share) * ma_paid_leave_taxable_wages, with no employer-size condition. paid_leave/ma_paid_leave_taxable_wages.py computes min_(max_(0, employment_income), gov.irs.payroll.social_security.cap); the cap parameter is 184_500 for 2026, labeled \"Official SSA determination.\" paid_leave/ma_employer_paid_leave_contribution.py computes the employer side as medical_rate * (1 - medical_employee_share) * wages, applied only when employer_headcount >= employer_headcount_threshold (25). Parameters under parameters/gov/states/ma/tax/payroll/paid_leave/: family_rate 0.0018, medical_rate 0.007 and medical_employee_share 0.4, all from 2026-01-01 (0 before that), and employer_headcount_threshold 25 from 2026-01-01. Each cites M.G.L. c. 175M s. 6 and the mass.gov contribution-rates page. No parameter models Acts 2026 c. 101 or a 2027 split change. The employee formula applies the same 0.46% to small and large employers, which matches s. 6(d) and DFML for 2026.", + "caveats": [ + "The employee amount is chosen by the employer, not set by law. The law caps the deduction at 100% of family plus 40% of medical (0.46%) but sets no minimum. An employer may deduct less or nothing and still must remit the full contribution (458 CMR 2.05(5)(c); DFML rates page). The engine's figure is therefore an upper bound that relies on its stated assumption that the employer withholds the maximum.", + "Acts 2026 c. 101 ss. 25-26 (approved June 12, 2026, before the 2026-07-03 cutoff) swap the caps in s. 6: up to 100% of medical and up to 40% of family. Section 45 of the act says ss. 25 and 26 'shall apply for taxable years beginning on or after January 1, 2026'. Section 43(a) of the same act describes the 'impact for employers and employees in calendar year 2027', and DFML says the change takes effect January 1, 2027, but DFML published that after the cutoff (page last updated July 10, 2026). The malegislature.gov General Laws text of s. 6 that I read on 2026-10-05 still shows the old 40/100 caps. Under the literal s. 45 reading with 2026 rates unchanged, the 2026 maximum employee share would be 0.772% ($1,351.02 for scenario_081), not 0.46%. I classify the swap as 2027 because the act's own s. 43(a), DFML, and the timing of the October 1 rate-setting under s. 7(e) all point to 2027, but the statute's text is ambiguous.", + "Employers with fewer than 25 employees (s. 6(d); 458 CMR 2.05(1)(d), counted by average total workforce including covered 1099 contract workers under 2.05(2)) owe no employer share. They still remit 0.46%, which they may deduct in full or pay themselves. The maximum employee share is still 0.46%.", + "Private and self-insured plans (s. 11; 458 CMR 2.07): employers with an approved exemption do not remit to the state fund. The plan may charge employees no more than the state program would (s. 11(a)(1)(xii)), so the employee cost could be anywhere from $0 up to the same ceiling.", + "Collective bargaining and group differences: an employer may deduct different percentages for different groups of covered individuals (458 CMR 2.05(5)(d)). DFML's examples include different bargaining units and full-time versus part-time staff.", + "Public employers: municipalities, districts, political subdivisions and authorities are covered only if they adopt c. 175M by vote (s. 10).", + "Self-employed people who elect coverage pay all contributions themselves (s. 6(b)). This is outside the benchmark's wage-earner scope.", + "Contributions stop at the SSA contribution and benefit base ($184,500 for 2026) per employer-employee relationship, under s. 6(f). The engine caps total employment_income at the SS cap rather than per employer. That makes no difference for single-employer scenario_081.", + "I read the agency pages through the Claude Browser pane and Wayback Machine copies because mass.gov returns 403 to WebFetch and curl. I read the 458 CMR text from the official mass.gov PDF by extracting it in the browser. That PDF is the version DFML labels 'Updated Jan. 6, 2023'." + ], + "confidence": "high", + "summary": "The engine's $805.01 is the most an employer may deduct from this employee, not an amount the employee must pay, so I classify MA PFML as an optional employer pass-through. The employer owes and must remit the full 0.88% contribution for 2026 (0.18% family plus 0.70% medical) on wages up to the $184,500 Social Security base (M.G.L. c. 175M ss. 6(a), 6(f), 7(d)). The employer may deduct from wages up to 100% of the family share and up to 40% of the medical share, a maximum of 0.46% (s. 6(c)). It may deduct less or nothing and remit the rest itself (458 CMR 2.05(5)(c)). DFML says employers \"may choose to cover a larger share.\" Employers with fewer than 25 employees owe no employer share but still remit 0.46%, which they may deduct in full or pay themselves. The engine's ma_employee_paid_leave_contribution computes (0.0018 + 0.007 x 0.4) x wages capped at the SS base, and its documentation says it assumes the employer withholds the maximum. For scenario_081, 175,002 x 0.0046 = $805.0092, so the engine exactly matches the legal maximum. Under the benchmark's prompt, \"mandatory employee state payroll taxes\", the legally mandatory employee amount is $0, unless one adopts the engine's assumption that the employer deducts the maximum. One flag on the cutoff rule: Acts 2026 c. 101 was approved June 12, 2026. It swaps the caps to 100% medical and 40% family, and its s. 45 literally applies that swap to taxable years beginning on or after Jan 1, 2026. DFML (after the cutoff) and the act's own s. 43 treat the swap as a 2027 change. If the literal 2026 reading held, the 2026 maximum would be 0.772% ($1,351.02)." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "(c)(1) For medical leave, an employer shall not deduct more than 40 per cent of the contribution required for an employee by subsection (a) from that employee's wages and shall remit the full contribution required under said subsection (a) to the trust fund.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "(2) For family leave, an employer may deduct not more than 100 per cent of the contribution required for an employee by subsection (a) from that employee's wages, and shall remit the full contribution required under subsection (a) to the trust fund.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "For each employee or covered contract worker, an employer or a covered business entity shall remit to the Family and Employment Security Trust Fund established in section 7 contributions in the form and manner as determined by the department.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section7", + "quote": "employers, covered business entities and self-employed individuals shall, unless subject to provisions under section 11, make contributions as required under section 6 and transmit those contributions to the trust fund in the manner determined by the director.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section1", + "quote": "''Contributions'', the payments made by an employer, a covered business entity, an employee or a self-employed individual to the Family and Employment Security Trust Fund, as required by this chapter.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section4", + "quote": "(ii) the employee's contribution amount and obligations under this chapter; (iii) the employer's contribution amount and obligations under this chapter;", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section8", + "quote": "An employer or covered business entity that fails or refuses to make contributions as required in section 6 shall be assessed an amount equal to its total annual payroll", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section11", + "quote": "(xii) providing that the cost to employees covered by a private plan shall not be greater than the cost charged to employees under the state program.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/SessionLaws/Acts/2026/Chapter101", + "quote": "SECTION 45. Sections 7, 13 through 15, inclusive, 21, 25 and 26, and subsection (kk) of section 6 of chapter 62 ... shall apply for taxable years beginning on or after January 1, 2026.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/SessionLaws/Acts/2026/Chapter101", + "quote": "shall provide guidance related to the changes in sections 25 and 26 and the impact for employers and employees in calendar year 2027", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "The employer or covered business entity in all instances shall be responsible for paying and remitting the remainder of any required contribution not lawfully deducted from the employee or covered contract worker.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity that opts to deduct a lower percentage of medical or family leave contributions than the maximum allowable deductions specified in 458 CMR 2.05(5)(a) and (b) shall still be required to remit the full amount", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "To do this, employers must withhold PFML contributions from employees' paychecks.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Employers must contribute at least the minimum share of the contribution described below; however, they may choose to cover a larger share of the contribution for some or all of their employees.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251110011839id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Up to 40% of the medical leave contribution can be withheld from a covered individual's wages (0.28% of eligible wages). Employers are responsible for contributing the remaining 60% (0.42% of eligible wages).", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260916051521id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "New Massachusetts legislation ( Chapter 101 of the Acts of 2026 ) shifts employer contributions from medical leave to family leave, effective January 1, 2027 .", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260825031701id_/https://www.mass.gov/info-details/pfml-workplace-poster-notices-and-rate-sheets-for-massachusetts-employers", + "quote": "Contribution rates for 2025 and 2026. Employers are responsible for sending contributions to DFML for all employees.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/ma/tax/payroll/paid_leave/ma_employee_paid_leave_contribution.py", + "quote": "assuming the employer withholds the maximum permitted employee share.", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None in the scenario_081 arithmetic. I recomputed 175,002 x (0.0018 + 0.007 x 0.4) = 805.0092, or $805.01. Wages are below the $184,500 base, which I confirmed in the 2025-11-03 Federal Register notice. The engine parameters I read are family_rate 0.0018, medical_rate 0.007 and medical_employee_share 0.4 from 2026-01-01, and SS cap 184_500 for 2026. The two alternatives also check: $1,351.02 under the literal c.101 reading (0.4 x 0.18% + 0.70%) and $1,540.02 at the full 0.88%.", + "Rate timing is confirmed. DFML's 2026 split (0.88% total = 0.18% family + 0.70% medical; employee up to 0.18% + 0.28%) appears in the 2025-11-10 Wayback capture of the rates page, which shows 'Last updated: October 1, 2025'. So it was published before the 2026-07-03 cutoff.", + "A nuance in the research's literal 2026 reading of Acts 2026 c.101. Sections 25 and 26 both amend 'line 33' of s.6. The current s.6 text has exactly three '40's (6(c)(1), 6(d), 6(e)(1)) and three '100's (6(c)(2), 6(d), 6(e)(2)), and only 6(d) contains both, so line 33 must be 6(d). Read sequentially and literally, s.25 turns 6(d)'s '40' into '100', and s.26 then strikes '100' 'each time it appears' on that line. That would give small employers 40%/40%, not a swap. The intended reading is clearly a swap. This affects only the alternative literal-2026 reading for employers with fewer than 25 employees, not the classification or the $805.01 check.", + "The malegislature.gov General Laws text of s.6, as I read it on 2026-10-05, still shows the pre-c.101 caps (40% medical, 100% family). That is a lag in the website's codification, not evidence of what the law says. Chapter 101 carries an emergency preamble and was 'Approved, June 12, 2026'." + ], + "citation_problems": [ + "The poster/rate-sheets page quote 'Contribution rates for 2026. Employers are responsible for sending contributions to DFML for all employees.' could not be confirmed verbatim. The 2026-08-25 Wayback capture I read says 'Contribution rates for 2025 and 2026.' The live page returns 403, and the browser pane refused to navigate to mass.gov, so the live wording may differ. 'Last updated: November 3, 2025' does match.", + "The live mass.gov URLs cited as fetched (the rates page and the 458 CMR 2 PDF) returned HTTP 403 to WebFetch and curl in this session. I confirmed every quote attributed to them only through Wayback captures of the same URLs: the rates page as captured 2025-11-10, 2026-06-17 and 2026-09-16, and the PDF as captured 2025-09-01 (PDF metadata subject 'Mass. Register #1486, 1/06/2023'). The two quotes credited to the live rates page ('Employers must contribute at least the minimum share...' and 'Small employers are responsible for sending the funds withheld...') appear verbatim in all three rates-page captures.", + "This is a selective quotation, not a misquotation. The research quotes the DFML rates-page Overview but leaves out the sentence just before its quote: 'To do this, employers must withhold PFML contributions from employees' paychecks.' That sentence appears in the 2025-11-10, 2026-06-17 and 2026-09-16 captures, and it is the strongest agency language pointing toward mandatory withholding. It should have been disclosed and addressed.", + "Two parts of the research's own analysis also go unaddressed. The definition of 'Contributions' in c.175M s.1 expressly includes payments made by 'an employee', and s.4(a)(ii) requires the employer to give each employee written notice of 'the employee's contribution amount and obligations under this chapter'. The research does not mention either. Neither makes a deduction mandatory, but both are relevant counter-evidence.", + "Every statutory quote in operative_text is verbatim at the cited malegislature.gov URLs: s.6(a), (c)(1), (c)(2), (d) as truncated, (f) with its ellipsis, s.7(d) and s.11(a)(1)(xii). So are the Acts 2026 c.101 s.45 quote (the ellipsis covers the c.62/c.63 cross-references) and the 458 CMR 2.05(5), (5)(c), (5)(d) and (7) quotes." + ], + "reasoning": "STATUTE LENS. I downloaded and read all eleven sections of M.G.L. c.175M from malegislature.gov. I searched every section for deduct, withh, 'employee shall', 'employees shall', payroll and contribut. No section says an employer 'shall deduct' or 'shall withhold', and none makes the employee liable to the fund. The only operative text on the employee portion is a set of ceilings. Section 6(c)(1) says an employer 'shall not deduct more than 40 per cent' of the medical contribution, and 6(c)(2) says it 'may deduct not more than 100 per cent' of the family contribution. Both add that the employer 'shall remit the full contribution'. Every liability provision names the employer side. Section 6(a) says the employer or covered business entity 'shall remit'. Section 7(d) lists who 'make contributions' as employers, covered business entities and self-employed individuals, and employees are left out. Section 8(g) assesses the employer that 'fails or refuses to make contributions', treats contributions as taxes under c.62C, and treats them as debts. Section 6(d) relieves small employers of the 'employer portion' but says they 'shall remit' 100% of family and 40% of medical. It does not require them to deduct those amounts. The only person the statute makes pay their own contributions is an electing self-employed individual (6(b)).\n\nREGULATION. 458 CMR 2.05(5), which I read in a Wayback capture of the official mass.gov PDF (the live URL returns 403), says the employer 'may deduct up to certain defined percentages'. It defines the employer's portion as the part 'not deducted from wages'. It makes the employer 'in all instances' responsible for the remainder. Subsection (5)(c) expressly covers an employer that 'opts to deduct a lower percentage' and still requires it to remit the full amount. Subsection (5)(d) allows different percentages for different groups. Subsection (7) bars an employer that 'failed to properly assess the allowable deduction' from recouping assessments from employees. Cornell LII, a secondary source, shows 2.05 was last amended in 2020, and I found no later amendment requiring deductions.\n\nCOUNTER-EVIDENCE CONSIDERED. Four things point the other way. The s.1 definition of 'Contributions' includes payments by 'an employee'. Section 4(a)(ii) requires notice of 'the employee's contribution amount and obligations'. The DFML Overview, in every capture from November 2025 to September 2026, says 'employers must withhold PFML contributions from employees' paychecks'. And Acts 2026 c.101 s.43 refers to 'tax burdens on employees'. These show that the law contemplates an employee contribution when the employer chooses to deduct one. None of them imposes a minimum deduction. The DFML sentence is non-binding summary language, and the same paragraph says employers 'may choose to cover a larger share... for some or all of their employees'. The same page tells small employers they 'may elect to cover some or all of the covered individuals' share'. Statute and regulation control, and both are permissive with a ceiling and no floor. The research's classification therefore holds. It should, however, have disclosed the 'must withhold' sentence and the s.1/s.4 language.\n\nENGINE. In the installed policyengine_us 2.15.17 source, I read ma_employee_state_payroll_tax.py (adds ma_employee_paid_leave_contribution), ma_employee_paid_leave_contribution.py ((family_rate + medical_rate * medical_employee_share) * ma_paid_leave_taxable_wages, with the documented assumption that the employer withholds the maximum), ma_paid_leave_taxable_wages.py (employment_income capped at the SS cap), ma_employer_paid_leave_contribution.py (headcount-gated employer side), the four paid_leave parameter YAMLs, and the SS cap of 184_500 for 2026. All match the research's description. $805.01 is the legal maximum the employer may deduct for scenario_081. The legally mandatory employee amount is $0.\n\nACTS 2026 c.101. I confirmed it was 'Approved, June 12, 2026', with an emergency preamble. Sections 25 and 26 swap the 40 and 100 figures in s.6. Section 43(a) refers to impacts in calendar year 2027. Section 45 applies ss.25-26 to 'taxable years beginning on or after January 1, 2026'. DFML's July 10, 2026 update (after the cutoff) says the change is effective January 1, 2027. Even under the swap, the provisions remain caps on what 'may' or 'shall not' be deducted, so the classification is the same for 2026 or 2027. Only the maximum amount changes." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "(c)(1) For medical leave, an employer shall not deduct more than 40 per cent of the contribution required for an employee by subsection (a) from that employee's wages and shall remit the full contribution", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "(2) For family leave, an employer may deduct not more than 100 per cent of the contribution required for an employee by subsection (a) from that employee's wages, and shall remit the full contribution", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "an employer employing less than 25 employees in the commonwealth shall not be required to pay the employer portion of premiums for family and medical leave; provided, however, that such employer shall remit, for each employee, 100 per cent of the family leave contribution and 40 per cent", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", + "quote": "(a) For each employee or covered contract worker, an employer or a covered business entity shall remit to the Family and Employment Security Trust Fund established in section 7 contributions", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section7", + "quote": "employers, covered business entities and self-employed individuals shall, unless subject to provisions under section 11, make contributions as required under section 6 and transmit those contributions to the trust fund", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section11", + "quote": "providing that the cost to employees covered by a private plan shall not be greater than the cost charged to employees under the state program", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section4", + "quote": "(ii) the employee's contribution amount and obligations under this chapter; (iii) the employer's contribution amount and obligations under this chapter", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "The employer or covered business entity in all instances shall be responsible for paying and remitting the remainder of any required contribution not lawfully deducted from the employee or covered contract worker.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity that opts to deduct a lower percentage of medical or family leave contributions than the maximum allowable deductions specified in 458 CMR 2.05(5)(a) and (b) shall still be required to remit the full amount", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "An employer or covered business entity that failed to properly assess the allowable deduction ... shall not recoup any assessment or cost of repayment through charges against employees or covered individual.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250901025442id_/https://www.mass.gov/doc/458-cmr-2-family-and-medical-leave/download", + "quote": "Contributions are the payments made to the Family and Employment Security Trust Fund established in M.G.L. c. 175M, \u00a7 7 by an employer, a covered business entity, an employee, a covered contract worker, or a self-employed individual electing coverage.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "To do this, employers must withhold PFML contributions from employees' paychecks. Employers are only required to send an employer contribution if they have 25 or more covered individuals in their workforce.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Employers must contribute at least the minimum share of the contribution described below; however, they may choose to cover a larger share of the contribution for some or all of their employees.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Small employers are responsible for sending the funds withheld from covered individuals' wages but are under no obligation to contribute themselves. However, they may elect to cover some or all of the covered individuals' share.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260617060228id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "Up to 40% of the medical leave contribution can be withheld from a covered individual's wages (0.28% of eligible wages). Employers are responsible for contributing the remaining 60% (0.42% of eligible wages).", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251110011839id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "2025 & 2026 ... Employers with 25 or more covered individuals must send to DFML a contribution of 0.88% of eligible wages ... Last updated: October 1, 2025", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260916051521id_/https://www.mass.gov/info-details/paid-family-and-medical-leave-employer-contribution-rates-and-calculator", + "quote": "New Massachusetts legislation (Chapter 101 of the Acts of 2026) shifts employer contributions from medical leave to family leave, effective January 1, 2027.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/SessionLaws/Acts/2026/Chapter101", + "quote": "SECTION 45. Sections 7, 13 through 15, inclusive, 21, 25 and 26 ... shall apply for taxable years beginning on or after January 1, 2026.", + "fetched_and_read": true + }, + { + "url": "https://malegislature.gov/Laws/SessionLaws/Acts/2026/Chapter101", + "quote": "shall provide guidance related to the changes in sections 25 and 26 and the impact for employers and employees in calendar year 2027", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251127060920id_/https://www.mass.gov/info-details/pfml-workplace-poster-notices-and-rate-sheets-for-massachusetts-employers", + "quote": "Contribution rates for 2025 and 2026. Employers are responsible for sending contributions to DFML for all employees.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base will be $184,500 for remuneration paid in 2026 and self-employment income earned in tax years beginning in 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/massachusetts/department-458-CMR/title-458-CMR-2.00", + "quote": "Amended by Mass Register Issue 1486, effective 1/6/2023 (secondary source, used only to confirm no later amendment of 458 CMR 2.00)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None for the engine amount. I read policyengine_us 2.15.17 myself. ma_employee_paid_leave_contribution = (family_rate 0.0018 + medical_rate 0.007 * medical_employee_share 0.4) * ma_paid_leave_taxable_wages, and taxable wages = min(max(0, employment_income), SS cap 184,500 for 2026). 175,002 x 0.0046 = 805.0092, which rounds to $805.01 and matches the engine. Wages are below the $184,500 base, so the cap does not bind.", + "The 2026 rates are confirmed. DFML's rates page is headed '2025 & 2026' and says 'Last updated: October 1, 2025' (Wayback copy of 2025-11-10). It gives 0.88% in total: 0.18% family, plus 0.70% medical implied by the 0.28%/0.42% split. Up to 0.18% + 0.28% = 0.46% may be withheld from the employee. The 2026-06-17 copy has the same text, apart from whitespace. The $184,500 base is in the Federal Register of 2025-11-03. All of these were published before 2026-07-03.", + "Small detail on the literal reading of Acts 2026 c. 101 s. 45: s. 25 replaces '40' with 100 and s. 26 replaces '100' with 40 in s. 6(c), 6(d) and 6(e). So the literal 2026 reading would also change what small employers must remit, to 40% of family plus 100% of medical (0.772%), not just the large-employer maximum deduction. The other researcher's 0.772% / $1,351.02 figure (175,002 x 0.00772 = 1,351.02) is correct arithmetic. The full 0.88% would be $1,540.02.", + "The DFML rates page itself shows a 0.46% employee maximum for both large and small employers. The small-employer section lists 'Up to 100% of the medical leave contribution' as 0.28%, because only 40% of medical is owed. This matches the engine applying 0.46% regardless of employer size." + ], + "citation_problems": [ + "DFML rate-sheets page: the other researcher quoted 'Contribution rates for 2026. Employers are responsible for sending contributions to DFML for all employees.' and dated the page 'last updated November 3, 2025'. The Wayback copy I read (captured 2025-11-27, 'Last updated: November 3, 2025') says 'Contribution rates for 2025 and 2026.' The second sentence matches. The first sentence may come from a later live version that I could not reach, since mass.gov returns 403 to WebFetch and curl and the Browser pane refused navigation. This does not affect the classification.", + "I could not fetch the live mass.gov copy of 458 CMR 2 (403). I verified every 458 CMR 2.05 quote against the Wayback copy of the same URL (captured 2025-09-01). Its PDF metadata reads 'Mass. Register #1486, 1/06/2023'. Cornell LII lists no amendment after 1/6/2023. All of the other researcher's 2.05(5), (5)(c), (5)(d) and (7) quotes match that text verbatim.", + "The other researcher left two things out of the agency guidance: DFML's overview sentence 'employers must withhold PFML contributions from employees' paychecks', and the page subtitle 'Most Massachusetts employers must make payroll withholdings'. These are the strongest 'must' language the agency uses. They should have been disclosed, though they do not overturn the classification (see reasoning).", + "I did not open the 2026 Word rate sheets or the 2026 employer notices. The Wayback availability API was rate-limiting me (HTTP 429)." + ], + "reasoning": "I tried to refute the classification through the administration lens and could not. Every operative quote the other researcher gave matches what I read in the primary sources: c. 175M ss. 6(a), 6(c)(1), 6(c)(2), 6(d), 6(f), 7(d) and 11(a)(1)(xii) on malegislature.gov, 458 CMR 2.05(5), (5)(c), (5)(d) and (7) in the official PDF via Wayback, and Acts 2026 c. 101 s. 45.\n\nWho is liable: the statute puts the remittance duty on the employer ('shall remit', s. 6(a); 'make contributions', s. 7(d)). On the employee share, the law sets only ceilings: 'shall not deduct more than 40 per cent' for medical and 'may deduct not more than 100 per cent' for family. It never orders a deduction. 458 CMR 2.05(5) defines the employer's portion as whatever is not deducted, and says the employer 'in all instances' pays the remainder. 2.05(5)(c) expressly covers an employer that 'opts to deduct a lower percentage' and still requires full remittance. 2.05(5)(d) allows different percentages for different groups. 2.05(7) bars an employer that failed to assess the deduction from recouping it from employees. That is the opposite of employee liability. No provision I read makes the employee liable to the state if nothing is deducted.\n\nThe counterpoints are textual only. 458 CMR 2.05(1)(a) lists 'an employee' among those who make contributions. s. 4(a) requires notice of 'the employee's contribution amount'. DFML's overview says 'employers must withhold PFML contributions from employees' paychecks'. These show that the program expects an employee share, but none of them sets a mandatory amount. The same DFML page then says employers 'may choose to cover a larger share of the contribution for some or all of their employees', and that small employers 'may elect to cover some or all of the covered individuals' share'. The binding regulation uses 'may deduct' and 'opts to deduct a lower percentage'. The 'must withhold' overview line is loose summary prose, contradicted by the page's own detailed text and by the regulation. So the employee share is an optional pass-through capped at 0.46% in 2026, with a legally mandatory employee amount of $0.\n\nEngine: I read the engine source this session. It documents 'assuming the employer withholds the maximum permitted employee share' and computes 0.46% x min(employment_income, 184,500). For scenario_081 that gives $805.01, the legal maximum, not a required amount.\n\nRate timing: the 0.88% rate, the 0.18%/0.70% split, the 40%/100% deduction caps and the $184,500 base were all published by November 2025. Acts 2026 c. 101 (approved June 12, 2026, an emergency law) swaps the caps. Its s. 45 literally applies ss. 25-26 to 'taxable years beginning on or after January 1, 2026', while its s. 43(a) refers to 'calendar year 2027'. DFML's statement that the swap takes effect January 1, 2027 is on a page last updated July 10, 2026, after the 2026-07-03 cutoff. The other researcher disclosed this ambiguity correctly. Either way, the swap changes only the cap, not the optional nature of the deduction.\n\nClassification confirmed: optional_employer_pass_through. refuted=false. The minor issues are a rate-sheet quote I could not match to the version I read, and the omission of DFML's 'must withhold' overview sentence." + } + ] + }, + { + "key": "NY", + "research": { + "program": "New York Disability Benefits Law (DBL) and Paid Family Leave (PFL) employee contributions (Workers' Compensation Law Article 9)", + "state": "NY", + "engine_variables": [ + "ny_employee_paid_family_leave_contribution", + "ny_employee_disability_benefits_contribution", + "ny_employee_state_payroll_tax", + "ny_paid_family_leave_taxable_wages", + "ny_disability_benefits_taxable_wages", + "employee_state_payroll_tax" + ], + "premium_liability": "The covered employer carries the liability. Under WCL s.211, a covered employer \"shall, with his or her own contributions and the contributions of his employees, provide\" DBL and PFL benefits. It does so by insuring with the State Insurance Fund or a private carrier, by self-insuring, or through a Board-accepted plan. The premium is owed by the employer to its carrier; it is not a contribution paid to the state. Under WCL s.210(1), the employer pays the DBL cost \"in excess of the contributions collected from his or her employees\". So if the employer collects nothing, the employer bears the whole cost. Employees owe nothing directly to the state or the carrier. Any employee money reaches the premium only through employer payroll deductions, which WCL s.209(4) authorizes but does not require. When the employer deducts, it acts as the employees' agent (WCL s.209(5)). For PFL, the employer must return any surplus over the premium (12 NYCRR 380-7.2(b)(3)). Penalties for failing to provide coverage fall on the employer (12 NYCRR 380-7.2(d)).", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "NY Workers' Compensation Law s.209(1)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "Every employee in the employment of a covered employer shall contribute to the cost of providing disability and after January first, two thousand eighteen, family leave benefits under this article, to the extent and in the manner herein provided.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209(3)(a) (DBL employee contribution)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "The contribution of each such employee to the cost of disability benefits ... shall be one-half of one per centum of the employee's wages ... but not in excess of sixty cents per week.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209(3)(b) (PFL maximum employee contribution set by DFS)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "On June first, two thousand seventeen and annually thereafter on September first, the superintendent of financial services shall set the maximum employee contribution ... No employer shall be required to fund any portion of the family leave benefit.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209(4) (employer authorized, not required, to collect via payroll deduction)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "Notwithstanding any other provision of law, the employer is authorized to collect from his or her employees, except as otherwise provided in any plan or agreement under ... section two hundred eleven of this article, the contribution ... through payroll deductions.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209(4), second sentence (late collection window)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "If the employer shall not make deduction for any payroll period he or she may thereafter, but not later than one month after payment of wages, collect such contribution through payroll deduction.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209(5) (employer acts as employees' agent)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "In collecting employee contributions through payroll deductions, the employer shall act as the agent of his or her employees and shall use the contributions only to provide disability and family leave benefits as required by this article.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209 (PFL contribution ceiling: per capita share of actual premium)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "In no event may the employee's annual contribution for family leave exceed his or her per capita share of the actual annual premium charged for the same year and must be determined consistent with the principle that employees should pay ...", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.209 (DBL ceiling restated)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "In no event may the employee's weekly contribution for disability premium exceed one-half of one per centum of the employee's wages paid to him or her, but not in excess of sixty cents per week.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.210(1) (employer pays the cost beyond what it collects)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/210", + "quote": "Every covered employer shall, on and after January first, nineteen hundred fifty, contribute the cost of providing disability benefits in excess of the contributions collected from his or her employees", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.211 lead-in (employer must provide benefits)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "A covered employer shall, with his or her own contributions and the contributions of his employees, provide disability and after January first, two thousand eighteen, family leave benefits to his or her employees ...", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.211(4)-(5) (Board-accepted plans and collective agreements)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "does not require contributions of any employee or of any class or classes of employees in excess of the statutory amount provided in section two hundred nine of this article...except by agreement", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.201(4) (public employers excluded from 'employer')", + "url": "https://www.nysenate.gov/legislation/laws/WKC/201", + "quote": "but does not include the state, a municipal corporation, local governmental agency, other political subdivisions or public authority.", + "fetched_and_read": true + }, + { + "citation": "NY WCL s.202(1) (covered employer: one or more employees on at least 30 days)", + "url": "https://www.nysenate.gov/legislation/laws/WKC/202", + "quote": "An employer who has in employment, after June thirtieth, nineteen hundred sixty-one, one or more", + "fetched_and_read": true + }, + { + "citation": "12 NYCRR 380-7.2(b)(1) (PFL: employer 'may deduct'); read via Cornell LII copy because govt.westlaw.com could not be fetched", + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "A covered employer may deduct employee contributions prior to the effective date of the policy and prior to an employee's eligibility for paid family leave.", + "fetched_and_read": true + }, + { + "citation": "12 NYCRR 380-7.2(c) (weekly maximum; limit on catch-up withholding); via Cornell LII copy", + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "The weekly contribution from an employee is the maximum per week that can be withheld. ... an employer's failure to withhold may not be recovered by withholding larger than the maximum employee contribution at a later date.", + "fetched_and_read": true + }, + { + "citation": "12 NYCRR 380-7.2(b)(3) (surplus returned to employees); via Cornell LII copy", + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "A covered employer shall use his or her employees' contributions to provide PFL benefits to employees and shall promptly return to employees any surplus in employee contributions that exceed the annual premium.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employer-disability-benefits.jsp", + "quote": "An employer is allowed, but not required, to collect contributions from its employees to offset the cost of providing disability and Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employer-disability-benefits.jsp", + "quote": "However, any employee who is receiving or is entitled to receive old age insurance benefits under title two of the social security act, shall be exempt upon filing Form DB130 with the employer and the chair.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employer-disability-benefits.jsp", + "quote": "Paid Family Leave coverage can only be waived if the employee is regularly scheduled for less than 20 hours per week and will not work 175 days in a year, or ...", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employee-disability-benefits.jsp", + "quote": "Your employer is allowed, but not required, to take a contribution from you to offset the cost of providing disability benefits.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/cost", + "quote": "An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The maximum employee contribution is the maximum amount a covered employer is authorized to collect from each of its employees to fund Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The 2026 PFL rate will be set at 0.432% of an employee's wages each pay period up to and not to exceed an annual maximum employee contribution of $411.91.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "Employers shall collect employee contributions consistent with this Decision.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/forms/PFLDocs/pfl-pay-deduction-notice-2026.pdf", + "quote": "The 2026 payroll contribution is 0.432% of an employee's wages each pay period and is capped at an annual maximum of $411.91.", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/PressRe/paid-family-leave-2026.jsp", + "quote": "NYS Paid Family Leave is insurance, fully paid for by employees, that provides job-protected, paid time off care for family in times of need.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/2026", + "quote": "New York State Paid Family Leave is insurance that may be funded by employees through payroll deductions.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/TheBoard/disability-benefits-fact-sheet.pdf", + "quote": "An employer may collect a contribution from its employees to offset the disability benefits insurance premium. The contribution is up to one-half of one percent of wages, up to 60 cents a week.", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/forms/db271s.pdf", + "quote": "Employers may collect a maximum contribution of 60 cents/week to offset the insurance premium (unless the additional contribution is part of an approved plan).", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employee-disability-benefits.jsp", + "quote": "There are some accepted disability benefits plans under which you are required to contribute more than 60 cents per week, but only by agreement and provided your contributions are reasonably related to the value of the benefits.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Yes, for both programs. Under the statute, WCL s.209(4) \"authorizes\" the employer to collect but does not require it, and WCL s.210(1) has the employer pay the cost \"in excess of the contributions collected\". For PFL, s.209(3)(b) says \"No employer shall be required to fund any portion\" of the benefit, which bars a mandate but does not bar voluntary funding. For PFL, 12 NYCRR 380-7.2(b)(1) says a covered employer \"may deduct\". Agency guidance is explicit. The WCB employer DB/PFL page says the employer \"is allowed, but not required, to collect contributions\" for both DBL and PFL. paidfamilyleave.ny.gov/cost says \"An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.\" DFS calls the 2026 maximum the most an employer is \"authorized to collect\".", + "rate_2026": { + "total_rate": "PFL: the DFS premium rate for 2026 coverage is 0.432% of each employee's wages per pay period (a +11.31% increase over 2025). DFS sets it so that employee contributions can cover the full PFL premium. DBL: no statutory total rate. The carrier sets the premium for each employer, the employee share is fixed by statute, and the employer pays the balance (WCL s.210).", + "max_employee_share": "PFL: 0.432% of wages per pay period, capped at $411.91 a year (DFS decision of Aug 29, 2025). PFL contributions also cannot exceed the employee's per capita share of the actual annual premium (WCL s.209). DBL: 0.5% of wages, capped at $0.60 a week (WCL s.209(3)(a)). That is $31.20 over 52 weekly pay periods; in a year with 53 weekly paydays it could reach $31.80. More is allowed only by agreement under a Board-accepted plan (s.211(4)-(5)).", + "wage_base_or_cap": "PFL: the $411.91 cap equals 0.432% of the 2026 NYSAWW of $1,833.63 a week times 52, which is $95,348.76 a year (0.00432 x 95,348.76 = $411.9066, rounding to $411.91). The WCB press release calls $411.91 '0.432% of the NYSAWW'. DBL: the $0.60 weekly cap binds once weekly wages reach $120 (0.005 x 120 = 0.60). There is no annual wage base; the cap is weekly.", + "sources": [ + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "published": "2025-08-29 (decision line reads 'Dated: August 29, 2025')" + }, + { + "url": "https://www.wcb.ny.gov/content/main/forms/PFLDocs/pfl-pay-deduction-notice-2026.pdf", + "published": "PDF created 2025-09-03, modified 2025-09-05 (form code PFL-PayDeduction-notice-2026 9-25)" + }, + { + "url": "https://apps.wcb.ny.gov/content/main/PressRe/paid-family-leave-2026.jsp", + "published": "2025-10-30 (date on the press release)" + }, + { + "url": "https://paidfamilyleave.ny.gov/2026", + "published": "no date on the page; read 2026-10-05" + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "published": "DBL $0.60 a week / 0.5% has been in the statute since 1950. The NY Senate page shows the section text as last updated April 8, 2016 and still current when read on 2026-10-05." + } + ], + "published_before_2026_07_03": "Yes. The 2026 PFL rate and cap were published by DFS on 2025-08-29 and in WCB materials in September and October 2025. The DBL rate and cap are long-standing statute (s.209(3)(a)) and were unchanged in the text read on 2026-10-05." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_082 (PFL)", + "engine_amount": 411.91, + "law_amount_at_max_employee_share": 411.91, + "matches_within_1_dollar": true, + "note": "Law maximum: 0.432% of wages each pay period until the $411.91 annual cap. Uncapped: 0.00432 x 100,195 = $432.84, which exceeds $411.91, so the maximum is $411.91. Because the cap is cumulative over the year, the per-pay-period rule gives the same annual total as the engine's min(0.00432 x annual wages, 411.91). Engine re-run this session (2026, NY, $100,195 wages) returned 411.91. The engine does not compute a SAWW cap; it applies the $411.91 parameter directly. The task's '$95,350.72' figure does not appear in the engine source or the agency sources I read. 1,833.63 x 52 = $95,348.76, and either figure rounds to roughly $411.91. This is the law's MAXIMUM. The law's minimum employee share is $0 if the employer chooses not to deduct." + }, + { + "scenario_id": "scenario_082 (DBL)", + "engine_amount": 31.2, + "law_amount_at_max_employee_share": 31.2, + "matches_within_1_dollar": true, + "note": "Weekly wages are about 100,195 / 52 = $1,926.83. 0.5% of that is $9.63, above the $0.60 cap, so the cap binds every week: 52 x $0.60 = $31.20. Engine: min(0.005 x 100,195 = $500.98, 31.20) = $31.20; the re-run returned 31.2. With 53 weekly paydays (for example a Thursday payroll in 2026) the law maximum would be $31.80, still within $1. This is the law's MAXIMUM; the minimum is $0 if the employer does not deduct." + }, + { + "scenario_id": "scenario_082 (NY total, ny_employee_state_payroll_tax)", + "engine_amount": 443.11, + "law_amount_at_max_employee_share": 443.11, + "matches_within_1_dollar": true, + "note": "$411.91 + $31.20 = $443.11. Engine re-run returned 443.11002. The match holds only under the assumption that the employer deducts the maximum permitted for both programs. Under NY law the amount anywhere from $0 to $443.11 (or more under an agreed plan) is consistent with the facts the prompt gives." + } + ], + "engine_documentation": "I read these files this session under policyengine_us 2.15.17.\n\n**Variables:**\n- `ny_employee_paid_family_leave_contribution`: label \"New York employee paid family leave contribution\"; documentation \"Employee-side New York Paid Family Leave contribution.\" The formula is min(employee_rate x ny_paid_family_leave_taxable_wages, annual_max_contribution).\n- `ny_employee_disability_benefits_contribution`: documentation \"Employee-side New York Disability Benefits withholding, annualized from the statutory weekly cap.\" The formula is min(employee_rate x ny_disability_benefits_taxable_wages, annual_max_contribution).\n- Both taxable-wage variables equal max(0, employment_income). Their documentation says they include \"federal pre-tax payroll deductions\".\n- `ny_employee_state_payroll_tax` adds the two contributions. Its documentation is \"Employee-side New York payroll-funded contributions, including Paid Family Leave and Disability Benefits withholding.\"\n- `ny_employee_state_payroll_tax` is listed in `EMPLOYEE_STATE_PAYROLL_TAX_COMPONENTS`, which `employee_state_payroll_tax` adds up (variables/gov/states/tax/payroll/employee_state_payroll_tax.py). That variable's documentation reads \"Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules.\"\n\n**Parameters:**\n- PFL `employee_rate`: 0 from 0000-01-01, 0.00432 from 2026-01-01.\n- PFL `annual_max_contribution`: 0 from 0000-01-01, 411.91 from 2026-01-01.\n- The PFL parameters cite WCL s.209 and the WCB 2026 PFL payroll deduction notice PDF.\n- DBL `employee_rate`: 0.005 from 0000-01-01.\n- DBL `annual_max_contribution`: 31.2 from 0000-01-01.\n- The DBL parameters cite WCL s.209 and the WCB employee disability benefits page.\n\n**What the NY files do not model:**\n- No NY file states an assumption that the employer withholds the maximum (grep for 'assum' and 'maximum permitted' found nothing).\n- No file models employer choice, plan or collective-agreement contributions, public-employer exclusion, waivers, the DB-130 exemption, or proration across multiple employers.\n\n**Test:** tests/core/test_payroll_contributions.py has an NY case expecting PFL 411.91, DBL 31.2 and a total of 443.11.\n\nI did not read the Minnesota variable.", + "caveats": [ + "Classification is the same for PFL and DBL: both are optional employer pass-throughs. The difference is in default design. PFL is designed to be paid entirely by employees: s.209(3)(b) bars requiring employers to fund it, and DFS sets the rate to cover the full premium. DBL is cost-shared: the employee pays at most 0.5% up to $0.60 a week and the employer pays the rest (s.210). For both, collection is at the employer's discretion: s.209(4) 'authorized', and WCB says 'allowed, but not required'.", + "Textual tension. WCL s.209(1) says every employee 'shall contribute ... to the extent and in the manner herein provided'. The DFS 2026 decision says 'Employers shall collect employee contributions consistent with this Decision'. The WCB press release calls PFL 'fully paid for by employees'. The operative mechanism, however, is permissive: s.209(4) 'is authorized to collect' and 12 NYCRR 380-7.2(b)(1) 'may deduct'. The WCB employer page explicitly says collection is 'allowed, but not required'. I read 'shall collect consistent with this Decision' as limiting how much may be collected, not as a mandate to collect.", + "Benchmark implication. The prompt defines payroll_tax to include 'mandatory employee state payroll taxes'. Under NY law, the DBL and PFL employee contributions are not mandatory withholdings. The engine's $443.11 is the maximum an employer may collect, so it is correct only if one assumes the employer deducts the maximum, and the prompt does not state that assumption. The NY engine files do not state it either. The cross-state aggregator documentation labels these contributions 'mandatory', which conflicts with the NY primary sources above.", + "Board-accepted plans and collective bargaining agreements (WCL s.211(4)-(5)): employer and employee obligations follow the plan or agreement. Employee contributions may exceed the statutory amounts only 'by agreement' and when reasonably related to the value of the benefits. Such a plan may also require less or nothing. s.209(4) defers to these plans.", + "Self-insured employers (s.211(3)) and employers insured through the State Insurance Fund or private carriers: the choice of funding method does not change the discretionary nature of employee deductions.", + "Public employers: WCL s.201(4) excludes the state, municipal corporations, local governmental agencies, other political subdivisions and public authorities from 'employer'. Public-sector workers therefore have no DBL or PFL deduction unless coverage is extended another way. The PFL site says requirements for other employer types differ; I did not fetch the public-employer opt-in provisions. The prompt does not say whether the employer is public or private.", + "Small employers: there is no small-employer exemption. Per s.202, which I read in summary plus a short quote, a private employer with one or more employees on each of at least 30 days in a calendar year is covered. Domestic or personal employers in a private home are covered only for workers who meet an hours threshold (about 20 hours a week, per the fetched summary).", + "Employee-level exemptions and waivers reduce or eliminate the deduction:", + "- DBL: an employee receiving or entitled to Social Security old-age benefits may be exempt by filing Form DB-130 (WCB employer page).", + "- PFL: may be waived by employees who are regularly scheduled under 20 hours a week and will not work 175 days in a year, or who work 20+ hours but won't be employed 26 consecutive weeks.", + "- s.201(5) excludes some categories of worker from 'employee', for example clergy, an executive officer who owns all of a corporation's stock, and certain professional or teaching roles at religious, charitable or educational institutions.", + "Multiple jobs (DBL): combined contributions across jobs may not exceed $0.60 a week, and the employee may ask the employers to prorate (WCB pages). The engine does not model this.", + "The DBL cap is weekly, not annual. The engine's annual min(0.5% x annual wages, $31.20) can differ from the law's maximum for workers with uneven weekly wages, for example some weeks under $120. It can also differ in years with 53 weekly paydays (law maximum $31.80). Neither difference matters for scenario_082.", + "Two further limits on PFL: the employee's contribution may not exceed the per capita share of the actual annual premium (s.209), and employers must refund any surplus over the premium (12 NYCRR 380-7.2(b)(3)).", + "Catch-up deductions: s.209(4) lets an employer that skipped a deduction collect it within one month after paying the wages. 12 NYCRR 380-7.2(c) bars recovering missed PFL withholding by later withholding above the weekly maximum.", + "Source limits:", + "- 12 NYCRR 380-7.2 was read through the Cornell LII copy, because the official govt.westlaw.com NYCRR site could not be fetched. LII's source note says the section was adopted in the NY State Register on July 19, 2017.", + "- The WCL text was read through WebFetch of nysenate.gov, because direct curl hit a Cloudflare challenge. Quotes are verbatim as returned by targeted extraction prompts.", + "- Agency pages and PDFs were checked verbatim with curl and text extraction.", + "The paidfamilyleave.ny.gov/cost page, which holds the 'employer may choose to pay' sentence, still shows 2025 figures ($354.53, NYSAWW $1,757.19), so it is out of date. Its 'may choose to pay' statement agrees with the WCB employer page and the statute.", + "A secondary bill tracker (billtrack50, NY A00084) describes a proposal to raise the DBL employee contribution from 2027. I did not verify it against a primary source, and it would not affect 2026 in any case." + ], + "confidence": "high", + "summary": "The employee shares the engine counts for both NY DBL and NY PFL are optional employer pass-throughs, not mandatory withholdings.\n\n**Who owes the cost.** The covered employer must provide the coverage by insuring with the State Insurance Fund or a private carrier, self-insuring, or using a Board-accepted plan (WCL s.211), and it owes the premium to its carrier.\n\n**Employee share.** WCL s.209(4) \"authorizes\" the employer to collect the employee share by payroll deduction. Agency guidance says this directly: the WCB employer page says the employer \"is allowed, but not required, to collect contributions\" for both programs, and paidfamilyleave.ny.gov/cost says \"An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.\" For DBL, the employer pays whatever it does not collect (s.210). The 2026 caps an employer may collect are:\n- **PFL:** 0.432% of wages per pay period, up to $411.91 a year (DFS decision dated Aug 29, 2025; equal to 0.432% x NYSAWW $1,833.63 x 52).\n- **DBL:** 0.5% of wages, up to $0.60 a week (statutory since 1950).\n\n**Engine check.** For scenario_082 ($100,195 wages) the engine gives PFL $411.91 and DBL $31.20, a total of $443.11. I re-ran it this session and got the same figures, which exactly equal the law's maximum employee shares. So the match holds only if the employer deducts the maximum, an assumption the NY engine files do not state. The engine's cross-state aggregator also calls these contributions \"mandatory\".\n\n**Caveats that change who pays:**\n- Collective-agreement or Board-accepted plans, where contributions may be higher by agreement, lower, or zero.\n- Public employers, which are excluded from \"employer\".\n- The DB-130 exemption for Social Security old-age beneficiaries, and PFL waivers for short-hours or short-tenure employees.\n- Proration across multiple jobs for DBL." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "Notwithstanding any other provision of law, the employer is authorized to collect from his or her employees, except as otherwise provided in any plan or agreement under the provisions of subdivisions four or five of section two hundred eleven of this article", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "If the employer shall not make deduction for any payroll period he or she may thereafter, but not later than one month after payment of wages, collect such contribution through payroll deduction.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "Every employee in the employment of a covered employer shall contribute to the cost of providing disability and after January first, two thousand eighteen, family leave benefits under this article, to the extent and in the manner herein provided.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "must be determined consistent with the principle that employees should pay the total costs of family leave premium. In no event may the employee's weekly contribution for disability premium exceed one-half of one per centum", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "the superintendent of financial services shall set the maximum employee contribution, using sound actuarial principles and the reports provided in section two hundred eight of this article. No employer shall be required to fund any portion of the family leave benefit.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/210", + "quote": "Every covered employer shall, on and after January first, nineteen hundred fifty, contribute the cost of providing disability benefits in excess of the contributions collected from his or her employees", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "A covered employer shall, with his or her own contributions and the contributions of his employees, provide disability and after January first, two thousand eighteen, family leave benefits to his or her employees", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "does not require contributions of any employee or of any class or classes of employees in excess of the statutory amount provided in section two hundred nine, subdivision three, except by agreement", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/213", + "quote": "such employer shall be fully and directly liable to each of his or her employees for the payment of benefits provided by this article.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/201", + "quote": "but does not include the state, a municipal corporation, local governmental agency, other political subdivisions or public authority.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/202", + "quote": "An employer who has in employment, after June thirtieth, nineteen hundred sixty-one, one or more employees on each of at least thirty days in any calendar year, shall be a covered employer", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "A covered employer may be responsible for payment of premium prior to collection of all employee contributions for a policy year.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "an employer's failure to withhold may not be recovered by withholding larger than the maximum employee contribution at a later date.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-2.5", + "quote": "as long as the contributions to the cost of family leave benefits, pursuant to section 209 of the Workers' Compensation Law and required by the employer of like employees, have been paid", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-2.6", + "quote": "An employee as described in subdivision (a) of this section who elects not to enter into a waiver shall make regular family leave benefit contributions for the full duration of his or her employment with the covered employer", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-2.4", + "quote": "An employer is permitted, but not required, to collect the weekly employee contribution on July 1, 2017 for paid family leave coverage beginning on January 1, 2018.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/11-NYCRR-363.3", + "quote": "Maximum employee contribution means the maximum amount a covered employer is authorized to collect from each of its employees to fund the family leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/11-NYCRR-363.7", + "quote": "Collection of the maximum employee contribution shall be done in accordance with the requirements set forth in Workers' Compensation Law section 209.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employer-disability-benefits.jsp", + "quote": "An employer is allowed, but not required, to collect contributions from its employees to offset the cost of providing disability and Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employee-disability-benefits.jsp", + "quote": "Your employer is allowed, but not required, to take a contribution from you to offset the cost of providing disability benefits.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/cost", + "quote": "An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/cost", + "quote": "Paid Family Leave is not optional for eligible employees. Coverage can only be waived if:", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/2026", + "quote": "New York State Paid Family Leave is insurance that may be funded by employees through payroll deductions.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The 2026 PFL rate will be set at 0.432% of an employee's wages each pay period up to and not to exceed an annual maximum employee contribution of $411.91.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "If an employee's contributions reach the annual maximum employee contribution of $411.91 before the end of the calendar year, the employee shall not be liable for any additional contributions for that year.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The maximum employee contribution is the maximum amount a covered employer is authorized to collect from each of its employees to fund Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/forms/PFLDocs/pfl-pay-deduction-notice-2026.pdf", + "quote": "The 2026 payroll contribution is 0.432% of an employee's wages each pay period and is capped at an annual maximum of $411.91.", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/PressRe/paid-family-leave-2026.jsp", + "quote": "Beginning January 1, 2026, employees will contribute 0.432% of their gross wages per pay period to pay for Paid Family Leave, with a maximum annual contribution of $411.91 (0.432% of the NYSAWW).", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/TheBoard/disability-benefits-fact-sheet.pdf", + "quote": "An employer may collect a contribution from its employees to offset the disability benefits insurance premium. The contribution is up to one-half of one percent of wages, up to 60 cents a week.", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/forms/db271s.pdf", + "quote": "Employers may collect a maximum contribution of 60 cents/week to offset the insurance premium (unless the additional contribution is part of an approved plan).", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "The task text calls the PFL cap '0.432% capped at the $95,350.72 annualized SAWW'. That figure is not supported. 2026 NYSAWW $1,833.63 x 52 = $95,348.76, and $411.91 / 0.00432 = $95,349.54. The engine never computes a SAWW cap; it applies the $411.91 parameter (annual_max_contribution.yaml) directly. The discrepancy does not change the $411.91 result.", + "DBL: the law caps the contribution at $0.60 per week (WCL s.209(3)(a), (5)). The engine instead uses an annual cap of $31.20 = 52 x $0.60. January 1, 2026 is a Thursday, so a weekly Thursday payroll has 53 paydays in 2026, and the legal maximum there is $31.80. That is within $1 of the engine for scenario_082. The annual min() can also differ from the law for workers with uneven weekly wages, for example weeks under $120.", + "Engine amounts for scenario_082 match the law's maximum employee shares: PFL min(0.00432 x 100,195 = 432.84, 411.91) = $411.91 and DBL min(500.98, 31.20) = $31.20, total $443.11. The installed test tests/core/test_payroll_contributions.py expects 411.91 / 31.2 / 443.11. The legal minimum is $0 if the employer chooses not to collect (WCL s.209(4); WCB 'allowed, but not required').", + "Rate provenance: the 2026 PFL rate (0.432%) and cap ($411.91) come from the DFS decision dated August 29, 2025. They were restated in the WCB deduction-notice PDF (created 2025-09-03, modified 2025-09-05) and the WCB press release of October 30, 2025. All of these predate 2026-07-03. The DBL 0.5% / $0.60-a-week figures are long-standing statute; the nysenate.gov page shows a 2016-04-08 revision of s.209." + ], + "citation_problems": [ + "No quote in the research failed verbatim confirmation. I checked every WCL quote (ss.201, 202, 209, 210, 211) with targeted WebFetch extraction on nysenate.gov, because curl and the browser pane are blocked by Cloudflare. So the statute text comes from model-extracted page text, not raw HTML. The s.211 'in excess of the statutory amount provided in section two hundred nine of this article ... except by agreement' string was confirmed as present.", + "Characterization, not a quote failure: the research cites 12 NYCRR 380-7.2(b)(1) ('A covered employer may deduct employee contributions prior to the effective date of the policy and prior to an employee's eligibility') as the PFL 'may deduct' rule. Its scope is timing: it permits deducting before the policy takes effect and before the employee is eligible. It does not by itself grant general discretion over whether to deduct. The general discretion rests on WCL s.209(4) 'is authorized to collect', 11 NYCRR 363.3(i), and the WCB and PFL agency statements.", + "The PFL ceiling quote and the restated DBL ceiling quote are cited as 's.209'. Both are in s.209(5). That is accurate but imprecise.", + "The research omits contrary-leaning primary text that a reviewer should see:\n- 12 NYCRR 380-2.6(b) and (d): an employee whose waiver is revoked 'shall be obligated to begin making contributions', and a non-waiving employee 'shall make regular family leave benefit contributions'.\n- The DFS 2026 decision: 'the employee shall not be liable for any additional contributions' after the cap.\n- paidfamilyleave.ny.gov/cost lists 'Collecting employee payroll contributions' among employer responsibilities and says PFL 'is not optional for eligible employees'. That statement is about coverage, not deductions.\n\nNone of these overrides the permissive collection mechanism, but the research understates the tension." + ], + "reasoning": "I read the statute and rules myself, and they support the classification for both programs: the employee share is an optional employer pass-through, not a mandatory withholding.\n\n**Who owes the premium.** The employer does.\n- WCL s.211 requires the covered employer to provide DBL and PFL benefits through the State Fund, a carrier, self-insurance or an accepted plan.\n- s.213 makes a non-complying employer 'fully and directly liable' for the benefits.\n- 12 NYCRR 380-7.2(b)(2) contemplates the employer paying the premium before it collects any employee contributions.\n- s.210(1) and (3) put the DBL cost 'in excess of the contributions collected' on the employer.\n\n**How the employee share reaches the premium.** Only through the employer's payroll deduction, which the law permits but does not require.\n- s.209(4) says the employer 'is authorized to collect' by payroll deduction. If it skips a pay period, it may catch up only within one month after paying the wages.\n- 12 NYCRR 380-7.2(c) bars recovering missed PFL withholding by later withholding above the weekly maximum. So contributions the employer does not collect are simply never owed.\n- 11 NYCRR 363.3(i), the DFS regulation, defines the PFL maximum employee contribution as the most 'a covered employer is authorized to collect'. 363.7(b) defers collection to s.209.\n- 12 NYCRR 380-2.5(d) refers to contributions 'required by the employer of like employees', which treats requiring them as the employer's choice.\n- The agencies say it directly. The WCB employer page says collection is 'allowed, but not required' for both DB and PFL. The WCB employee page says the same for DB. paidfamilyleave.ny.gov/cost says 'An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.' DFS calls the 2026 maximum what an employer is 'authorized to collect'.\n\n**Where 'shall deduct' or 'shall withhold' language would appear, it does not.** I searched Article 9 for it. s.209 says 'authorized'. s.220 (penalties) has nothing on employee deductions. s.213 addresses only employer liability.\n\n**Contrary text.** Some language frames the employee as obligated:\n- s.209(1): every employee 'shall contribute ... to the extent and in the manner herein provided'.\n- 12 NYCRR 380-2.6(b) and (d): employees 'shall make regular' PFL contributions, and become 'obligated' once the employer notifies them.\n- The DFS decision: 'Employers shall collect employee contributions consistent with this Decision', and the employee is 'not liable' for more once the cap is reached.\n- The WCB press release says employees 'will contribute' and calls PFL 'fully paid for by employees'.\n\nThese describe the employee side, and the default design: PFL is meant to be fully employee-funded (s.209(5); 'No employer shall be required to fund'). But 'the manner herein provided' is employer collection, which s.209(4) makes permissive. The official PFL site expressly allows the employer to absorb the cost. No primary source makes the employee liable to the state or the carrier independent of the employer collecting, and none requires the employer to withhold.\n\n**DBL versus PFL.** For DBL the case is even clearer. The employee share is capped at $0.60 a week, the employer pays the balance by statute, and both the WCB fact sheet and Form DB-271S say employers 'may collect'.\n\n**Engine.** The engine counts the maximum permitted share for both programs: PFL $411.91 + DBL $31.20 = $443.11 for scenario_082, which equals the legal maxima. The NY variables and parameters state no 'employer withholds the maximum' assumption; a grep for 'assum' and 'maximum permitted' found nothing. Meanwhile the employee_state_payroll_tax aggregator's documentation labels these 'mandatory', which NY primary law does not support.\n\n**Verdict.** The quotes check out and the classification stands. The research overstates the reach of 380-7.2(b)(1) and omits the 380-2.6 and DFS 'liable' language, but neither changes the result." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "optional_employer_pass_through", + "evidence": [ + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "the employer is authorized to collect from his or her employees", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "he or she may thereafter, but not later than one month after payment of wages, collect such contribution", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "the employer shall act as the agent of his or her employees", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "No employer shall be required to fund any portion of the family leave benefit.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "Every employee in the employment of a covered employer shall contribute to the cost of providing disability and after January first, two thousand eighteen, family leave benefits under this article, to the extent and in the manner herein provided.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/209", + "quote": "shall be one-half of one per centum of the employee's wages paid to him or her on and after July first, nineteen hundred fifty, but not in excess of sixty cents per week.", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/210", + "quote": "contribute the cost of providing disability benefits in excess of the contributions collected from his or her employees", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "A covered employer shall, with his or her own contributions and the contributions of his employees, provide disability and after January first, two thousand eighteen, family leave benefits", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/211", + "quote": "in excess of the statutory amount provided in section two hundred nine, subdivision three, except by agreement and provided the contribution", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "Unless otherwise specified in this Part or section 209 of the Workers' Compensation Law, an employer's failure to withhold may not be recovered by withholding larger than the maximum employee contribution at a later date.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "A covered employer may deduct employee contributions prior to the effective date of the policy and prior to an employee's eligibility for paid family leave.", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-york/12-NYCRR-380-7.2", + "quote": "shall promptly return to employees any surplus in employee contributions that exceed the annual premium.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employer-disability-benefits.jsp", + "quote": "An employer is allowed, but not required, to collect contributions from its employees to offset the cost of providing disability and Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/DisabilityBenefits/employee-disability-benefits.jsp", + "quote": "Your employer is allowed, but not required, to take a contribution from you to offset the cost of providing disability benefits.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/cost", + "quote": "An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/employer-responsibilities-and-resources", + "quote": "Ensure that you are collecting the employee contributions that pay for this insurance.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The 2026 PFL rate will be set at 0.432% of an employee's wages each pay period up to and not to exceed an annual maximum employee contribution of $411.91.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "The maximum employee contribution is the maximum amount a covered employer is authorized to collect from each of its employees to fund Paid Family Leave benefits.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "the employee shall not be liable for any additional contributions for that year.", + "fetched_and_read": true + }, + { + "url": "https://www.dfs.ny.gov/apps-and-licensing/health-insurers/pfl-rate-decision-2026-page", + "quote": "Employers shall collect employee contributions consistent with this Decision.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/forms/PFLDocs/pfl-pay-deduction-notice-2026.pdf", + "quote": "The 2026 payroll contribution is 0.432% of an employee's wages each pay period and is capped at an annual maximum of $411.91. Employees earning less than the New York State Average Weekly Wage ($1,833.63 per week)", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/PressRe/paid-family-leave-2026.jsp", + "quote": "with a maximum annual contribution of $411.91 (0.432% of the NYSAWW).", + "fetched_and_read": true + }, + { + "url": "https://paidfamilyleave.ny.gov/2026", + "quote": "New York State Paid Family Leave is insurance that may be funded by employees through payroll deductions.", + "fetched_and_read": true + }, + { + "url": "https://www.wcb.ny.gov/content/main/TheBoard/disability-benefits-fact-sheet.pdf", + "quote": "An employer may collect a contribution from its employees to offset the disability benefits insurance premium. The contribution is up to one-half of one percent of wages, up to 60 cents a week.", + "fetched_and_read": true + }, + { + "url": "https://apps.wcb.ny.gov/content/main/forms/db271s.pdf", + "quote": "Employers may collect a maximum contribution of 60 cents/week to offset the insurance premium (unless the additional contribution is part of an approved plan).", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/201", + "quote": "but does not include the state, a municipal corporation, local governmental agency, other political subdivisions or public authority", + "fetched_and_read": true + }, + { + "url": "https://www.nysenate.gov/legislation/laws/WKC/202", + "quote": "on each of at least thirty days in any calendar year", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None of the engine amounts is wrong as a law maximum. I re-ran policyengine-us 2.15.17 (version confirmed from the dist-info) for NY 2026 with $100,195 employment income and got: PFL $411.91, DBL $31.20, ny_employee_state_payroll_tax $443.11002, employee_state_payroll_tax $443.11002.", + "PFL check: 0.00432 x 100,195 = $432.84, above the cap, so $411.91. DBL check: 0.005 x 100,195 = $500.98, above the cap, so $31.20. Weekly wages are $1,926.83 and 0.5% of that is $9.63 a week, so the $0.60 weekly cap binds every week, and 52 x $0.60 = $31.20.", + "The task's '$95,350.72 annualized SAWW' does not appear in any source I read, and the engine does not use it. The engine applies the $411.91 parameter directly. The 2026 NYSAWW of $1,833.63 (WCB 2026 deduction notice, paidfamilyleave.ny.gov/2026, WCB press release) x 52 = $95,348.76, and 0.00432 of that is $411.9066, which rounds to $411.91. By contrast 0.00432 x 95,350.72 = $411.9151, which rounds to $411.92. The task's base figure therefore cannot reproduce the $411.91 cap exactly. This does not change the engine result.", + "DBL cap is weekly, not annual. 2026 has 53 Thursdays (computed), so a Thursday weekly payroll could legally collect up to 53 x $0.60 = $31.80. The engine's $31.20 is within $1 of that. The engine's annual min(0.5% x annual wages, $31.20) can also differ from the law for workers with uneven weeks, for example weeks under $120. Neither difference matters for scenario_082.", + "Publication dates, all before 2026-07-03. DFS decision: 'Dated: August 29, 2025', with a +11.31% premium rate increase over 2025. WCB 2026 deduction notice PDF: created 2025-09-03, modified 2025-09-05. WCB press release: October 30, 2025. NYSAWW: updated March 31, 2025 per paidfamilyleave.ny.gov/2026. DBL 0.5%/$0.60 is in WCL s.209(3)(a); the nysenate.gov page shows the section last updated 2016-04-08. A proposed DBL increase to $2.20 a week from 2027 (A00084) did not become law, according to search results I did not verify against a primary source. It would not apply to 2026 either way." + ], + "citation_problems": [ + "WCL s.211(4)-(5) composite quote. The researcher's quote 'in excess of the statutory amount provided in section two hundred nine of this article...except by agreement' joins two subdivisions and misstates the wording. s.211(4) reads 'provided in subdivision three of section two hundred nine of this...'. s.211(5) reads 'provided in section two hundred nine, subdivision three, except by agreement and provided the contribution...'. The substance holds; the quoted string is not verbatim.", + "12 NYCRR 380-7.2(b)(1) 'may deduct'. The text is verbatim, but the provision is about timing: an employer may deduct before the policy's effective date and before the employee is eligible. It is weak support for 'deduction is optional'. The stronger regulatory support is 380-7.2(c): an employer's 'failure to withhold may not be recovered by withholding larger than the maximum employee contribution at a later date.'", + "Official NYCRR (govt.westlaw.com) and direct curl of nysenate.gov both returned a Cloudflare 403. I read 12 NYCRR 380-7.2 through the Cornell LII copy, which notes adoption in the NY State Register on July 19, 2017, eff. 7/19/2017. I read WCL text through WebFetch, which returns excerpts of at most 125 characters, so I confirmed key fragments verbatim rather than whole subdivisions.", + "The researcher left out mandatory-sounding counter-evidence from official sources. paidfamilyleave.ny.gov/employer-responsibilities-and-resources lists 'Collect Employee Payroll Contributions' as an employer responsibility and says 'Ensure that you are collecting the employee contributions that pay for this insurance.' The DFS 2026 decision says 'the employee shall not be liable for any additional contributions for that year' once the cap is reached. The WCB 2026 deduction notice says 'Employees pay for these benefits through a small payroll deduction'. These do not override the statute's 'authorized' or WCB's explicit 'allowed, but not required', but they belong in the tension caveat.", + "paidfamilyleave.ny.gov/cost says 'funded by employees through payroll deductions', with no 'may', and still shows the 2025 figures (0.388%, $354.53, NYSAWW $1,757.19). The researcher correctly attributed the 'may be funded' wording to /2026 and noted the page is out of date." + ], + "reasoning": "I set out to refute the classification but could not. The primary statute, the regulations and the administering agency's own employer guidance all support 'optional employer pass-through' for both NY DBL and NY PFL.\n\nStatute (WCL, read on nysenate.gov via WebFetch):\n- s.209(4) says the employer 'is authorized to collect' the employee contribution through payroll deductions, and only within one month after paying the wages.\n- s.209(5) makes the employer the employees' agent when it collects.\n- s.210(1) has the employer pay the DBL cost 'in excess of the contributions collected', so whatever is not collected falls on the employer.\n- s.211 makes the covered employer the party that must provide the benefits.\n- s.209(3)(b) bars requiring employers to fund PFL but does not bar them from doing so voluntarily.\n- No subdivision says the employer 'shall deduct', 'shall withhold' or 'must collect'.\n\nRegulations (12 NYCRR 380-7.2, via Cornell LII):\n- (c) says missed PFL withholding cannot be recovered later by withholding above the weekly maximum.\n- Together with the one-month window in s.209(4), nothing makes the employee liable for a contribution the employer did not deduct.\n- Penalties for failing to provide coverage fall on the employer (380-7.2(d)).\n\nAgency guidance:\n- The WCB employer page says 'An employer is allowed, but not required, to collect contributions' and covers both DB and PFL.\n- The WCB employee page says the same for DB.\n- The WCB DB fact sheet and Form DB-271S both use 'may collect'.\n- paidfamilyleave.ny.gov/cost says 'An employer may choose to pay for the Paid Family Leave benefit on behalf of employees.'\n- DFS defines the 2026 maximum as the amount an employer is 'authorized to collect'.\n\nCounter-evidence I weighed:\n- WCL s.209(1): 'Every employee ... shall contribute ... to the extent and in the manner herein provided.' The manner provided is permissive employer collection under s.209(4).\n- DFS: 'Employers shall collect employee contributions consistent with this Decision' and 'the employee shall not be liable for any additional contributions' after the cap. I read both as setting the ceiling and method of collection, not as a mandate to collect.\n- The PFL employer-responsibilities page: 'Ensure that you are collecting the employee contributions that pay for this insurance.'\n- The WCB press release: PFL is 'fully paid for by employees'.\n\nThese describe the default design, especially for PFL, which DFS rates so that the employee share covers the full premium. None of them requires deduction. None creates an employee debt to the state or the carrier, and the WCB's explicit 'allowed, but not required' and the PFL site's 'may choose to pay' govern. So the classification stands for both programs. PFL is the closer case: it is designed to be employee-funded, but collection is still discretionary. It does not reach 'mixed_or_unclear', because the agency answers the may/must question directly.\n\nEngine and rates:\n- I read the NY engine variables and parameters in policyengine_us 2.15.17. They compute min(rate x wages, annual cap) with PFL 0.00432/$411.91 from 2026-01-01 and DBL 0.005/$31.20.\n- No NY file states the assumption that the employer withholds the maximum.\n- The aggregator employee_state_payroll_tax documents these as 'Employee-side mandatory state payroll taxes and payroll-funded contributions'.\n- The engine amounts for scenario_082 ($411.91 + $31.20 = $443.11, confirmed by a re-run) equal the law maximum only if the employer deducts the maximum. Under NY law the employee share consistent with the prompt's facts ranges from $0 to $443.11.\n\nThe researcher's quotes are substantively accurate, except the spliced s.211(4)-(5) quote and the weak framing of 380-7.2(b)(1). Neither problem affects the classification." + } + ] + }, + { + "key": "CT", + "research": { + "program": "Connecticut Paid Leave (Paid Family and Medical Leave Insurance Program), employee contribution", + "state": "CT", + "engine_variables": [ + "ct_employee_paid_leave_contribution", + "ct_employee_state_payroll_tax", + "employee_state_payroll_tax", + "taxable_earnings_for_social_security", + "payroll_tax_gross_wages", + "gov.states.ct.tax.payroll.paid_leave.employee_rate", + "gov.irs.payroll.social_security.cap" + ], + "premium_liability": "The employee owes it. Under Conn. Gen. Stat. 31-49g(b)(1), each employee (and each enrolled self-employed person or sole proprietor) must contribute a percentage of subject earnings, up to the Social Security contribution and benefit base, to the Family and Medical Leave Insurance Trust Fund. Under 31-49g(b)(3), each employer paying wages must deduct and withhold that contribution from wages every payroll period. The employer must register, report and remit (31-49g(b)(5)), and it bears the penalties for late or missing remittance. Under 31-49g(b)(4), the state can collect from either the employee or the employer. There is no employer share: the Authority's employer page says \"there is no employer match.\" The Authority's policy CTPL-003-CTRB(c) makes an employer that under-deducts liable for the shortfall, and it may not recoup that from later wages without CT DOL permission.", + "employee_share_rule": "employee_must_contribute", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "Conn. Gen. Stat. 31-49g(b)(1) (2026 Supplement; text from P.A. 19-25, 2019)", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "each employee and each self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m shall contribute a percentage of such employee's", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49g(b)(1), rate ceiling", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Such percentage shall be established by the authority, provided that the percentage shall not exceed one-half of one per cent.", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49g(b)(3), mandatory employer withholding", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Each employer making payment of any wages to an employee shall deduct and withhold from such wages for each payroll period a contribution", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49g(b)(4), collection from employee or employer", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "If, after notice, an employee or employer or self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m fails to make a payment required by this section", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49e(18) (2026 Supp.; was 31-49e(17) in the Jan. 1, 2025 revision), definition of subject earnings and cap", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cSubject earnings\u201d means total wages, as defined in subsection (b) of section 31-222 and self-employment income as defined in 26 USC 1402(b), as amended from time to time, that shall not exceed the Social Security contribution and benefit base", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49e(8), public employers excluded (except covered public employees)", + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cEmployer\u201d does not mean the federal government, the state or a municipality, a public school operator or a nonpublic elementary or secondary school", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49o(c), private plans: no state contribution", + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "Notwithstanding subsection (b) of section 31-49g, employees enrolled in an approved private plan shall not contribute a percentage of their earnings to the Family and Medical Leave Trust Fund.", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49o(c), private plans: optional employer withholding up to the state rate", + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "an employer may withhold or divert up to the portion of an employee's wages that corresponds to the contribution rate established pursuant to subsection (b) of section 31-49g", + "fetched_and_read": true + }, + { + "citation": "Conn. Gen. Stat. 31-49m(a), self-employed opt-in", + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "A self-employed individual or sole proprietor, upon application to the authority, in a form and manner prescribed by the authority, may enroll in the Family and Medical Leave Insurance Program", + "fetched_and_read": true + }, + { + "citation": "SSA, Cost-of-Living Increase and Other Determinations for 2026, 90 FR 49047 (Nov. 3, 2025), FR Doc. 2025-19763 (the 42 USC 430 base incorporated by 31-49g(b)(1) and 31-49e(18))", + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026 and self-employment income earned in tax years beginning in 2026.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/contributions", + "quote": "If you work for a covered employer in Connecticut, you are required to contribute 0.5% of your income up to the Social Security contribution limit. Individuals may not opt-out of CT Paid Leave.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers/i-am-an-employer", + "quote": "The funding to support CT Paid Leave comes from employee payroll deductions; there is no employer match. These payroll deductions are capped at 0.5% on earnings up to the Social Security contribution limit.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers", + "quote": "Under the law, you are required to withhold payroll deductions of 0.5% from your workers and remit these contributions to the CT Paid Leave Authority quarterly.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/remit-contributions", + "quote": "The CT Paid Leave Authority Board of Directors voted to maintain the contribution rate at 0.5% for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/091125-board-draft-meeting-minutes.pdf?rev=ce295323765443f9a39f48670452af1e", + "quote": "All voting members of the Board voted in favor of maintaining the 0.5% contribution rate for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/consolidated-policies---pa-25-174---09112025---clean-version.pdf?rev=58b65e2cf7bb42a88c6585f1c853a540", + "quote": "if an employer fails to appropriately deduct the full contribution amount from employees, that employer shall be responsible for the difference between the amount deducted and the amount owed.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "The responsibility for remitting timely contributions and wage reporting to the Authority belongs to employers, as do penalties for late payments or failure to submit payments and wage reports.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/coverage-and-eligibility", + "quote": "A municipality is not a covered employer under CT Paid Leave unless one or more of its bargaining units collectively bargains to be included.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/law-and-policies/contribution-rate-change-policy.pdf?rev=dc3e141324034cf28049fdb21a5f5840", + "quote": "If the Authority decides to change the contribution rate for the following calendar year, the Authority must announce such change no later than November first of the preceding calendar year.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Unclear. No primary source I read authorizes it. In the public program, 31-49g(b)(3) says the employer \"shall deduct and withhold\" the contribution, and 31-49g(b)(1) puts the obligation on the employee. None of the statute, the Authority's Consolidated Policies (Sept. 11, 2025 and 07/09/2026 versions) or the Authority's contribution, employer and remittance pages says an employer may pay the employee's contribution in place of withholding it. The Authority's site says employees \"may not opt-out.\" Under CTPL-003-CTRB(c), an employer that fails to deduct the full amount is responsible for the difference, so an employer that does not withhold ends up paying, but this is a liability rule, not an election. Payroll-vendor and broker pages (help.paysoft.com, marshmma.com, seen only as search results) claim employers may voluntarily cover the contribution; that is secondary and was not used here. The exception is an approved private plan (31-49o(c)): the employee owes nothing to the state fund, and the employer \"may withhold or divert up to\" the state-rate portion of wages. There, the employee share is optional for the employer.", + "rate_2026": { + "total_rate": "0.5% of subject earnings, all from the employee (statutory ceiling of one-half of one percent under 31-49g(b)(1); no employer share or match)", + "max_employee_share": "0.5% of wages up to $184,500 = $922.50 per year (applied per employer; employees with several employers who overpay can get a refund of the excess under CTPL-003-CTRB(g)/(l))", + "wage_base_or_cap": "$184,500, the 2026 OASDI contribution and benefit base, incorporated by 31-49g(b)(1) and 31-49e(18) through 42 USC 430", + "sources": [ + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/091125-board-draft-meeting-minutes.pdf?rev=ce295323765443f9a39f48670452af1e", + "published": "Board vote on September 11, 2025 to keep 0.5% for 2026 (draft minutes for that meeting; the rate was announced ahead of the Nov. 1, 2025 statutory deadline under 31-49g(b)(2))" + }, + { + "url": "https://www.ctpaidleave.org/remit-contributions", + "published": "Undated Authority web page, read 2026-10-05, stating the 0.5% rate for 2026" + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "published": "November 3, 2025 (Federal Register, FR Doc. 2025-19763; federalregister.gov API publication_date 2025-11-03)" + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "published": "2026 Supplement to the General Statutes (revised to Jan. 1, 2026). The rate ceiling and withholding duty in 31-49g(b)(1) and (b)(3) date from P.A. 19-25 (effective June 25, 2019); P.A. 25-75 made technical changes effective June 23, 2025. The Supplement's exact publication date was not confirmed." + } + ], + "published_before_2026_07_03": "Yes. The 0.5% rate for 2026 was voted on Sept. 11, 2025; the $184,500 base was published Nov. 3, 2025; the statutory ceiling and withholding duty have been law since 2019. One caveat: the Consolidated Policies at the live URL are 'Revised 07/09/2026', after the cutoff. The provisions quoted from them also appear, identically, in the Sept. 11, 2025 version I read." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_120", + "engine_amount": 827.98, + "law_amount_at_max_employee_share": 827.99, + "matches_within_1_dollar": true, + "note": "Wages $165,597 are below the 2026 cap of $184,500, so subject earnings = $165,597. 0.005 x 165,597 = $827.985 (law amount shown rounded to $827.99). Running policyengine-us 2.15.17 from the triage venv with a CT single adult earning $165,597 in 2026 gave ct_employee_paid_leave_contribution = employee_state_payroll_tax = 827.985 and taxable_earnings_for_social_security = 165,597. The benchmark's $827.98 differs from the legal amount by less than $0.01. The law's 0.5% is the mandatory contribution, not an optional ceiling, so the engine's assumption matches the law for a public-program employee." + } + ], + "engine_documentation": "Files read this session from policyengine_us 2.15.17 in .venv-pe21517:\n- variables/gov/states/ct/tax/payroll/paid_leave/ct_employee_paid_leave_contribution.py: label \"Connecticut employee paid leave contribution\"; documentation \"Employee-side Connecticut Paid Leave contribution.\"; Person, YEAR, defined_for StateCode.CT. Formula: parameters.gov.states.ct.tax.payroll.paid_leave.employee_rate x taxable_earnings_for_social_security. Unlike the Minnesota variable, it states no assumption about the employer withholding a \"maximum permitted\" share.\n- parameters/gov/states/ct/tax/payroll/paid_leave/employee_rate.yaml: description \"Employee-side Connecticut Paid Leave contribution rate.\"; value 0.005 from 2021-01-01, unchanged since. References: CGS Section 31-49e (cga.ct.gov chap_557) and ctpaidleave.org/remit-contributions. It does not cite 31-49g(b), where the contribution and rate ceiling actually sit.\n- variables/gov/states/ct/tax/payroll/ct_employee_state_payroll_tax.py: label \"Connecticut employee state payroll tax\"; documentation \"Employee-side Connecticut payroll-funded contributions.\"; adds [\"ct_employee_paid_leave_contribution\"].\n- variables/gov/states/tax/payroll/employee_state_payroll_tax.py: TaxUnit aggregator whose documentation reads \"Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules\"; it adds ct_employee_state_payroll_tax among 14 state components.\n- taxable_earnings_for_social_security = min_(gov.irs.payroll.social_security.cap, payroll_tax_gross_wages).\n- payroll_tax_gross_wages = max_(0, employment_income - fica_pre_tax_contributions).\n- parameters/gov/irs/payroll/social_security/cap.yaml: 2026-01-01 value 184_500, annotated \"Official SSA determination.\"\n- The CT unemployment files (ct_employer_state_unemployment_tax, default_rate 0.025, taxable_wage_base 25,000) are employer-side and are not part of ct_employee_state_payroll_tax.\n- tests/core/test_payroll_contributions.py has a CT case expecting ct_employee_paid_leave_contribution = ct_employee_state_payroll_tax = employee_state_payroll_tax = 500.\nNone of these files models private plans, public-employer exclusions, work location, or self-employed opt-in.", + "caveats": [ + "The task's lead was wrong: Conn. Gen. Stat. 31-49m covers voluntary enrollment by self-employed people and sole proprietors. The employee contribution and the employer's withholding duty are in 31-49g(b)(1) and (b)(3). Definitions are in 31-49e; after P.A. 25-174, subject earnings is 31-49e(18) in the 2026 Supplement.", + "Approved private plans (31-49o) change who pays. Employees in an approved private plan do not contribute to the state fund. The employer 'may withhold or divert up to' the state-rate share of wages, and the plan may cost employees no more than the state premium. For those employees the share is optional for the employer, and the law amount could be anywhere from $0 to 0.5% of capped wages. The benchmark prompt does not say whether the employer has a private plan.", + "Public employers are excluded under 31-49e(8): the federal government, the state, municipalities, public school operators (for certified staff) and nonpublic K-12 schools (for certified staff). State employees in a bargaining unit, and municipal or certified school staff, are covered only if their union bargains for inclusion; non-represented state employees are covered. The Authority's coverage page also lists railroads, other states' and countries' governments, and sovereign nations or tribes (unless there is a 31-49u memorandum of understanding). Employees of excluded employers owe $0.", + "There is no small-employer exemption. 'Employer' means anyone who employs one or more employees, and nonprofits and religious organizations are covered (Consolidated Policies). The obligation does not depend on unemployment insurance coverage.", + "Collective bargaining matters only for public employers (bargaining in). Under 31-49s, a private-sector CBA cannot reduce statutory rights, and no provision I read lets a private CBA opt employees out of contributing.", + "Work location: 'Employee' means service to an employer 'in this state' (31-49e(7)). Subject earnings exclude service outside Connecticut (Consolidated Policies CTPL-001-DEF). The Authority's page says some interstate truck drivers and military spouses may be excluded. The engine applies the contribution through defined_for StateCode.CT; these files have no separate work-location input. A Connecticut resident working in another state would owe $0 to Connecticut.", + "Wage definition: the law uses 'total wages' under 31-222(b). The Authority tells employers to exclude benefits excluded from FICA wages. The engine uses FICA wages (employment_income minus fica_pre_tax_contributions), capped at the Social Security base. For a plain-wages household these are the same.", + "With several employers, each employer withholds up to the Social Security base, and the employee can get a refund of any excess. The engine applies a single per-person cap, which equals the net amount after refund.", + "Self-employed people and sole proprietors owe nothing unless they enroll under 31-49m. The engine bases the amount only on wages (payroll_tax_gross_wages), which matches this.", + "The $2,325 minimum in a quarter is an eligibility test for benefits ('covered employee'), not a contribution threshold. Every employee contributes from the first dollar.", + "Timing: the Consolidated Policies at the live URL are 'Revised 07/09/2026', after the benchmark cutoff. Every provision I quoted from them also appears in the Sept. 11, 2025 version. The 2026 rate (vote of Sept. 11, 2025) and wage base (published Nov. 3, 2025) both predate 2026-07-03.", + "The cga.ct.gov statute pages failed in WebFetch (TLS certificate error), so I downloaded them with curl and read the text locally. ssa.gov/oact/cola/cbb.html returned 403, so the 2026 base comes from the Federal Register notice instead. The Authority's FAQ (service.ctpaidleave.org) is rendered by JavaScript and could not be read." + ], + "confidence": "high", + "summary": "Connecticut Paid Leave is a mandatory employee contribution, not an optional pass-through from the employer. Conn. Gen. Stat. 31-49g(b)(1) requires each employee to contribute a percentage of subject earnings, up to the Social Security base, set by the Authority at no more than 0.5%. 31-49g(b)(3) requires every employer paying wages to deduct and withhold it each payroll period. There is no employer share. The Authority's pages say individuals \"may not opt-out\" and that there is \"no employer match.\" For 2026, the Authority's board voted on Sept. 11, 2025 to keep the rate at 0.5%, and the Federal Register set the base at $184,500 on Nov. 3, 2025, so the maximum is $922.50 per year. Both predate the 2026-07-03 cutoff. The engine computes 0.005 x min($184,500, FICA wages). It is not the Minnesota-style \"maximum permitted\" assumption: for a public-program employee, the mandatory share is the legal amount. For scenario_120 the engine gives 0.005 x $165,597 = $827.985, which matches the law (I confirmed it by running the engine). The main exceptions, which the prompt does not rule out, are: approved private plans, where the employee owes nothing to the state and the employer may withhold up to 0.5%; excluded public employers unless their workers bargain in; and work outside Connecticut. No primary source I read lets an employer choose to pay the employee's share instead of withholding it; vendor pages claim it is allowed." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "each employee and each self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m shall contribute a percentage of such employee's or self-employed individual's or sole proprietor's subject earnings", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Such percentage shall be established by the authority, provided that the percentage shall not exceed one-half of one per cent.", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Each employer making payment of any wages to an employee shall deduct and withhold from such wages for each payroll period a contribution", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "If, after notice, an employee or employer or self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m fails to make a payment required by this section", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Any employer that fails to comply with the provisions of this subsection shall be subject to penalties established by the authority pursuant to subsection (b) of section 31-49h.", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Establishing the percentage of subject earnings each employee and self-employed individuals or sole proprietors who have enrolled in the program pursuant to section 31-49m shall contribute to the Family and Medical Leave Insurance Trust Fund", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cEmployee\u201d means an individual engaged in service to an employer in this state in the business of the employer;", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cSubject earnings\u201d means total wages, as defined in subsection (b) of section 31-222 and self-employment income as defined in 26 USC 1402(b), as amended from time to time, that shall not exceed the Social Security contribution and benefit base", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cEmployer\u201d does not mean the federal government, the state or a municipality, a public school operator or a nonpublic elementary or secondary school", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "Notwithstanding subsection (b) of section 31-49g, employees enrolled in an approved private plan shall not contribute a percentage of their earnings to the Family and Medical Leave Trust Fund.", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "an employer may withhold or divert up to the portion of an employee's wages that corresponds to the contribution rate established pursuant to subsection (b) of section 31-49g", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "A self-employed individual or sole proprietor, upon application to the authority, in a form and manner prescribed by the authority, may enroll in the Family and Medical Leave Insurance Program", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/contributions", + "quote": "If you work for a covered employer in Connecticut, you are required to contribute 0.5% of your income up to the Social Security contribution limit. Individuals may not opt-out of CT Paid Leave.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers/i-am-an-employer", + "quote": "The funding to support CT Paid Leave comes from employee payroll deductions; there is no employer match.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers", + "quote": "Under the law, you are required to withhold payroll deductions of 0.5% from your workers and remit these contributions to the CT Paid Leave Authority quarterly.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/remit-contributions", + "quote": "The CT Paid Leave Authority Board of Directors voted to maintain the contribution rate at 0.5% for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/091125-board-draft-meeting-minutes.pdf?rev=ce295323765443f9a39f48670452af1e", + "quote": "All voting members of the Board voted in favor of maintaining the 0.5% contribution rate for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/consolidated-policies---pa-25-174---09112025---clean-version.pdf?rev=58b65e2cf7bb42a88c6585f1c853a540", + "quote": "if an employer fails to appropriately deduct the full contribution amount from employees, that employer shall be responsible for the difference between the amount deducted and the amount owed.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "The responsibility for remitting timely contributions and wage reporting to the Authority belongs to employers, as do penalties for late payments or failure to submit payments and wage reports.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/ct-paid-leave-community-toolkit.pdf?rev=3bf18499da6c4fd6854ab464c9eb474b", + "quote": "Employers who fail to deduct CT Paid Leave contributions in a timely fashion will need to pay the owed amount themselves.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/English-PDFs-and-Docs/Employee-Resource-PDFs/CTPL-Employee-Fact-Sheet-English-rev-0523.pdf", + "quote": "As an employee, you must participate in the CT Paid Leave program unless you are specifically excluded by law", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/coverage-and-eligibility", + "quote": "A municipality is not a covered employer under CT Paid Leave unless one or more of its bargaining units collectively bargains to be included.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/law-and-policies/contribution-rate-change-policy.pdf?rev=dc3e141324034cf28049fdb21a5f5840", + "quote": "If the Authority decides to change the contribution rate for the following calendar year, the Authority must announce such change no later than November first of the preceding calendar year.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026 and self-employment income earned in tax years beginning in 2026.", + "fetched_and_read": true + }, + { + "url": "https://portal.ct.gov/-/media/DDS/Self_Determination/Employer_Toolkit.pdf", + "quote": "Although employers may apply for an exemption, individual employees are not permitted to opt-out.", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None that change the classification or the scenario_120 check. 0.005 x $165,597 = $827.985 exactly. I ran policyengine-us 2.15.17 from the triage venv for a CT single adult with $165,597 wages in 2026. ct_employee_paid_leave_contribution, ct_employee_state_payroll_tax and employee_state_payroll_tax all returned 827.9849853515625 (float32), and taxable_earnings_for_social_security returned 165,597. That rounds to $827.98, within $0.01 of the legal amount. In practice, rounding each paycheck could move the total by a few cents.", + "The 2026 wage base of $184,500 is confirmed verbatim in 90 FR 49047 (FR Doc. 2025-19763; the federalregister.gov API gives publication_date 2025-11-03). The engine's cap.yaml has 2026-01-01: 184_500. The maximum employee share is 0.005 x 184,500 = $922.50.", + "The engine's employee_rate.yaml has a single value, 0.005 from 2021-01-01. Its references are CGS 31-49e and the remit-contributions page, not 31-49g(b)(1), where the contribution and the 0.5% ceiling actually sit. This is a citation weakness, not an amount error.", + "Minor imprecision in the research: it says the employer must 'register, report and remit (31-49g(b)(5))'. In fact (b)(5) covers only registering and submitting reports. The remittance duty comes from (b)(3) together with Consolidated Policies CTPL-003-CTRB(a). Also, (b)(5) and (b)(6), the employer registration, reporting and penalty clauses, were added by P.A. 24-5 (2024), not P.A. 19-25. The 0.5% ceiling and the withholding duty in (b)(1) and (b)(3) do date from P.A. 19-25 (2019)." + ], + "citation_problems": [ + "No substantive problems. All nine statutory quotes in operative_text appear verbatim in the cga.ct.gov text I downloaded: six in the 2026 Supplement and three in the current/pub chapter 557, with the 31-49o(c) and 31-49m(a) text unchanged because the Supplement does not amend those sections. All agency-page and PDF quotes also appear verbatim.", + "The i-am-an-employer quote matches only once whitespace is normalized. The raw HTML puts a link around 'Social Security contribution limit', so the extracted text reads 'limit .' The rendered wording is the same.", + "WebFetch's summarizer did not return the for-businesses-and-employers quote ('Under the law, you are required to withhold...'). I confirmed it verbatim in that page's raw HTML, downloaded with curl, in the 'What if I'm a small business?' FAQ.", + "The Sept. 11, 2025 Consolidated Policies 'clean version' carries the footer 'rev. MM/DD/YYYY'. It is the proposed revision the Board adopted at its Sept. 11, 2025 meeting (minutes item 3). The URL names the minutes as a 'draft'. Both quotes the research takes from these documents appear in that version and in the live 'Revised 07/09/2026' version.", + "The 2020 CT Paid Leave Employer Toolkit hosted on portal.ct.gov (DDS) says an approved private plan must 'Deduct the same amount from employee paychecks as the state plan'. That conflicts with 31-49o(c), which says the employer 'may withhold or divert up to'. The toolkit is outdated and the statute controls, so this does not affect the research, which relies on the statute." + ], + "reasoning": "I tried to refute the classification from the statute, reading the full chapter rather than only the quoted lines. I downloaded cga.ct.gov chapter 557 (2026 Supplement and current/pub) with curl. The first fetch used -k; a later plain curl returned HTTP 200. I read 31-49e (definitions), 31-49g, 31-49h, 31-49i through 31-49o, and 31-49q through 31-49u in full. I then searched every PFML section for 'may deduct', 'may withhold', 'deduct', 'withh', 'on behalf of' and 'employer may'.\n\nFindings:\n(1) 31-49g(b)(1) puts the obligation on the employee: 'each employee ... shall contribute'. 31-49h(b)(11)(E) repeats it ('each employee ... shall contribute').\n(2) 31-49g(b)(3) requires withholding: every employer paying wages 'shall deduct and withhold' the contribution each payroll period.\n(3) 31-49g(b)(4) lets the state collect from either the employee or the employer. Under (b)(6), an employer that does not comply faces penalties.\n(4) The only 'may withhold' language in the PFML sections is 31-49o(c), for approved private plans. There, employees owe nothing to the state fund, and the employer's deduction of up to the state rate is optional.\n(5) No clause anywhere in 31-49e to 31-49u lets an employer pay instead of withholding, or makes the share for public-program employees optional.\n(6) The contribution applies to 'each employee', defined in 31-49e(7) as an individual serving an employer in Connecticut, not only to a 'covered employee'. So the $2,325 quarterly test affects benefit eligibility only, not who contributes.\n(7) 'Employer' covers anyone with one or more employees. Only governmental and certain school employers are excluded, unless their employees bargain in.\n\nAgency sources agree. The contributions page says workers 'are required to contribute 0.5%' and 'may not opt-out'. The employer pages say 'no employer match' and 'you are required to withhold'. Both versions of the Consolidated Policies, CTPL-003-CTRB(a)(3) and (c), make the employer responsible for any amount it fails to deduct. The Fall 2025 Community Toolkit says the same, and the 2020 toolkit says employees 'are not permitted to opt-out'.\n\nThe 2026 rate rests on sources published before the 2026-07-03 cutoff: the 0.5% Board vote on Sept. 11, 2025 (minutes confirmed), the statutory ceiling in force since 2019, and the $184,500 base published Nov. 3, 2025.\n\nA secondary-source search (a WebSearch summary, apparently drawing on payroll-vendor pages) claims employers may voluntarily pay the employee's contribution. I found no primary source that authorizes or forbids this. Even if allowed, the statute leaves the contribution owed by the employee. An employer that pays it would be paying the employee's legal obligation, not choosing whether the obligation exists.\n\nEngine (read this session): ct_employee_paid_leave_contribution = employee_rate (0.005) x taxable_earnings_for_social_security, which is min(SS cap, payroll_tax_gross_wages). It is aggregated through ct_employee_state_payroll_tax into employee_state_payroll_tax. Unlike the Minnesota variable, it states no 'maximum permitted' assumption. Running it gave 827.98498 for scenario_120, which matches the law's 0.5% x $165,597 = $827.985.\n\nThe classification holds for a public-program employee of a covered private employer working in Connecticut. The caveats the research already lists still apply: approved private plans (where the share is optional and $0 to 0.5% goes to the employer's plan), excluded public employers, and work outside Connecticut. The prompt rules none of these out, but none turns the public-program contribution into an optional pass-through." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "each employee and each self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m shall contribute a percentage of such employee's or self-employed individual's or sole proprietor's subject earnings", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Such percentage shall be established by the authority, provided that the percentage shall not exceed one-half of one per cent.", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "Each employer making payment of any wages to an employee shall deduct and withhold from such wages for each payroll period a contribution computed in such manner as to result, so far as practicable, in withholding from the employee's wages", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "If, after notice, an employee or employer or self-employed individual or sole proprietor who has enrolled in the program pursuant to section 31-49m fails to make a payment required by this section, a state collection agency", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cSubject earnings\u201d means total wages, as defined in subsection (b) of section 31-222 and self-employment income as defined in 26 USC 1402(b), as amended from time to time, that shall not exceed the Social Security contribution and benefit base", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/2026/sup/chap_557.htm", + "quote": "\u201cEmployer\u201d does not mean the federal government, the state or a municipality, a public school operator or a nonpublic elementary or secondary school", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "Notwithstanding subsection (b) of section 31-49g, employees enrolled in an approved private plan shall not contribute a percentage of their earnings to the Family and Medical Leave Trust Fund.", + "fetched_and_read": true + }, + { + "url": "https://www.cga.ct.gov/current/pub/chap_557.htm", + "quote": "Notwithstanding section 31-71e, an employer may withhold or divert up to the portion of an employee's wages that corresponds to the contribution rate established pursuant to subsection (b) of section 31-49g", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/contributions", + "quote": "If you work for a covered employer in Connecticut, you are required to contribute 0.5% of your income up to the Social Security contribution limit. Individuals may not opt-out of CT Paid Leave.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers", + "quote": "Under the law, you are required to withhold payroll deductions of 0.5% from your workers and remit these contributions to the CT Paid Leave Authority quarterly.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers", + "quote": "CT Paid Leave is designed to be entirely employee funded. There is no employer match.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/for-businesses-and-employers/i-am-an-employer", + "quote": "The funding to support CT Paid Leave comes from employee payroll deductions; there is no employer match. These payroll deductions are capped at 0.5% on earnings up to the Social Security contribution limit.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/remit-contributions", + "quote": "The CT Paid Leave Authority Board of Directors voted to maintain the contribution rate at 0.5% for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/remit-contributions", + "quote": "The employer bears the ultimate responsibility for complying with their contribution obligation, even if they use a third-party administrator to act on their behalf. Interest and penalties for late or missed payments will be assessed against the employer.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/091125-board-draft-meeting-minutes.pdf?rev=ce295323765443f9a39f48670452af1e", + "quote": "All voting members of the Board voted in favor of maintaining the 0.5% contribution rate for 2026. None opposed. None abstained.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/law-and-policies/contribution-rate-change-policy.pdf?rev=dc3e141324034cf28049fdb21a5f5840", + "quote": "If the Authority decides to change the contribution rate for the following calendar year, the Authority must announce such change no later than November first of the preceding calendar year.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "if an employer fails to appropriately deduct the full contribution amount from employees, that employer shall be responsible for the difference between the amount deducted and the amount owed.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "The employer shall not deduct money from employees' future wages to remit that owed amount, unless the employer has obtained specific permission from the Connecticut Department of Labor.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "If contributions are not paid as the result of the employee's failure to disclose wages accurately or due to any other reason in the control of the employee, such employee shall be subject to the same penalties", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "The responsibility for remitting timely contributions and wage reporting to the Authority belongs to employers, as do penalties for late payments or failure to submit payments and wage reports.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/english-pdfs-and-docs/consolidated-policies-document.pdf?rev=b440cde1574c42adb2565fb846c6bb7f", + "quote": "an employee shall not be required to contribute to the Family and Medical Leave Insurance Trust Fund for wages that exceed the Social Security contribution and benefit base in a calendar year.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pages/meetings-and-agendas/board-of-directors/sept-bod-2025/consolidated-policies---pa-25-174---09112025---clean-version.pdf?rev=58b65e2cf7bb42a88c6585f1c853a540", + "quote": "if an employer fails to appropriately deduct the full contribution amount from employees, that employer shall be responsible for the difference between the amount deducted and the amount owed.", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/-/media/ctpl/pdfs-and-docs/employer-resource-pdfs/employer-fact-sheet-english.pdf", + "quote": "These employers must register with the CT Paid Leave Authority and are required to withhold 0.5% of employee wages via payroll deduction (called contributions).", + "fetched_and_read": true + }, + { + "url": "https://www.ctpaidleave.org/how-ct-paid-leave-works/coverage-and-eligibility", + "quote": "A municipality is not a covered employer under CT Paid Leave unless one or more of its bargaining units collectively bargains to be included.", + "fetched_and_read": true + }, + { + "url": "https://www.federalregister.gov/documents/full_text/text/2025/11/03/2025-19763.txt", + "quote": "The OASDI contribution and benefit base is $184,500 for remuneration paid in 2026 and self-employment income earned in tax years beginning in 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.thehartford.com/paid-family-medical-leave/ct", + "quote": "In the State program, employers are prohibited from making financial contributions on behalf of employees.", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/ct/tax/payroll/paid_leave/ct_employee_paid_leave_contribution.py", + "quote": "rate = parameters(period).gov.states.ct.tax.payroll.paid_leave.employee_rate\n return rate * person(\"taxable_earnings_for_social_security\", period)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. 2026 rate 0.5% (board vote Sept. 11, 2025; draft minutes PDF created Sept. 17, 2025; the remit page confirms it). 2026 cap $184,500 (90 FR 49047, published 2025-11-03 per the federalregister.gov API). Maximum per employer is $922.50. Both predate 2026-07-03.", + "Engine parameter employee_rate.yaml has a single value, 0.005 from 2021-01-01, with no 2026 entry. Because the board kept 0.5% for 2026, carrying the old value forward is correct. The parameter cites 31-49e, but the rate ceiling sits in 31-49g(b)(1).", + "scenario_120 recomputed: I ran pe_case.py under policyengine-us 2.15.17 with PYTHONDONTWRITEBYTECODE set. It gave ct_employee_paid_leave_contribution = ct_employee_state_payroll_tax = employee_state_payroll_tax = 827.985, with taxable_earnings_for_social_security = payroll_tax_gross_wages = 165,597 and fica_pre_tax_contributions = 0. The law amount is 0.005 x 165,597 = 827.985. The benchmark's $827.98 and the research's rounded $827.99 differ only by display rounding (under $0.01). Total payroll_tax 13,496.155 = 7.65% x 165,597 + 827.985, consistent.", + "Wording nuance, not an error: the Authority's employer page says deductions are 'capped at 0.5%'. Read alone, that sounds like a ceiling with room to choose less. But the board set the rate at exactly 0.5%, and the for-businesses page and the employer fact sheet both say employers are 'required to withhold' 0.5%. So the phrase does not make the share optional." + ], + "citation_problems": [ + "None of the research's quotes failed verification. I confirmed every statute quote verbatim in curl downloads of cga.ct.gov: 31-49g(b)(1), (b)(1) rate ceiling, (b)(3), (b)(4), 31-49e(8), 31-49e(18), 31-49o(c) both sentences, and 31-49m(a). I also confirmed every agency quote verbatim: the contributions, i-am-an-employer, for-businesses, remit-contributions and coverage pages, the Sept. 11, 2025 minutes, the rate-change policy, and CTPL-003-CTRB(a)(3) and (c) in both the Sept. 2025 and 07/09/2026 policy versions.", + "Minor date gap the research left open, now closed: the research could not confirm when the 2026 Supplement was published. I compared 31-49g(a)-(b) in the Jan. 1, 2025 revision (cga.ct.gov/current/pub) with the 2026 Supplement after normalizing whitespace, and the text is identical. So the withholding duty and rate ceiling were published law well before the cutoff.", + "The minutes the research cites are a draft (the filename says so). The rate is independently confirmed on the remit-contributions and contributions pages, which are undated.", + "Context, not an error: the rate-change policy PDF's metadata date is May 2026, but the document itself says 'Adopted August 2024'. The live Consolidated Policies say 'Revised 07/09/2026' (PDF created July 14, 2026). The provisions quoted from them are identical in the Sept. 2025 version." + ], + "reasoning": "I tried to refute the classification through the administration lens and could not. The primary sources I read myself support it.\n\nStatute: 31-49g(b)(1) puts the duty on the employee, who \"shall contribute\" a percentage of subject earnings up to the Social Security base. The rate is set by the Authority at no more than 0.5%. Under (b)(3), every employer paying wages \"shall deduct and withhold\" the contribution each payroll period. Under (b)(4), the state can collect from a non-paying employee or employer after notice. Under (b)(5)-(6), employers must register and report, and face penalties. The (b) text is identical in the Jan. 1, 2025 revision and the 2026 Supplement.\n\nFormal rules: the only Connecticut regulations in this area (31-49p) cover appeals. Contribution rules sit in the Authority's Consolidated Policies, adopted under CGS 1-121.\n\nAgency wording is uniformly \"must\"/\"required\", never \"may\":\n- \"you are required to withhold payroll deductions of 0.5%\" (for-businesses page)\n- \"required to withhold 0.5% of employee wages\" (employer fact sheet, Oct. 2024)\n- individuals \"may not opt-out\" (contributions page)\n- the program is \"entirely employee funded\" with \"no employer match\" (for-businesses page)\n\nCan an employer choose to pay the employee share? No primary source authorizes that. Withholding is mandatory under 31-49g(b)(3). CTPL-003-CTRB(c) makes an employer that under-deducts liable for the shortfall and bars recouping it from future wages without CT DOL permission. So an employer that skips withholding ends up bearing the cost, but as a compliance failure, not a lawful election. CTRB(a)(4) adds that money collected from employees is held in trust for remittance.\n\nIs the employee ever liable? Yes, in two places:\n- 31-49g(b)(4) allows collection from the employee.\n- CTRB(i) makes an employee subject to the same penalties when contributions go unpaid for reasons in the employee's control, such as tipped income they did not disclose.\n\nThe secondary sources conflict, and I relied on neither. The Hartford says employers are \"prohibited\" from contributing on employees' behalf in the state program. Payroll vendors say employers may cover it voluntarily.\n\nThe real optional pass-through is the approved private plan under 31-49o(c). There, employees owe nothing to the state fund, and the employer \"may withhold or divert up to\" the state-rate share. Excluded public employers (unless their workers bargain in), work outside Connecticut, and non-enrolled self-employed people owe $0. The prompt does not rule these out, but none of them changes the default program's classification.\n\n2026 rate and cap:\n- 0.5%: board vote Sept. 11, 2025, under the policy requiring any change to be announced by Nov. 1.\n- $184,500: Federal Register, Nov. 3, 2025.\n- Both predate 2026-07-03.\n\nEngine, read this session: 0.005 x min(SS cap, FICA wages), defined_for StateCode.CT. The cap parameter for 2026 is 184,500. The engine states no \"maximum permitted\" assumption for Connecticut, and the 0.5% is the mandatory amount, not a ceiling the employer picks from. Running scenario_120 under 2.15.17 gives 827.985, matching 0.005 x 165,597 and the benchmark's $827.98 to within display rounding.\n\nEngine caveats the research already notes: the engine keys on residence (StateCode.CT), while the law keys on work in Connecticut, and it does not model private plans or public-employer exclusions. These matter only when the prompt's facts would differ.\n\nLocal working copies of everything I fetched are in /private/tmp/claude-501/-Users-maxghenis-Library-Application-Support-Claude-scratch-workspaces-ee3763f4-e7ea-4177-bcaf-1362266768c1-4457aa9d-2dc0-4bcf-a128-e50adae6be25-scratch-2026-09-22-c3da5e/224fe5b6-f6e8-4204-af8f-2508970193bc/scratchpad/ct/. I edited no project files." + } + ] + }, + { + "key": "CA", + "research": { + "program": "California State Disability Insurance (SDI), the worker contribution that funds both Disability Insurance (DI) and Paid Family Leave (PFL)", + "state": "CA", + "engine_variables": [ + "ca_employee_state_disability_insurance_contribution", + "ca_employee_state_payroll_tax (aggregator: adds = [ca_employee_state_disability_insurance_contribution])", + "ca_payroll_tax_gross_wages (base)", + "parameter gov.states.ca.tax.payroll.disability.employee_rate" + ], + "premium_liability": "Under the State Plan, the worker owes the SDI contribution. UIC s. 984(a)(1) says \"Each worker shall pay worker contributions\", and s. 2901 says each individual in employment \"shall contribute to the Disability Fund\". The employer collects it but does not owe it: s. 986(a)(1) requires every employer to withhold the workers' contributions in trust from wages when paid, and s. 986(b) requires it to send them to the Disability Fund. Under the State Plan the employer pays no SDI contribution of its own. EDD's DE 44 says \"UI and ETT are employer paid contributions. SDI and PIT are withheld from employee wages.\" The exception is an approved Voluntary Plan. There, s. 3252(a) says neither the employee nor the employer is liable for worker contributions to the state. The plan pays the state an assessment of 14% of the worker rate times taxable wages (s. 3252(b)), and s. 3260 lets the employer deduct up to the State Plan amount from wages, or pay all or part of the cost itself.", + "employee_share_rule": "employee_must_contribute", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "Cal. Unemp. Ins. Code s. 984(a)(1) (as amended by Stats. 2019, ch. 24 (SB 83))", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "Each worker shall pay worker contributions at the rate determined by the director pursuant to this section with respect to wages, as defined by Sections 926, 927, and 985.", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 984(a)(3)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "The rate of worker contributions shall not exceed 1.5 percent or be less than 0.1 percent.", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 2901", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2901", + "quote": "Each individual performing services for an employer in employment shall contribute to the Disability Fund the contributions required of such individual by Sections 984 and 985.", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 986(a)(1) (introduced by 'each employer shall:')", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "withhold in trust the amount of his workers' contributions from their wages at the time the wages are paid, shall show the deduction on his payroll records", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 986(a)(2)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "Hold in trust the amount of his workers' contributions, at the time their wages are paid, where he undertakes or agrees to pay without deduction from the wages of his workers the amount of worker contributions required of his workers", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 985(b) (as amended by Stats. 2023, ch. 196 (SB 143), effective Sept. 13, 2023): taxable wage ceiling removed", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=985.", + "quote": "This section shall not apply with respect to wages paid on or after January 1, 2024.", + "fetched_and_read": true + }, + { + "citation": "SB 951 (Durazo), Stats. 2022, ch. 878, Legislative Counsel's Digest (approved by the Governor Sept. 30, 2022)", + "url": "https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202120220SB951", + "quote": "This bill would remove that limitation on January 1, 2024.", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 3252(a) (voluntary plans)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3252", + "quote": "neither an employee nor his or her employer shall be liable for the worker contributions required under this division with respect to wages paid by the employer while the employee is covered by an approved voluntary plan.", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 3260 (voluntary plans)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3260", + "quote": "An employer may, but need not, assume all or part of the cost of the plan, and may deduct from the wages of an employee covered by the plan ... an amount not in excess of that which would be required by Sections 984 and 985", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 2902 (religious exemption)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2902", + "quote": "shall be exempt from contributions under this division in respect to any wages paid to him by any such employer in the calendar quarter in which such statement is filed", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 926 (definition of wages)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=926", + "quote": "\"wages\" means all remuneration payable to an employee for personal services, whether by private agreement or consent or by force of statute, including commissions and bonuses", + "fetched_and_read": true + }, + { + "citation": "Cal. Unemp. Ins. Code s. 710.5 (public agency elective coverage through a negotiated agreement)", + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=710.5", + "quote": "provided the election is the result of a negotiated agreement between the public agency and the recognized employee organization", + "fetched_and_read": true + }, + { + "citation": "Cal. Code Regs. tit. 22, s. 3260-1 (voluntary plan employee contributions are trust funds). Read on LII, not on the official OAL/Westlaw host.", + "url": "https://www.law.cornell.edu/regulations/california/22-CCR-3260-1", + "quote": "The amounts deducted by an employer from the wages of his or her employees as their contributions under a voluntary plan are trust funds.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "California employees pay mandatory contributions to reimburse the Disability Insurance (DI) Fund for State Disability Insurance (SDI) coverage.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260618174055/https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "2026 Voluntary Plan Employee Contribution and Benefit Rate / Effective January 1, 2026 / Employee Contribution Rate / 1.3% (Wayback capture of 2026-06-18; page dateModified 2026-01-01; same 'mandatory contributions' definition)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "The SDI withholding rate for 2026 is 1.3 percent. Effective January 1, 2024, all wages are subject to SDI contributions.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251110012902/https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "The SDI withholding rate for 2026 is 1.3 percent. (Earliest Wayback capture that shows it: 2025-11-10, page dateModified 2025-11-07. The 2025-10-08 and 2025-10-17 captures, dateModified 2025-10-03, list only 2025.)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "UI and ETT are employer paid contributions. SDI and PIT are withheld from employee wages. (DE 44 2026 California Employer's Guide, Rev. 52 (4-26), Last-Modified 2026-04-17)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "The SDI program, which includes DI and PFL, is funded through mandatory employee payroll deductions for most California workers.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "Employers withhold 1.3 percent (.013) for SDI on all subject wages paid to each employee in a calendar year. The SDI rate is set by law and may change each year.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "Once a VP is approved, the employer is no longer required to send SDI withholdings to EDD for those employees covered by the VP.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de201.pdf", + "quote": "Effective January 1, 2024, there is no SDI taxable wage limit. (DE 201 2026 California Payroll Taxes, Rev. 12 (1-26). The table gives SDI: Who Pays = Employee; Rate = 1.3%; Wage Limit = None. PDF created 2025-12-03.)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/vp-general-release-letter-2026.pdf", + "quote": "Starting January 1, 2026, the SDI worker contribution rate will be 1.3 percent of an employee's annual gross taxable wages. (Letter dated December 31, 2025)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employer may pay an employee's share of Social Security and Medicare (taxes under the Federal Insurance Contributions Act [FICA]), State Disability Insurance (SDI), and federal and state income taxes without taking any money from the employee's salary.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employee's wages will increase if an employer pays their employees' share of FICA and SDI without taking it out from their wages.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de2040.pdf", + "quote": "An employer is authorized to deduct from a VP employee's wages an amount that does not exceed the current SDI plan contribution rate.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/unemployment/pdf/january-2026-di-fund-forecast.pdf", + "quote": "increased to 1.3 percent in 2026, and is projected to remain at 1.3 percent in 2027. (January 2026 DI Fund Forecast, PDF created 2026-01-07)", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Yes, under the State Plan, but the employee's contribution remains owed. UIC s. 986(a)(2) covers an employer that \"undertakes or agrees to pay without deduction\" the worker contributions. The employer must still hold that amount in trust and send it to the Disability Fund. EDD's DE 231Q (Rev. 43 (1-26)) says an employer may pay an employee's SDI without taking money from salary. That payment counts as extra wages: subject wages are grossed up by W/(1-R) with R = .0130 for 2026, and household and agricultural employers are included for SDI. So the worker contribution is still due, and is computed on the grossed-up wage. The employer is only the one funding it. Under an approved Voluntary Plan the answer is different. UIC s. 3260 says the employer \"may, but need not\" assume all or part of the plan's cost and may deduct up to the State Plan amount. That is an optional employer pass-through capped at 1.3% of wages.", + "rate_2026": { + "total_rate": "1.3% of all SDI-subject wages. The whole rate is the worker contribution (UIC s. 984). Under the State Plan the employer pays no SDI rate.", + "max_employee_share": "1.3%, which is the full rate. The statutory band is 0.1% to 1.5% (s. 984(a)(3)). Under a Voluntary Plan, the most an employer may deduct is the same 1.3% (s. 3260; DE 2040).", + "wage_base_or_cap": "None. UIC s. 985(b) says the wage ceiling does not apply to wages paid on or after Jan 1, 2024 (SB 951, Stats. 2022 ch. 878; s. 985 as amended by SB 143, Stats. 2023 ch. 196). So there is no maximum contribution. EDD's DE 201 and DE 44 both say 'None' and 'No maximum'.", + "sources": [ + { + "url": "https://web.archive.org/web/20251110012902/https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "published": "The 2026 rate is on the EDD Rates and Withholding page by the 2025-11-10 Wayback capture (page dateModified 2025-11-07). It is absent from the 2025-10-08 and 2025-10-17 captures." + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de201.pdf", + "published": "PDF created 2025-12-03; server Last-Modified 2025-12-31; labeled Rev. 12 (1-26)" + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/vp-general-release-letter-2026.pdf", + "published": "Letter dated 2025-12-31 (PDF Last-Modified 2026-02-04)" + }, + { + "url": "https://edd.ca.gov/siteassets/files/unemployment/pdf/january-2026-di-fund-forecast.pdf", + "published": "PDF created 2026-01-07" + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "published": "DE 44 Rev. 52 (4-26); server Last-Modified 2026-04-17" + }, + { + "url": "https://web.archive.org/web/20260618174055/https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "published": "Page dateModified 2026-01-01; captured 2026-06-18" + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=985.", + "published": "No-cap rule: SB 951 approved 2022-09-30; s. 985(b) current text effective 2023-09-13" + } + ], + "published_before_2026_07_03": "Yes. The 1.3% rate was public on EDD's site by early November 2025, and in the DE 201 (Dec 2025), the VP General Release Letter (Dec 31, 2025) and DE 44 (Apr 2026). The no-cap rule has been law since 2022/2023. I did not locate the director's formal s. 984(a)(1) rate statement, which the statute requires by Oct 31, 2025." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_005", + "engine_amount": 5590, + "law_amount_at_max_employee_share": 5590, + "matches_within_1_dollar": true, + "note": "$430,000 x 0.013 = $5,590.00, no cap. The engine gives $5,590. I also ran the installed engine for a single CA adult with $430,000 wages in 2026: ca_employee_state_disability_insurance_contribution = 5590.0. The rate is flat, uncapped and per worker, so splitting household wages among earners does not change the total." + }, + { + "scenario_id": "scenario_022", + "engine_amount": 684.26, + "law_amount_at_max_employee_share": 684.26, + "matches_within_1_dollar": true, + "note": "$52,635 x 0.013 = $684.255, shown as $684.26. A local engine run gave 684.255." + }, + { + "scenario_id": "scenario_023", + "engine_amount": 226.75, + "law_amount_at_max_employee_share": 226.75, + "matches_within_1_dollar": true, + "note": "$17,442.65 x 0.013 = $226.75445, shown as $226.75. A local engine run gave 226.7545." + }, + { + "scenario_id": "scenario_099", + "engine_amount": 2145, + "law_amount_at_max_employee_share": 2145, + "matches_within_1_dollar": true, + "note": "$165,000 x 0.013 = $2,145.00, no cap. Before 2024 a cap would have bound at this wage; it no longer does. A local engine run gave 2145.0." + } + ], + "engine_documentation": "I read these files this session, under policyengine_us 2.15.17 (.venv-pe21517).\n\n(1) variables/gov/states/ca/tax/payroll/disability/ca_employee_state_disability_insurance_contribution.py. Label: \"California employee state disability insurance contribution\". Documentation: \"Employee-side California State Disability Insurance withholding, which funds SDI and Paid Family Leave.\" Person, YEAR, defined_for StateCode.CA. Formula: rate * ca_payroll_tax_gross_wages, with rate from parameters gov.states.ca.tax.payroll.disability.employee_rate. There is no wage cap and no voluntary-plan, religious-exemption or coverage logic. Unlike the Minnesota variable, this file states no \"maximum permitted employee share\" assumption.\n\n(2) parameters/gov/states/ca/tax/payroll/disability/employee_rate.yaml. Description: \"Employee-side California State Disability Insurance contribution rate.\" It has one value, 0000-01-01: 0.013, so 1.3% applies to every year. References: UIC s. 984, UIC s. 985, and \"2026 California Payroll Taxes (DE 201)\".\n\n(3) variables/.../ca_employee_state_payroll_tax.py. Documentation: \"Employee-side California payroll-funded contributions, including state disability insurance withholding.\" It is defined as adds = [ca_employee_state_disability_insurance_contribution].\n\n(4) variables/.../ca_payroll_tax_gross_wages.py. It equals max(0, min(employment_income, payroll_tax_gross_wages + health_savings_account_payroll_contributions)). It cites EDD DE 231TP p.7, which I did not fetch.\n\nThe other CA payroll files are employer-side and not counted in the employee total: ETT at 0.001, and UI at a 0.034 default rate on a $7,000 base.\n\nI also ran the engine locally for 2026 single-adult CA households at the four wage levels. It returned 5590.0, 684.255, 226.7545 and 2145.0 for both the SDI variable and employee_state_payroll_tax. I did not read the employee_state_payroll_tax source file itself.", + "caveats": [ + "Voluntary Plans (UIC ss. 3251-3272). For employees covered by an approved VP, neither the employee nor the employer owes the State Plan worker contribution (s. 3252(a)). The employer 'may, but need not' assume the plan's cost and may deduct up to the State Plan amount (s. 3260; DE 2040). This is the one case where the engine's 1.3% is a maximum permitted deduction, not a mandatory one. A VP needs majority employee consent (s. 3254(c)). The benchmark prompt does not say whether the employer has a VP.", + "Employer-paid employee share (s. 986(a)(2); DE 231Q). If the employer pays SDI without deducting it, the payment counts as extra SDI-subject wages, grossed up as W/(1-0.013). The contribution is then slightly higher than 1.3% of stated cash wages, and the employer funds it. The engine does not model this gross-up.", + "Religious exemption (s. 2902). A worker who files a statement adhering to healing by prayer is exempt from contributions, and also gives up benefits.", + "Coverage exclusions. DE 44 lists some domestic workers, some governmental employees, employees of some nonprofit agencies, employees of interstate railroads, and people claiming a religious exemption. Public-entity service is covered only in narrow cases or by elective coverage (s. 2606(a)). A public agency can elect coverage only through an agreement negotiated with a recognized employee organization (s. 710.5), which is the only collective-bargaining hook I found. Self-employed people are not covered unless they elect coverage under ss. 708/708.5, at a separately set rate. The engine applies 1.3% to all CA wages and models none of these exclusions.", + "No small-employer exemption exists. The State Plan rate and the worker's liability do not depend on employer size.", + "Engine time-invariance. employee_rate.yaml has one value (0.013 from 0000-01-01) and the formula has no cap. That is correct for 2026. For other years it would be wrong: EDD lists 1.1% for 2024, 1.2% for 2025, and for 2023 0.9% with a $153,164 cap and $1,378.48 maximum. This does not affect the TY2026 benchmark.", + "Engine base. ca_payroll_tax_gross_wages adds HSA payroll contributions back to the FICA wage base, citing DE 231TP (not fetched this session). If a scenario has no HSA payroll contributions, the base is just wages.", + "Publication timing. I could not find the director's formal s. 984(a)(1) public-record rate statement, due by 2025-10-31. Rate publication is instead shown by the EDD web page (Wayback 2025-11-10; dateModified 2025-11-07), the DE 201 (Dec 2025), the VP General Release Letter (2025-12-31) and DE 44 (Apr 2026). All of these predate 2026-07-03.", + "The tit. 22 CCR s. 3260-1 text was read on LII (Cornell), a re-host of the official regulation, not on the official OAL/Westlaw site. It only backs a caveat, not the classification." + ], + "confidence": "high", + "summary": "California SDI, including PFL, is a mandatory employee contribution under the default State Plan. The law puts the liability on the worker: UIC s. 984(a)(1) says \"Each worker shall pay worker contributions\", and s. 2901 says the same. Every employer must withhold the contribution in trust from wages and send it to the Disability Fund (s. 986(a)(1), (b)). Under the State Plan the employer pays no SDI of its own (DE 44), and EDD describes the contributions as \"mandatory\".\n\nAn employer may choose to pay the worker's contribution instead of deducting it (s. 986(a)(2); DE 231Q). The contribution is still owed, and the payment counts as extra wages.\n\nFor 2026 the rate is 1.3% on all wages, with no wage base or maximum since Jan 1, 2024 (s. 985(b); SB 951). That figure was published on EDD's site by November 2025 and in the DE 201, the VP General Release Letter (Dec 31, 2025) and DE 44 (Apr 2026), all before 2026-07-03.\n\nThe engine computes 0.013 x CA payroll gross wages with no cap. All four benchmark amounts equal 1.3% of wages, within rounding: $5,590; $684.255 shown as $684.26; $226.75445 shown as $226.75; $2,145.\n\nThe only optional pass-through case is an approved Voluntary Plan. There the employer \"may, but need not\" deduct up to the State Plan amount (ss. 3252(a), 3260). The other caveats are the religious exemption (s. 2902) and coverage exclusions such as some public, domestic, railroad and nonprofit employees. The benchmark prompt does not specify any of these, and the engine does not model them.\n\nClassification: mandatory_employee_withholding." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "Each worker shall pay worker contributions at the rate determined by the director pursuant to this section with respect to wages, as defined by Sections 926, 927, and 985.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "The rate of worker contributions shall not exceed 1.5 percent or be less than 0.1 percent.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2901", + "quote": "Each individual performing services for an employer in employment shall contribute to the Disability Fund the contributions required of such individual by Sections 984 and 985.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "(a) Notwithstanding any provision of law in this state to the contrary, each employer shall: (1) Except as provided in subdivision (a)(2) of this section, withhold in trust the amount of his workers\u2019 contributions from their wages at the time the wages are paid", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "where he undertakes or agrees to pay without deduction from the wages of his workers the amount of worker contributions required of his workers under this division.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=987", + "quote": "Each employer shall be liable for any and all contributions required to be made by his workers on account of wages which he has paid to them regardless of whether or not he has deducted the contributions from the workers\u2019 wages", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=988", + "quote": "contributions by workers, payable as provided in this article, shall not be considered any part of the employer\u2019s assets and shall be paid to the director prior to the payment of any other claim against the employer.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=976", + "quote": "The contributions are due and shall be paid to the department for the Unemployment Fund by each employer in accordance with this division and shall not be deducted in whole or in part from the wages of individuals in his employ.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=985.", + "quote": "(b) This section shall not apply with respect to wages paid on or after January 1, 2024. (Amended by Stats. 2023, Ch. 196, Sec. 25. (SB 143) Effective September 13, 2023.)", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202120220SB951", + "quote": "This bill would remove that limitation on January 1, 2024.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3252", + "quote": "neither an employee nor his or her employer shall be liable for the worker contributions required under this division with respect to wages paid by the employer while the employee is covered by an approved voluntary plan.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3260", + "quote": "An employer may, but need not, assume all or part of the cost of the plan, and may deduct from the wages of an employee covered by the plan", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3254", + "quote": "(c) A majority of the employees of the employer employed in this state or a majority of the employees employed at any one distinct, separate establishment maintained by the employer in this state have consented to the plan.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2902", + "quote": "shall be exempt from contributions under this division in respect to any wages paid to him by any such employer in the calendar quarter in which such statement is filed", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=926", + "quote": "\u201cwages\u201d means all remuneration payable to an employee for personal services, whether by private agreement or consent or by force of statute, including commissions and bonuses", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=710.5", + "quote": "provided the election is the result of a negotiated agreement between the public agency and the recognized employee organization", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2606", + "quote": "with respect to service for any public entity as defined by Section 605 \u201cemployment\u201d for the purposes of this part includes only:", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/california/22-CCR-986-1", + "quote": "Contributions from employees at the rates prescribed in the code shall be deducted by the employer from wages paid to the employee at the time such wages are paid and shall be transmitted to the department by the employer", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/california/22-CCR-3260-1", + "quote": "The amounts deducted by an employer from the wages of his or her employees as their contributions under a voluntary plan are trust funds.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "California employees pay mandatory contributions to reimburse the Disability Insurance (DI) Fund for State Disability Insurance (SDI) coverage.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260618174055/https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "2026 Voluntary Plan Employee Contribution and Benefit Rate Effective January 1, 2026 Employee Contribution Rate 1.3% Taxable Wage Ceiling (per employee per year) $NA", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251110012902/https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "The SDI withholding rate for 2026 is 1.3 percent. Effective January 1, 2024, all wages are subject to SDI contributions.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "The SDI withholding rate for 2026 is 1.3 percent. Effective January 1, 2024, all wages are subject to SDI contributions.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "UI and ETT are employer paid contributions. SDI and PIT are withheld from employee wages.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "The SDI program, which includes DI and PFL, is funded through mandatory employee payroll deductions for most California workers.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "Employers withhold 1.3 percent (.013) for SDI on all subject wages paid to each employee in a calendar year. The SDI rate is set by law and may change each year.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "Once a VP is approved, the employer is no longer required to send SDI withholdings to EDD for those employees covered by the VP.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de201.pdf", + "quote": "Effective January 1, 2024, there is no SDI taxable wage limit.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de3395.pdf", + "quote": "2026 7,000 None1 3.4 6.2 0.1 1.3 (DE 3395 Rev. 69 (1-26) row: Year, UI wage base, DI wage base, UI unrated/rated %, ETT %, SDI tax %; PDF created 2025-12-17)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/vp-general-release-letter-2026.pdf", + "quote": "Starting January 1, 2026, the SDI worker contribution rate will be 1.3 percent of an employee's annual gross taxable wages. Reference: California Unemployment Insurance Code (CUIC) section 984(a) (1).", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employer may pay an employee's share of Social Security and Medicare (taxes under the Federal Insurance Contributions Act [FICA]), State Disability Insurance (SDI), and federal and state income taxes without taking any money from the employee's salary.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employee's wages will increase if an employer pays their employees' share of FICA and SDI without taking it out from their wages.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de2040.pdf", + "quote": "An employer is authorized to deduct from a VP employee's wages an amount that does not exceed the current SDI plan contribution rate.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/unemployment/pdf/january-2026-di-fund-forecast.pdf", + "quote": "increased to 1.3 percent in 2026, and is projected to remain at 1.3 percent in 2027.", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/ca/tax/payroll/disability/ca_employee_state_disability_insurance_contribution.py", + "quote": "rate = parameters(period).gov.states.ca.tax.payroll.disability.employee_rate / return rate * person(\"ca_payroll_tax_gross_wages\", period)", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/parameters/gov/states/ca/tax/payroll/disability/employee_rate.yaml", + "quote": "description: Employee-side California State Disability Insurance contribution rate. values: 0000-01-01: 0.013", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/tax/payroll/employee_state_payroll_tax.py", + "quote": "\"Employee-side mandatory state payroll taxes and payroll-funded \" \"contributions, aggregated from jurisdiction-specific rules.\" ... adds = EMPLOYEE_STATE_PAYROLL_TAX_COMPONENTS", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. All four engine amounts equal 1.3% of wages with no cap: 430,000 x 0.013 = 5,590.00; 52,635 x 0.013 = 684.255 (shown 684.26); 17,442.65 x 0.013 = 226.75445 (shown 226.75); 165,000 x 0.013 = 2,145.00. I re-ran the installed policyengine_us 2.15.17 for single CA adults in 2026. It returned 5590.0, 684.255, 226.75446 and 2145.0, the same for ca_employee_state_disability_insurance_contribution and employee_state_payroll_tax. The research reported 226.7545 for the third; the difference is float display only.", + "The 2026 rate of 1.3% with no wage base is confirmed by: the EDD Rates and Withholding page (Wayback 2025-11-10, dateModified 2025-11-07; still live, with dateModified now 2026-07-07); DE 201 Rev. 12 (1-26) (PDF created 2025-12-03, Last-Modified 2025-12-31); DE 3395 Rev. 69 (1-26), which shows 2026 SDI 1.3% and DI wage base 'None' (PDF created 2025-12-17); the VP General Release Letter dated December 31, 2025 (but the PDF itself was created 2026-01-29 and modified 2026-02-04); the January 2026 DI Fund Forecast (created 2026-01-07); and DE 44 Rev. 52 (4-26) (Last-Modified 2026-04-17). All predate 2026-07-03.", + "The no-cap rule is confirmed. UIC s. 985(b) reads 'This section shall not apply with respect to wages paid on or after January 1, 2024', per Stats. 2023 ch. 196 (SB 143), effective 2023-09-13. SB 951 (ch. 878) was approved 2022-09-30.", + "Engine time-invariance, as the research notes: employee_rate.yaml has a single value, 0000-01-01: 0.013, and the formula has no cap. That is correct for TY2026 and wrong for 2023-2025. It does not affect this benchmark.", + "The live Contribution Rates and Benefit Amounts page now shows dateModified 2026-09-17, after the cutoff. Its 2026-06-18 Wayback capture (dateModified 2026-01-01) is the right pre-cutoff evidence, and it carries the same 'mandatory contributions' sentence and 1.3% VP rate.", + "Like the research, I did not locate the director's formal s. 984(a)(1) public-record rate statement, which was due by 2025-10-31. CDX shows the EDD rates page changed between the 2025-10-20 capture and the 2025-11-10 capture." + ], + "citation_problems": [ + "Characterization error (not a quote error). The research says the employer 'collects it but does not owe it'. UIC s. 987, which the research did not cite, makes each employer 'liable for any and all contributions required to be made by his workers ... regardless of whether or not he has deducted the contributions.' The employer has collection and secondary liability for the worker contribution. This strengthens the mandatory-withholding classification rather than weakening it, but the sentence as written is inaccurate.", + "The s. 3260 quote uses an ellipsis. It omits 'for the purpose of providing the disability benefits specified in this part,'. The elision does not change the meaning.", + "DE 231Q formula notation. The source writes 'S = W (1 \u2013 R)', where S is the actual salary paid and W is the reportable (total subject) wage, so W = S/(1-R). The research writes 'W/(1-R)' with W as cash wages. The math is the same but the variable labels are swapped relative to the source.", + "The DE 44 PDF's embedded Title metadata reads '2025 California Employer's Guide (DE 44) Rev. 52 (4-26)'. The cover text reads 'California Employer\u2019s Guide 2026' and 'DE 44 Rev. 52 (4-26)'. The research's '2026' label matches the cover.", + "22 CCR s. 3260-1, and s. 986-1 which I added, were read on LII (Cornell), not on the official OAL/Westlaw host.", + "Every other quote was confirmed verbatim at its stated URL: UIC ss. 984(a)(1), 984(a)(3), 2901, 986(a)(1), 986(a)(2), 985(b), 3252(a), 2902, 926 and 710.5; the SB 951 digest; and EDD's Contribution Rates page (live and Wayback), Rates and Withholding page (live and Wayback), DE 44 (four quotes), DE 201, VP letter, DE 231Q (two quotes), DE 2040 and the January 2026 forecast. Differences were curly versus straight apostrophes only." + ], + "reasoning": "I fetched the full statutory sections myself from leginfo (raw HTML, not a summary): UIC ss. 926, 976, 984, 985, 986, 987, 988, 2601-2606, 2901-2903, 3251, 3252, 3254, 3260-3263 and 710.5. I also read the SB 951 chaptered text, 22 CCR ss. 986-1 and 3260-1 (on LII), and the EDD pages and PDFs (DE 44, DE 201, DE 3395, DE 231Q, DE 2040, the VP 2026 letter, the January 2026 DI forecast, and Wayback captures).\n\nStatute lens. The State Plan contribution is mandatory and the liability sits on the worker:\n- s. 984(a)(1): \"Each worker shall pay worker contributions\".\n- s. 2901: each individual in employment \"shall contribute to the Disability Fund\".\n- s. 986(a)(1): \"each employer shall ... withhold in trust\" the workers' contributions from wages.\n- 22 CCR 986-1 repeats this: contributions \"shall be deducted by the employer\".\n- s. 987 makes the employer liable for the workers' contributions whether or not it deducted them.\n- s. 988 treats withheld worker contributions as not part of the employer's assets.\n\nThe UI contribution is the contrast case. s. 976 says the employer's own UI contribution \"shall not be deducted\" from wages, which confirms SDI is the employee-side levy.\n\nThe only \"may\" language for the State Plan is s. 986(a)(2). It lets the employer pay the worker contribution without deducting it, but the contribution stays \"required of his workers\" and must be held in trust and sent in. That makes the employer's payment a choice about who funds the amount, not about whether it is owed.\n\nThe only real optional pass-through is the approved Voluntary Plan. There, s. 3252(a) relieves the State Plan worker liability, and s. 3260 says the employer \"may, but need not\" assume the cost and \"may deduct\" up to the State Plan amount. A VP needs employer consent and majority employee consent (s. 3254), so it is an exception the benchmark prompt never invokes. The religious exemption (s. 2902) and coverage limits for public entities (s. 2606, with s. 710.5 elective coverage) are also exceptions, not the default.\n\nNo small-employer, collective-bargaining or self-insurance condition makes the default State Plan worker contribution optional. The classification mandatory_employee_withholding therefore holds, with the VP caveat correctly flagged.\n\nThe 2026 rate is 1.3% with no wage cap. It was published on EDD's site by 2025-11-10 and in DE 201 and DE 3395 (December 2025), all before 2026-07-03.\n\nOn the engine, I read the CA files and the aggregator. It computes 0.013 x ca_payroll_tax_gross_wages, and employee_state_payroll_tax is documented as \"Employee-side mandatory state payroll taxes\". My local runs reproduce all four benchmark amounts.\n\nThe one substantive flaw I found is the research's statement that the employer \"does not owe\" the contribution, which s. 987 contradicts. It does not change the classification." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "Each worker shall pay worker contributions at the rate determined by the director pursuant to this section with respect to wages, as defined by Sections 926, 927, and 985.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.", + "quote": "On or before October 31 of each calendar year, the director shall prepare a statement, which shall be a public record, declaring the rate of worker contributions for the calendar year", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=2901", + "quote": "Each individual performing services for an employer in employment shall contribute to the Disability Fund the contributions required of such individual by Sections 984 and 985.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "withhold in trust the amount of his workers\u2019 contributions from their wages at the time the wages are paid, shall show the deduction on his payroll records", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=986.", + "quote": "Hold in trust the amount of his workers\u2019 contributions, at the time their wages are paid, where he undertakes or agrees to pay without deduction from the wages of his workers the amount of worker contributions required of his workers", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=985.", + "quote": "(b) This section shall not apply with respect to wages paid on or after January 1, 2024. (Amended by Stats. 2023, Ch. 196, Sec. 25. (SB 143) Effective September 13, 2023.)", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=1126", + "quote": "compute and assess the amounts of employer and worker contributions payable by the employing unit, adding thereto a penalty of 15 percent of the amount of contributions.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=1112", + "quote": "Any employer who without good cause fails to pay any contributions required of him or her or of his or her workers ... within the time required shall pay a penalty of 15 percent", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3252", + "quote": "neither an employee nor his or her employer shall be liable for the worker contributions required under this division with respect to wages paid by the employer while the employee is covered by an approved voluntary plan.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=3260", + "quote": "An employer may, but need not, assume all or part of the cost of the plan, and may deduct from the wages of an employee covered by the plan", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/california/22-CCR-986-1", + "quote": "Contributions from employees at the rates prescribed in the code shall be deducted by the employer from wages paid to the employee at the time such wages are paid and shall be transmitted to the department by the employer", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "UI and ETT are employer paid contributions. SDI and PIT are withheld from employee wages.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "The SDI program is funded through a payroll deduction from employees\u2019 wages. Employers withhold 1.3 percent (.013) for SDI on all subject wages paid to each employee in a calendar year.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "The SDI program, which includes DI and PFL, is funded through mandatory employee payroll deductions for most California workers.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de44.pdf", + "quote": "Once a VP is approved, the employer is no longer required to send SDI withholdings to EDD for those employees covered by the VP.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de201.pdf", + "quote": "Effective January 1, 2024, there is no SDI taxable wage limit. (Table: Who Pays = Employee; Rate = 1.3%; Wage Limit = None. DE 201 Rev. 12 (1-26), PDF created 2025-12-03.)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employer may pay an employee\u2019s share of Social Security and Medicare (taxes under the Federal Insurance Contributions Act [FICA]), State Disability Insurance (SDI), and federal and state income taxes without taking any money from the employee\u2019s salary.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231q.pdf", + "quote": "An employee\u2019s wages will increase if an employer pays their employees\u2019 share of FICA and SDI without taking it out from their wages.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/disability/employer_requirements/", + "quote": "Employers must: Withhold mandatory payroll deductions for DI and PFL. Send contributions to the EDD Tax Branch.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "California employees pay mandatory contributions to reimburse the Disability Insurance (DI) Fund for State Disability Insurance (SDI) coverage.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20260618174055/https://edd.ca.gov/en/disability/Contribution_Rates_and_Benefit_Amounts/", + "quote": "2026 Voluntary Plan Employee Contribution and Benefit Rate Effective January 1, 2026 Employee Contribution Rate 1.3% (capture 2026-06-18; page dateModified 2026-01-01)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "The SDI withholding rate for 2026 is 1.3 percent. Effective January 1, 2024, all wages are subject to SDI contributions.", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251110012902/https://edd.ca.gov/en/payroll_taxes/rates_and_withholding/", + "quote": "SDI withholding rate for 2026 is 1.3 percent (capture 2025-11-10, page dateModified 2025-11-07; the 2025-10-17 capture, dateModified 2025-10-03, lists only 2025 and earlier)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/vp-general-release-letter-2026.pdf", + "quote": "Starting January 1, 2026, the SDI worker contribution rate will be 1.3 percent of an employee\u2019s annual gross taxable wages. Reference: California Unemployment Insurance Code (CUIC) section 984(a) (1).", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de3395.pdf", + "quote": "2026 7,000 None1 3.4 6.2 0.1 1.3 (DE 3395 Rev. 69 (1-26), columns UI base, DI base, UI unrated, UI rated, ETT %, SDI Tax %; PDF created 2025-12-17)", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/unemployment/pdf/january-2026-di-fund-forecast.pdf", + "quote": "increased to 1.3 percent in 2026, and is projected to remain at 1.3 percent in 2027.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de2040.pdf", + "quote": "An employer is authorized to deduct from a VP employee\u2019s wages an amount that does not exceed the current SDI plan contribution rate.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de2040.pdf", + "quote": "Any employer who operates a VP without EDD approval will be responsible for all SDI contributions withheld from the wages of employees.", + "fetched_and_read": true + }, + { + "url": "https://edd.ca.gov/siteassets/files/pdf_pub_ctr/de231tp.pdf", + "quote": "B. Employee contributions to a qualified cash or deferred compensation plan as defined in Section 401(k) of the IRC. [SDI column:] Subject Section 926 of the CUIC, unless benefit under an IRC Section 125 cafeteria plan.", + "fetched_and_read": true + }, + { + "url": "https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202120220SB951", + "quote": "This bill would remove that limitation on January 1, 2024. [Approved by Governor September 30, 2022.]", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/ca/tax/payroll/disability/ca_employee_state_disability_insurance_contribution.py", + "quote": "rate = parameters(period).gov.states.ca.tax.payroll.disability.employee_rate / return rate * person(\"ca_payroll_tax_gross_wages\", period) [employee_rate.yaml: 0000-01-01: 0.013]", + "fetched_and_read": true + }, + { + "url": "file:///Users/maxghenis/PolicyEngine/policybench/results/local/adds202609/triage/.venv-pe21517/lib/python3.13/site-packages/policyengine_us/variables/gov/states/tax/payroll/employee_state_payroll_tax.py", + "quote": "Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules. (TaxUnit; adds includes ca_employee_state_payroll_tax)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. I independently confirmed the 2026 rate (1.3%), employee share (the full rate, since the State Plan has no employer SDI rate) and the absence of a wage cap. Sources: UIC 984/985(b), DE 201, DE 3395 Rev. 69 (1-26), DE 44 Rev. 52 (4-26), the VP General Release Letter dated 2025-12-31, the January 2026 DI Fund Forecast, and the EDD rates page (Wayback 2025-11-10, dateModified 2025-11-07). All predate 2026-07-03.", + "The researcher's engine checks used single-adult households. The real scenarios differ: scenario_005 and scenario_099 are joint two-earner households, and both they and scenario_023 include traditional and Roth 401(k) deferrals. I re-ran the installed engine (policyengine_us 2.15.17) with those compositions for 2026. scenario_005 gave per-person SDI 2340 + 3250 = 5590. scenario_099 gave 1105 + 1040 = 2145. scenario_022 gave 684.255. scenario_023 gave 226.75446 on $17,442.65. The 401(k) deferrals do not reduce the engine's SDI base (ca_payroll_tax_gross_wages = 180,000 / 250,000 etc.). That is correct law: DE 231TP marks employee 401(k) contributions 'Subject' for SDI under UIC 926.", + "scenario_023: the prompt shows wages of $17,443, which at 1.3% gives $226.759 ($226.76). The engine's $226.75 comes from the unrounded $17,442.65. The gap is $0.01, well within $1.", + "Engine parameter employee_rate.yaml has a single value, 0.013 from 0000-01-01, and no cap. That is correct for TY2026 but wrong for other years: DE 3395 shows 1.2% for 2025, 1.1% for 2024, and 0.9% with a $153,164 DI wage base for 2023. It does not affect this benchmark.", + "Minor notation slip in the researcher's caveat. In DE 231Q, S is the salary paid and W is the reportable (grossed-up) wage, so W = S/(1-R) (example: $1,000 \u00f7 (1 \u2013 .0895) = $1,098.30). The researcher wrote 'grossed up by W/(1-R)'. The substance is right: the gross-up applies when the employer absorbs the employee SDI share, and the engine does not model it." + ], + "citation_problems": [ + "No quoted statute or EDD text failed verification. I re-fetched the leginfo pages for UIC 984, 985, 986, 2901, 2902, 3252, 3260, 926 and 710.5, plus the SB 951 bill text, and the EDD PDFs and pages. Each quote the researcher relied on appears verbatim, allowing for curly versus straight apostrophes.", + "Title 22 CCR regulations (986-1, 986-2, 926-1, 3260-1) could only be read on the LII (Cornell) re-host. The official govt.westlaw.com/calregs host returned a Cloudflare JavaScript challenge, so the regulation text is not confirmed on the official host.", + "The DE 44 PDF's embedded metadata title reads '2025 California Employer's Guide (DE 44) Rev. 52 (4-26)'. Its cover and body say 2026 and give the 2026 SDI rate of 1.3%. This is cosmetic.", + "The DE 2040 page footer reads 'DE 2040 Rev. 7 (12-20)' while its PDF metadata says 'Revision 5'. The server Last-Modified is 2022-03-30, so this guidance predates 2026 but is the current posted version.", + "The director's formal UIC 984(a)(1) public-record rate statement, due by 2025-10-31, was not located by me either. The earliest dated public evidence of the 2026 rate I found is the EDD rates page with dateModified 2025-11-07, captured by Wayback on 2025-11-10." + ], + "reasoning": "I could not refute the classification; the sources I read myself support it.\n\nStatute: UIC 984(a)(1) says \"Each worker shall pay worker contributions\", and s. 2901 repeats this. UIC 986(a)(1) says each employer \"shall\" withhold the contribution in trust from wages when they are paid. Under 986(a)(2), an employer that agrees to pay the contribution without deducting it must still hold the amount in trust and send it in (986(b)). The employer has no SDI rate of its own under the State Plan (986(b) contrasts the worker contributions with the employer's \"own contributions for the Unemployment Fund\").\n\nAdministration lens:\n- Regulation: 22 CCR 986-1 (read on LII) says employee contributions \"shall be deducted by the employer\".\n- EDD guidance: the DE 44, the Employer Requirements page and the Contribution Rates page describe SDI as mandatory employee payroll deductions. \"Employers must: Withhold mandatory payroll deductions for DI and PFL.\" The DE 44 says \"Who Pays? Employee\".\n- Paying the employee share: the employer may pay it instead of deducting it (DE 231Q; s. 986(a)(2)). This mirrors the FICA option. The amount the employer pays becomes extra SDI-subject wages, grossed up as W = S/(1-R) with R = .0130 for 2026. So the contribution is still owed on the worker's wages; only who funds it changes. That is the reverse of an optional pass-through, where the premium is the employer's liability and deducting is discretionary.\n- Liability if not deducted: I found no rule that makes the employee liable to the state. Collection runs against the employer: UIC 1126 assesses \"employer and worker contributions payable by the employing unit\", s. 1112 imposes a 15% penalty on an employer that fails to pay contributions \"required of ... his or her workers\", s. 1113 adds interest, and s. 1735 makes responsible officers personally liable. This makes the withholding duty enforceable. It does not make it optional.\n\nThe only genuinely optional pass-through is an approved Voluntary Plan. Under one, neither the employee nor the employer owes the State Plan contribution (s. 3252(a)). The employer \"may, but need not\" bear the cost and may deduct up to the State Plan amount (s. 3260; DE 2040). A VP needs majority employee consent and EDD approval (s. 3254), and the benchmark prompt gives no facts suggesting one. Other narrow carve-outs are the religious exemption (s. 2902) and coverage exclusions (public entities except as listed in s. 2606, some domestic workers, and others). None of them is triggered by the household facts.\n\nRate and cap: 1.3% for 2026 on all wages with no cap. The no-cap rule comes from s. 985(b), enacted by SB 951 (approved 2022-09-30) as amended by SB 143 (effective 2023-09-13). The rate was published on EDD's site by 2025-11-07/10 and in the DE 201 (created 2025-12-03), DE 3395 (2025-12-17), the VP letter (2025-12-31), the DI forecast (2026-01-07) and DE 44 (2026-04-17), all before 2026-07-03.\n\nEngine: I read the source this session. The engine computes 0.013 \u00d7 ca_payroll_tax_gross_wages, and that base does not subtract 401(k) deferrals, which matches DE 231TP. I re-ran the actual household compositions and got 5590, 684.255, 226.754 and 2145, matching all four benchmark amounts within $0.01.\n\nClassification: mandatory_employee_withholding. The researcher's classification, rates, amounts and caveats stand. The only corrections are minor: the gross-up notation, the single-adult engine runs, and the LII-only regulation host." + } + ] + }, + { + "key": "PA", + "research": { + "program": "Pennsylvania Unemployment Compensation employee contribution (UC Law s. 301.4, 43 P.S. s. 781.4)", + "state": "PA", + "engine_variables": [ + "pa_employee_unemployment_compensation_contribution", + "pa_employee_state_payroll_tax", + "employee_state_payroll_tax", + "parameter gov.states.pa.tax.payroll.unemployment.employee_rate" + ], + "premium_liability": "The employee owes it. Section 301.4(a) says each employe \"shall pay\" contributions on all wages paid for covered employment. Section 301.4(b) makes the employer the mandatory collection agent: it \"shall withhold, in trust\" those contributions from wages when the wages are paid and must send them to the Department. The employer's own UC contributions are a separate tax (s. 301 and related sections) on the first $10,000 of wages. The s. 301.4 employee contribution is not a share of an employer premium.", + "employee_share_rule": "employer_must_withhold", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "UC Law s. 301.4(a), 43 P.S. s. 781.4(a) (official PA General Assembly unconsolidated statutes, Chapter 3)", + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "Each employe shall pay contributions at a rate ... as set forth in section 301.7 for each calendar year thereafter of all wages paid for \"employment\" as defined by the act without regard to the limitation specified in section 4(x)(1)", + "fetched_and_read": true + }, + { + "citation": "UC Law s. 301.4(b), 43 P.S. s. 781.4(b)", + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "Each employer subject to this act shall be responsible for withholding and shall withhold, in trust, such contributions from the wages of his employes at the time such wages are paid", + "fetched_and_read": true + }, + { + "citation": "UC Law s. 301.4(c), 43 P.S. s. 781.4(c)", + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "Any employer who is an individual, or any officer or agent of any employer, who violates the trust provision of this section, fails to withhold, hold in trust or fails to transmit to the department all contributions withheld", + "fetched_and_read": true + }, + { + "citation": "UC Law s. 301.8(a), 43 P.S. s. 781.8(a) (DLI compilation, 2022 edition, A-1936-1 REV 12-22; the booklet says it is not an official copy)", + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/uc_law.pdf", + "quote": "the secretary shall redetermine the rates of the surcharge, employe tax, additional contributions and benefit reduction otherwise applicable under sections 301.7 and 404(e)(4)", + "fetched_and_read": true + }, + { + "citation": "UC Law s. 301.8(b)(3), 43 P.S. s. 781.8(b)(3) (DLI compilation, 2022 edition)", + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/uc_law.pdf", + "quote": "the employe tax under section 301.4 shall yield two hundred thirty million dollars ($230,000,000)", + "fetched_and_read": true + }, + { + "citation": "UC Law s. 4(x)(1), 43 P.S. s. 753(x)(1): the $10,000 limit applies only to employer contributions (DLI compilation, 2022 edition)", + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/uc_law.pdf", + "quote": "For purposes of paying employer contributions, that part of the remuneration paid to an individual by each of his employers during a calendar year that exceeds", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "Employers subject to the PA UC Law are responsible for withholding employee contributions based on total gross wages earned in covered employment as defined under the Law.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "Employee contributions are based on an individual's total (gross) wages and are not limited to the taxable wage base in effect for employer contributions.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/reimbursable-employers", + "quote": "When applicable, all employers are required to withhold employee contributions at the time wages are paid, regardless of the method used to pay UC liability.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/yearly-tax-highlights", + "quote": "Employee contributions (UC withholding) is 0.07 percent (70 cents per $1,000 gross wages) ... There is no cap on the gross wages upon which employee withholding is calculated.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-749%20rev%2009-25.pdf", + "quote": "Employee Withholding on Gross Wages [A] ... 2026 .0007 ... A. No cap on gross wages for employee withholding. ... The Trigger Percentage as of July 1, 2025 calculated to 119%", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/budget/documents/for-commonwealth-agencies-and-employees/for-agencies/payroll/payroll-memo-26-01-federal-withholding-state-unemployment-tax-2026.pdf", + "quote": "The Department of Labor and Industry confirmed that the employee Unemployment Tax will remain at .07 percent for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-748%20rev%2009-25.pdf", + "quote": "THERE IS A 9.2% SURCHARGE ADJUSTMENT, 0.60% ADDITIONAL CONTRIBUTIONS AND 0.0% INTEREST FACTOR IN EFFECT FOR 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/ucp-forms/ucp-36.pdf", + "quote": "Both reimbursable and contributory employers are required to withhold employee contributions from each employee's gross wages.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/calculating-contributions--penalties-and-interest", + "quote": "The employee contribution rate has been set at 0.07 percent for the calendar year 2027.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pabull?file=/secure/pabulletin/data/vol56/56-3/88.html", + "quote": "The trigger percentage calculated by the Secretary on July 1, 2025, was not at least 250%.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pacode?file=/secure/pacode/data/034/chapter63/chap63toc.html", + "quote": "(No quote. I reviewed the table of contents of 34 Pa. Code ch. 63. No section addresses s. 301.4 employee contributions or an employer paying the employee share.)", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "No, as far as I can find. Section 301.4(a) puts the contribution on each employe (\"shall pay\"), and s. 301.4(b) requires the employer to withhold it from wages (\"shall withhold, in trust\"). Section 301.4(c) penalizes failing to withhold. Neither s. 301.4 nor the 34 Pa. Code ch. 63 table of contents provides for an employer to pay the employee contribution instead of withholding it, and none of the DLI pages I read mention such an election. The only \"employer pays without deduction\" provision I saw is s. 4(x)(3), and it covers the federal FICA employee tax, not this contribution. I searched for an express prohibition and did not find one; what I found is mandatory withholding language and no authorization to elect otherwise.", + "rate_2026": { + "total_rate": "Employee contribution: 0.07% (.0007) of all gross wages in covered employment. This is a separate tax, not a split of a combined premium. For context only, employer contributions are a different tax on the first $10,000: the non-delinquent experience range in 2026 is .014190 to .103734, including a 9.2% surcharge adjustment, 0.60% additional contributions and a 0.0% interest factor. The trigger percentage as of July 1, 2025 was 119%.", + "max_employee_share": "0.07% of gross wages. The whole s. 301.4 contribution is the employee's, so there is no cap or 'up to' share.", + "wage_base_or_cap": "No cap for the employee contribution. Section 301.4(a) applies 'without regard to' the s. 4(x)(1) limit. The $10,000 taxable wage base applies only to employer contributions.", + "sources": [ + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-749%20rev%2009-25.pdf", + "published": "DLI form UC-749 REV 09-25. PDF metadata: created 2025-09-17, modified 2025-09-22. Server Last-Modified: 2025-11-07. Shows 2026 employee withholding .0007, no cap, trigger 119%." + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-748%20rev%2009-25.pdf", + "published": "DLI form UC-748 REV 09-25, 'Effective January 1, 2026'. PDF created 2025-09-17. Server Last-Modified: 2025-11-07. Employer-side 2026 rates." + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/budget/documents/for-commonwealth-agencies-and-employees/for-agencies/payroll/payroll-memo-26-01-federal-withholding-state-unemployment-tax-2026.pdf", + "published": "PA Office of the Budget, BCPO Payroll Memo #26-01, dated January 13, 2026. Says DLI confirmed .07% for 2026." + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/yearly-tax-highlights", + "published": "DLI web page, undated. Read 2026-10-05. Gives 0.07% for both 2026 and 2025, with no cap." + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "published": "Statute s. 301.4, last amended July 2, 2013 (P.L.195, No.34). Sets the no-cap base and points the rate to ss. 301.7/301.8." + } + ], + "published_before_2026_07_03": "Yes. UC-749 and UC-748 REV 09-25 were created in September 2025 and served since at least 2025-11-07. The BCPO memo is dated 2026-01-13. Statute s. 301.4 dates from 2013. All are before 2026-07-03." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_028", + "engine_amount": 42, + "law_amount_at_max_employee_share": 42, + "matches_within_1_dollar": true, + "note": "$60,000 x 0.0007 = $42.00. No wage cap applies (s. 301.4(a)). I ran policyengine-us 2.15.17 for a PA 2026 single earner with $60,000 employment_income: pa_employee_unemployment_compensation_contribution = 42.0, employee_state_payroll_tax = 42.0." + }, + { + "scenario_id": "scenario_123", + "engine_amount": 101.5, + "law_amount_at_max_employee_share": 101.5, + "matches_within_1_dollar": true, + "note": "$145,000 x 0.0007 = $101.50. The rate is flat with no cap, so the result is the same however the wages are split between earners or employers. Engine runs: one earner at $145,000 gives 101.5. Two earners at $100,000 + $45,000 give 70.00 + 31.50 = 101.5 total employee_state_payroll_tax." + } + ], + "engine_documentation": "Read this session from the installed policyengine_us (2.15.17 venv).\n\n- **variables/gov/states/pa/tax/payroll/unemployment/pa_employee_unemployment_compensation_contribution.py:** label \"Pennsylvania employee unemployment compensation contribution\". Documentation: \"Employee-side Pennsylvania unemployment compensation withholding.\" Entity Person, YEAR, defined_for StateCode.PA. Formula: parameters.gov.states.pa.tax.payroll.unemployment.employee_rate * max_(0, employment_income). There is no cap, no covered-employment test and no employer-election input.\n- **parameters/.../unemployment/employee_rate.yaml:** 2015-01-01: 0.0007; 2018-01-01: 0.0006; 2023-01-01: 0.0007. References: DLI Contribution Rate Chart UC-749 Rev 08-21, DLI 'Calculating Contributions, Penalties, and Interest', and DLI 'Employee Withholding'.\n- **variables/.../payroll/pa_employee_state_payroll_tax.py:** label \"Pennsylvania employee state payroll tax\". Documentation: \"Employee-side Pennsylvania payroll-funded contributions, including unemployment compensation withholding.\" It adds only pa_employee_unemployment_compensation_contribution.\n- **variables/gov/states/tax/payroll/employee_state_payroll_tax.py:** TaxUnit. Documentation: \"Employee-side mandatory state payroll taxes and payroll-funded contributions, aggregated from jurisdiction-specific rules.\" Its adds list includes pa_employee_state_payroll_tax among 14 states.\n- **Employer-side files:** taxable_wage_base.yaml ($10,000, cites UC Law s. 4(x)(1)) and default_rate.yaml (0.0382) are used only by pa_employer_state_unemployment_tax, not by the employee formula.\n- **Test** tests/policy/baseline/gov/states/pa/tax/payroll/unemployment/pa_employee_unemployment_compensation_contribution.yaml: 2026 with $27,000 gives 18.90; 2022 gives 16.20.\n- None of the PA files I read contain a 'maximum permitted employee share' assumption like the Minnesota variable's. The PA contribution is modeled as a flat mandatory withholding.", + "caveats": [ + "Coverage: s. 301.4(a) applies only to wages for 'employment' as the Act defines it (s. 4(l)). DLI's UCP-36 handbook lists common exclusions with no withholding: clergy and religious organizations, students working for their school, a sole proprietor's spouse or minor children, commission-only real estate and insurance agents, domestic workers paid under $1,000 a quarter, small agricultural employers, elected officials and inmates. The engine applies 0.07% to all positive employment_income of PA residents. For the benchmark's plain-wage households this is the standard case.", + "Where the work is done, not where the worker lives: UC coverage follows employment localized in PA (UCP-36 refers to the 'Localized and Nonlocalized Employment' pamphlet), while the engine keys on household state_code. A PA resident working in another state would generally not have PA UC withholding, and a nonresident working in PA would. The benchmark prompt gives only the state, so residence = work state is the natural reading.", + "I did not verify this session whether federal civilian or railroad employment falls outside PA UC coverage.", + "No employer-size threshold, private/self-insured plan alternative or collective-bargaining opt-out exists for this contribution. Withholding applies to reimbursable employers (nonprofits and governments) as well as contributory employers, and DLI says it is not subject to appeal. Unlike Minnesota-style employer pass-through programs, there is no 'up to' employee share.", + "Rate mechanics: s. 301.4(a) points to the s. 301.7 trigger schedule, and s. 301.8 redetermines the employe tax every five years to yield $230M. On their face the s. 301.7(b) schedule numbers (e.g. 0.05% for a 110-125% trigger) differ from the published 0.07%, because the s. 301.8 redetermined rates replace them for the five-year period. I read s. 301.7/301.8 only in DLI's 2022 compilation, which says it is not an official copy. I checked s. 301.4 itself against the official palegis.us text.", + "Post-cutoff material: the DLI 'Calculating Contributions' page now says 0.07% for calendar year 2027, an update after July 2026. The UCP-36 handbook I downloaded is marked REV 06-26, and its PDF was created 2026-07-14, so I used it only for mechanics, not for the rate. The 2026 rate rests on pre-cutoff sources: UC-749/UC-748 REV 09-25 (September 2025) and the BCPO memo of 2026-01-13.", + "Failure to withhold: s. 301.4(c) subjects the employer and its officers and agents to penalty and collection provisions (ss. 301(a)(2), 308-309). The statute text I read does not say who owes an unwithheld amount; the employee's s. 301.4(a) obligation stands either way.", + "WebFetch quotes of DLI HTML pages come from a model-mediated page extraction. I extracted the PDF quotes (statute compilation, UC-749, UC-748, BCPO memo, UCP-36) directly with pdftotext. An attempt to fetch Wayback snapshots of the DLI HTML pages from before the cutoff did not finish." + ], + "confidence": "high", + "summary": "This is a mandatory employee contribution collected by mandatory employer withholding, not an optional employer pass-through.\n\n- **Who owes it:** UC Law s. 301.4(a) (43 P.S. s. 781.4) says each employe \"shall pay\" contributions on all wages for covered employment, \"without regard to\" the $10,000 s. 4(x)(1) limit.\n- **How it is collected:** s. 301.4(b) says each employer \"shall withhold, in trust\" the contributions and send them to the Department. Section 301.4(c) penalizes failing to withhold.\n- **Employer election:** I found no provision letting an employer pay the employee contribution instead of withholding it, in the statute, in the 34 Pa. Code ch. 63 table of contents, or on the DLI pages. DLI says all employers, reimbursable ones included, must withhold.\n- **2026 rate:** 0.07% of gross wages with no cap. Sources are DLI's UC-749 REV 09-25 rate chart (created September 2025; it records the July 1, 2025 trigger at 119%) and a PA Office of the Budget memo of January 13, 2026. Both predate 2026-07-03.\n- **Engine:** pa_employee_unemployment_compensation_contribution = 0.0007 x employment_income, with no cap.\n- **Engine vs. law:** scenario_028 is $60,000 x 0.0007 = $42.00, and scenario_123 is $145,000 x 0.0007 = $101.50. Both match the law exactly, and I confirmed both by running the engine. Because the rate is flat and uncapped, the result does not depend on how wages are split.\n- **Main caveat:** the engine applies the rate to all wages of PA residents. The law applies to covered employment localized in PA and excludes listed services such as clergy, student workers and family employment." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "Section 301.4. Contributions by Employes.--(a) Notwithstanding any other provision of this act, each employe shall pay contributions", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "of all wages paid for \"employment\" as defined by the act without regard to the limitation specified in section 4(x)(1) of this act.", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "(b) Each employer subject to this act shall be responsible for withholding and shall withhold, in trust, such contributions from the wages of his employes at the time such wages are paid", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "who violates the trust provision of this section, fails to withhold, hold in trust or fails to transmit to the department all contributions withheld from the wages of his employes", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "if the trigger percentage as of July 1 of the preceding calendar year is less than two hundred fifty per centum (250%), the rates determined under paragraph (3) shall apply.", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "the employe tax under section 301.4 shall yield two hundred thirty million dollars ($230,000,000)", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "\"Contributions\" also means, where appropriate in this act, money payments required to be paid into the Unemployment Compensation Fund by employes as provided in this act.", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "(1) For purposes of paying employer contributions, that part of the remuneration paid to an individual by each of his employers during a calendar year that exceeds", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "(3) The payment by an employer (without deduction from remuneration of the employe) of the tax imposed upon an employe under section three thousand one hundred one of the Federal Internal Revenue Code", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "For the periods in which employee contributions are in effect, all employers are required to withhold employee contributions at the time wages are paid, regardless of whether an employer is contributory or reimbursable.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/yearly-tax-highlights", + "quote": "Employee contributions (UC withholding) is 0.07 percent (70 cents per $1,000 gross wages). This applies to all employees, including employees of reimbursable employers, and is not subject to appeal.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/reimbursable-employers", + "quote": "When applicable, all employers are required to withhold employee contributions at the time wages are paid, regardless of the method used to pay UC liability.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/calculating-contributions--penalties-and-interest", + "quote": "The employee contribution rate has been set at 0.07 percent for the calendar year 2027.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-749%20rev%2009-25.pdf", + "quote": "Employee Withholding on Gross Wages [A] ... 2026 .0007 ... A. No cap on gross wages for employee withholding. ... The Trigger Percentage as of July 1, 2025 calculated to 119%", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-748%20rev%2009-25.pdf", + "quote": "THERE IS A 9.2% SURCHARGE ADJUSTMENT, 0.60% ADDITIONAL CONTRIBUTIONS AND 0.0% INTEREST FACTOR IN EFFECT FOR 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/budget/documents/for-commonwealth-agencies-and-employees/for-agencies/payroll/payroll-memo-26-01-federal-withholding-state-unemployment-tax-2026.pdf", + "quote": "The Department of Labor and Industry confirmed that the employee Unemployment Tax will remain at .07 percent for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/ucp-forms/ucp-36.pdf", + "quote": "Both reimbursable and contributory employers are required to withhold employee contributions from each employee's gross wages.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/uc_law.pdf", + "quote": "the secretary shall redetermine the rates of the surcharge, employe tax, additional contributions and benefit reduction otherwise applicable under sections 301.7 and 404(e)(4)", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pabull?file=/secure/pabulletin/data/vol56/56-3/88.html", + "quote": "The trigger percentage calculated by the Secretary on July 1, 2025, was not at least 250%.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pacode?file=/secure/pacode/data/034/chapter63/chap63toc.html", + "quote": "(No quote. The 34 Pa. Code ch. 63 table of contents has no section on s. 301.4 employee contributions, withholding, or an employer paying the employee share.)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None that change the classification or the amounts. 2026 employee rate 0.07% (.0007) with no wage cap is confirmed by UC-749 REV 09-25 (PDF created 2025-09-17, modified 2025-09-22; server Last-Modified 2025-11-07), the BCPO memo dated 2026-01-13 (server Last-Modified 2026-01-13), and DLI's yearly highlights page. All predate 2026-07-03.", + "Engine amounts reproduced this session with policyengine-us 2.15.17: $60,000 gives 42.0; $145,000 gives 101.5; $100,000 + $45,000 gives 70.00 + 31.50 = 101.5. Both match 0.0007 x gross wages, so they agree with the law.", + "Small imprecision in the research's rate mechanics: it says the s. 301.8 redetermined rates 'replace' the s. 301.7(b) schedule for the five-year period. The official text of s. 301.8(b)(1) applies the redetermined rates only when the trigger percentage as of July 1 of the preceding year is below 250%. That condition holds for 2026: the trigger was 119% (UC-749), and the PA Bulletin (56 Pa.B. 448) says it 'was not at least 250%'. The conclusion stands. Without s. 301.8, the s. 301.7(b)(3) schedule alone would give 0.05% at a 119% trigger.", + "Statute dating: the research says s. 301.4 was 'last amended July 2, 2013 (P.L.195, No.34)'. The official text also shows later amendments: (e)(2) amended, and (f), (g) and (h) added, by Act 1 of 2017 and Act 60 of 2017 (Dec. 20, 2017, P.L.1191). Subsections (a) and (b), which set the employee liability, the no-cap base and the withholding duty, are not affected. The cutoff conclusion is unchanged." + ], + "citation_problems": [ + "s. 301.4(a) quote is not exactly verbatim. The research quotes 'Each employe shall pay contributions at a rate ...'. The official palegis.us text reads '(a) Notwithstanding any other provision of this act, each employe shall pay contributions at a rate of zero per centum (0.0%) for calendar year 1989 and at a rate as set forth in section 301.7 ...'. 'Each' is capitalized in the research but lowercase in the source, and the 'Notwithstanding' clause in front of it is left out. The meaning is unchanged; the omitted clause actually strengthens the mandate. Every other word in the quote matches, and the ellipsis is used correctly.", + "The research cites the s. 301.8(a), 301.8(b)(3) and 4(x)(1) quotes only to the DLI compilation (uc_law.pdf), which says it is 'NOT an OFFICIAL copy'. I confirmed all three verbatim in the official palegis.us text: s. 301.8 in Chapter 3 and s. 4(x)(1) in the Chapter 1 definitions. The quotes are correct; only the source cited is weaker than it needs to be.", + "All other quotes were confirmed verbatim at their URLs by direct download: the 301.4(b) and 301.4(c) quotes, DLI employee-withholding (2 quotes), reimbursable-employers, yearly-tax-highlights, calculating-contributions, UC-749, UC-748, BCPO memo, UCP-36 and the PA Bulletin 56-3/88. The calculating-contributions page's '2027' sentence is post-cutoff content, as the research itself says." + ], + "reasoning": "I could not refute the classification. I read the official PA General Assembly text of UC Law Article III (Chapter 3) in full for ss. 301, 301.4, 301.7 and 301.8, and the Article I s. 4 definitions (Chapter 1) myself.\n\nWhat the statute says:\n- **Who owes it.** s. 301.4(a) begins 'Notwithstanding any other provision of this act, each employe shall pay contributions', on all wages for 'employment' and without regard to the s. 4(x)(1) $10,000 limit. That is a direct liability on the employee.\n- **How it is collected.** s. 301.4(b) says each employer 'shall be responsible for withholding and shall withhold, in trust' those contributions from wages when the wages are paid.\n- **Penalty.** s. 301.4(c) applies penalties to an employer or officer who 'fails to withhold'.\n- **Definitions.** s. 4(g) says 'Contributions' also means payments 'required to be paid ... by employes'.\n- **Kept separate from the employer premium.** s. 301.6(d) treats 'employe contributions under section 301.4' separately from employer liability.\n\nWhat I searched for and did not find:\n- I searched Chapter 3 for 'deduct', 'withh', 'in lieu', 'on behalf of', 'without deduction', 'may pay' and 'employe shall'. No 'may deduct' language and no employer-election clause turned up.\n- The only 'without deduction' text is s. 4(x)(3). It covers an employer paying the federal FICA employee tax, which is a wage-definition exclusion and has nothing to do with this contribution.\n- The 34 Pa. Code ch. 63 table of contents has no rule on employee contributions.\n- DLI pages say all employers, reimbursable ones included, must withhold, and that the contribution 'is not subject to appeal'.\n\nContext in the law:\n- There is no employer-size threshold, private-plan alternative or collective-bargaining opt-out.\n- The 2026 rate of 0.07% follows from s. 301.8(b)(1): the trigger was 119%, below 250%, so the redetermined rates apply. The figure itself is published in UC-749 REV 09-25 (September 2025) and the BCPO memo (January 13, 2026), both before the 2026-07-03 cutoff.\n\nWhat I confirmed in the engine source:\n- pa_employee_unemployment_compensation_contribution = employee_rate x max_(0, employment_income). It has no cap and no employer-share input.\n- The employee_rate parameter gives 0.0007 from 2023 on.\n- employee_state_payroll_tax adds pa_employee_state_payroll_tax.\n- Running the engine gives 42.0 for scenario_028 and 101.5 for scenario_123, both exact matches to 0.0007 x wages.\n- Unlike the Minnesota variable, the PA files contain no 'maximum permitted employee share' assumption.\n\nRemaining issues are minor and do not change the classification:\n- The capitalization in the s. 301.4(a) quote.\n- The imprecise 'last amended 2013' date.\n- The research's coverage caveats, which I agree with: the engine keys on residence, not on where the work is localized, and it does not apply the s. 4(l) service exclusions." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "(a) Notwithstanding any other provision of this act, each employe shall pay contributions at a rate of zero per centum (0.0%) for calendar year 1989 and at a rate as set forth in section 301.7 for each calendar year thereafter", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "(b) Each employer subject to this act shall be responsible for withholding and shall withhold, in trust, such contributions from the wages of his employes at the time such wages are paid", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "who violates the trust provision of this section, fails to withhold, hold in trust or fails to transmit to the department all contributions withheld ... shall be subject to the provisions of clause (2) of subsection (a) of section 301 and sections 308, 308.1, 308.2, 308.3 and 309", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "Beginning in 1992 and each fifth year thereafter, the secretary shall redetermine the rates of the surcharge, employe tax, additional contributions and benefit reduction otherwise applicable under sections 301.7 and 404(e)(4).", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3", + "quote": "The rates as redetermined shall take effect on January 1 of the following calendar year and shall remain in effect for five years.", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "\"Contributions\" also means, where appropriate in this act, money payments required to be paid into the Unemployment Compensation Fund by employes as provided in this act.", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "(1) For purposes of paying employer contributions, that part of the remuneration paid to an individual by each of his employers during a calendar year that exceeds", + "fetched_and_read": true + }, + { + "url": "https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=1", + "quote": "(6) Service performed in the employ of the United States Government or of an instrumentality of the United States exempt under the Constitution of the United States from the contributions imposed by this act", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "For the periods in which employee contributions are in effect, all employers are required to withhold employee contributions at the time wages are paid, regardless of whether an employer is contributory or reimbursable.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "Failure to withhold employee contributions and to promptly pay such contributions withheld in trust can result in civil fines and/or criminal prosecution of the employer and/or personal liability", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/employee-withholding", + "quote": "Employee contributions are based on an individual's total (gross) wages and are not limited to the taxable wage base in effect for employer contributions.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/reimbursable-employers", + "quote": "When applicable, all employers are required to withhold employee contributions at the time wages are paid, regardless of the method used to pay UC liability.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/yearly-tax-highlights", + "quote": "Employee contributions (UC withholding) is 0.07 percent (70 cents per $1,000 gross wages). This applies to all employees, including employees of reimbursable employers, and is not subject to appeal.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-749%20rev%2009-25.pdf", + "quote": "Employee Withholding on Gross Wages [A] ... 2026 ... .0007 ... A. No cap on gross wages for employee withholding. ... The Trigger Percentage as of July 1, 2025 calculated to 119%", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/employer/uc-tax-rates/uc-748%20rev%2009-25.pdf", + "quote": "THERE IS A 9.2% SURCHARGE ADJUSTMENT, 0.60% ADDITIONAL CONTRIBUTIONS AND 0.0% INTEREST FACTOR IN EFFECT FOR 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/budget/documents/for-commonwealth-agencies-and-employees/for-agencies/payroll/payroll-memo-26-01-federal-withholding-state-unemployment-tax-2026.pdf", + "quote": "The Department of Labor and Industry confirmed that the employee Unemployment Tax will remain at .07 percent for 2026.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pabull?file=/secure/pabulletin/data/vol56/56-3/88.html", + "quote": "The trigger percentage calculated by the Secretary on July 1, 2025, was not at least 250%.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/agencies/dli/resources/for-employers-and-educators/how-to-file/uc-tax/calculating-contributions--penalties-and-interest", + "quote": "Employers covered under the PA UC Law are responsible for withholding employee contributions from each employee's gross wages. The employee contribution rate is determined annually.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/ucp-forms/ucp-36.pdf", + "quote": "Both reimbursable and contributory employers are required to withhold employee contributions from each employee's gross wages.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pacode?file=/secure/pacode/data/034/chapter63/chap63toc.html", + "quote": "(No operative quote.) I searched the full text of 34 Pa. Code ch. 63 for withholding and employee contributions. The only hit is \u00a7 63.99(1), on experience rating after a reimbursable election. Nothing lets an employer pay the employee share instead of withholding it.", + "fetched_and_read": true + }, + { + "url": "https://www.pacodeandbulletin.gov/Display/pacode?file=/secure/pacode/data/034/chapter61/chap61toc.html", + "quote": "(No operative quote.) 34 Pa. Code ch. 61 (including \u00a7 61.3 Wages) and ch. 65 have no provision on s. 301.4 withholding or on an employer paying the employee share.", + "fetched_and_read": true + }, + { + "url": "https://www.pa.gov/content/dam/copapwp-pagov/en/dli/documents/uc/uc_law.pdf", + "quote": "the employe tax under section 301.4 shall yield two hundred thirty million dollars ($230,000,000)", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. The 2026 rate is 0.0007 (0.07%) of all gross wages in covered employment, with no cap. Sources: UC-749 REV 09-25 (PDF created 2025-09-17, modified 2025-09-22, server Last-Modified 2025-11-07), the BCPO memo #26-01 dated 2026-01-13 (Last-Modified 2026-01-13), and the DLI yearly highlights page (undated; 0.07% for 2026). The trigger fact is in 56 Pa.B. 448 (Jan 17, 2026), and the official statute supports the rest. All of these predate 2026-07-03.", + "Rate mechanics, verified on the official palegis.us text: s. 301.8(a) and (d) redetermine the employe tax every fifth June 30 (\u20262017, 2022, 2027\u2026), effective the next Jan 1 for five years. The 2022 redetermination therefore fixes 2023-2027. This matches DLI's table (0.06% for 2018-2022, 0.07% for 2023 and later). Under s. 301.8(b)(1), the redetermined rate applies because the July 1, 2025 trigger (119%) was below 250%, so the s. 301.7(b)(3) schedule rate of 0.05% does not apply.", + "scenario_028: $60,000 x 0.0007 = $42.00. I re-ran policyengine-us 2.15.17 this session: pa_employee_unemployment_compensation_contribution = 42.0, employee_state_payroll_tax = 42.0. Matches.", + "scenario_123: the benchmark prompt gives the head $100,000, the spouse $0, and Child 1 (age 16, a tax-unit member) $45,000. $100,000 x 0.0007 = $70.00 and $45,000 x 0.0007 = $31.50, total $101.50. Engine re-run: person values [70.0, 0, 31.499998], employee_state_payroll_tax = 101.5. Matches. The child's wages count because 'employe' includes minors (s. 4(i)). The s. 4(l)(4)(5) exclusion covers a child under 18 only when the child works for a parent, and the prompt does not say that.", + "Small rounding note: DLI requires withholding to be computed each payroll period and rounded to the cent. Actual annual withholding can therefore differ from the flat annual product by a few cents, well within $1.", + "Unverified: whether the 2022 redetermination used s. 301.8(b)(2) (target $166,666,666) or (b)(3) (target $230M) depends on the s. 301.8(b)(4) year, which turns on whether there are no federal advances and no outstanding bonds. The research cites only (b)(3). This does not change the published 0.07%." + ], + "citation_problems": [ + "The s. 301.4(a) quote starts 'Each employe shall pay contributions\u2026' with a capital E and leaves out the leading clause. The official text reads '(a) Notwithstanding any other provision of this act, each employe shall pay contributions\u2026'. The substance matches. This is a capitalization and truncation issue, not a fabrication.", + "The research says s. 301.4 was 'last amended July 2, 2013 (P.L.195, No.34)'. The palegis.us text shows later changes: (e)(2) was amended by Act 1 and Act 60 of 2017, (f) was added by Act 1 of 2017, and (g) and (h) were added by Act 60 of 2017. Subsections (a)-(c) are unaffected, so the error does not touch the operative text.", + "A caveat says ss. 301.7/301.8 were read only in DLI's unofficial 2022 compilation. Both sections are on the official palegis.us Chapter 3 page. I confirmed the s. 301.8(a) quote and the s. 301.8(b)(3) $230M quote there verbatim. The s. 4(x)(1) quote is also verbatim on the official Chapter 1 page. The caveat can be dropped.", + "An open caveat ('did not verify whether federal civilian or railroad employment falls outside PA UC coverage') is resolved by the official Chapter 1 text. Section 4(l)(4)(6) excludes service for the United States Government. A separate s. 4(l)(4) clause excludes service subject to the Railroad Unemployment Insurance Act. This is a coverage limit the engine does not model. It does not affect the benchmark's unspecified-employer households.", + "UCP-36 is REV 06-26 (PDF created 2026-07-14, Last-Modified 2026-07-14), which is after the cutoff. A search snippet titles the same URL 'REV 01/2026', so an earlier version likely existed, but I could not retrieve it. The research uses UCP-36 only for mechanics, which is appropriate.", + "I could not confirm pre-cutoff Wayback snapshots of the undated DLI HTML pages (archive.org returned 429). The pre-cutoff support for the 2026 rate rests on the UC-749/UC-748 PDFs, the BCPO memo and the statute.", + "I confirmed every other quote in the research verbatim from raw downloads (curl plus HTML or pdftotext extraction). This covers palegis s. 301.4(b) and (c), the DLI employee-withholding, reimbursable-employers, yearly-highlights and calculating-contributions pages, UC-749, UC-748, the BCPO memo, UCP-36 and 56 Pa.B. 448." + ], + "reasoning": "Administration lens. On 'may' versus 'must', every primary source I read uses mandatory language and none uses 'may'. The statute (s. 301.4(b), official palegis.us text) says the employer 'shall be responsible for withholding and shall withhold, in trust'. DLI's employee-withholding page says 'all employers are required to withhold \u2026 regardless of whether an employer is contributory or reimbursable'. The reimbursable-employers page says the same, 'regardless of the method used to pay UC liability'. The yearly highlights say the contribution 'applies to all employees \u2026 and is not subject to appeal'. UCP-36 says both employer types 'are required to withhold'.\n\nThe liability sits on the employee: s. 301.4(a) says 'each employe shall pay contributions'. Section 4(g) defines 'Contributions' to include 'money payments required to be paid \u2026 by employes'. This is a separate employee tax, not a share of an employer premium. DLI says it is not credited to the employer's reserve account, and it is not a FUTA-certified contribution.\n\nOn whether an employer can choose to pay it, I found no provision allowing that. I checked s. 301.4, the full text of 34 Pa. Code chs. 61, 63 and 65, and the DLI employer pages. The only 'employer pays without deduction' language is s. 4(x)(3), and it concerns the federal FICA employee tax. I found no express prohibition either. The mandatory 'shall withhold' plus the s. 301.4(c) penalty for failing to withhold leave no lawful opt-out.\n\nOn employee liability if nothing is withheld, the statute is silent. Enforcement through s. 301.4(c), and from there ss. 301(a)(2), 308-309, runs against the employer and its officers and agents (interest, liens, civil collection, injunction). DLI warns of fines, prosecution and personal liability for the employer and its officers. I found no rule making the employee pay an unwithheld amount and no employer recovery right. The employee's s. 301.4(a) duty stands regardless. This gap does not turn the program into an optional pass-through, because the employer has no discretion.\n\nThe 2026 rate, the absence of a cap and the employer-only $10,000 base are confirmed by pre-cutoff primary sources. Section 301.8's five-year redetermination explains why 0.07% runs from 2023 through 2027.\n\nEngine: I read the installed 2.15.17 source. pa_employee_unemployment_compensation_contribution = employee_rate x max_(0, employment_income); employee_rate is 0.0007 from 2023-01-01; there is no cap and no covered-employment or work-location test. pa_employee_state_payroll_tax adds only that variable, and employee_state_payroll_tax lists it among 14 states. The PA files carry no 'maximum permitted employee share' assumption. That phrase appears in the MN, MA, ME, DE and VT files, not PA.\n\nI re-ran both households: 42.0 and 101.5 (70.00 for the head plus 31.50 for the 16-year-old child), both matching the law exactly. Remaining caveats limit coverage and do not change the classification. The engine ignores covered-employment exclusions such as federal civilian service, railroad service, family employment and clergy, and it ignores work-state localization. The benchmark households are plain wage earners with unspecified employers.\n\nThe citation problems I found are small: a capitalization and truncation in one quote, an incomplete amendment history, and an outdated caveat. None undermines a key quote or the classification. Not refuted." + } + ] + }, + { + "key": "NJ", + "research": { + "program": "New Jersey Temporary Disability Insurance (TDI, State plan) and Family Leave Insurance (FLI, Family Temporary Disability Leave Account) worker contributions", + "state": "NJ", + "engine_variables": [ + "nj_employee_temporary_disability_insurance_contribution", + "nj_employee_family_leave_insurance_contribution", + "nj_temporary_disability_insurance_taxable_wages", + "nj_family_leave_insurance_taxable_wages", + "nj_employee_state_payroll_tax", + "employee_state_payroll_tax", + "gov.states.nj.tax.payroll.temporary_disability_insurance.employee_rate", + "gov.states.nj.tax.payroll.family_leave_insurance.employee_rate", + "gov.states.nj.tax.payroll.temporary_disability_insurance.taxable_wage_base" + ], + "premium_liability": "The worker. Under N.J.S.A. 43:21-7(d)(1)(G)(i), each worker of a covered employer \"shall ... make contributions\" to the State disability benefits fund (TDI). Under (G)(ii), each worker \"shall contribute\" to the Family Temporary Disability Leave Account (FLI). Under 43:21-7(d)(1)(E), the employer must withhold those contributions in trust from wages and send them to the controller. The employer owes its own separate TDI contribution under 43:21-7(e)(1), which \"shall not be deducted\" from wages. FLI has no employer contribution at all. The liability moves to the employer only as a sanction: if it fails to deduct by the next payroll, \"he alone shall thereafter be liable\", and the amount is treated as an employer contribution for collection. Under an approved private plan, the employer funds the benefits. The employer may then charge employees up to the State-plan worker rate, or charge them nothing.", + "employee_share_rule": "employee_must_contribute", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "N.J.S.A. 43:21-7(d)(1)(G)(i) (TDI worker contributions), NJDOL compilation of the Unemployment Compensation Law, Dec 2024, p.52", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "Each worker ... unless the employer is covered by an approved private disability plan or is exempt ... shall, for calendar year 2012 and each subsequent calendar year, make contributions to the State disability benefits fund", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-7(d)(1)(G)(ii) (FLI worker contributions), p.53", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "Each worker shall contribute to the State disability benefits fund, in addition to any amount contributed pursuant to subparagraph (i) ... unless the employer is covered by an approved private disability plan for benefits during periods of family temporary disability leave.", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-7(d)(1)(E) (mandatory employer withholding), p.51", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "Each employer shall, notwithstanding any provision of law in this State to the contrary, withhold in trust the amount of his workers' contributions from their wages at the time such wages are paid", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-7(d)(1)(E) (consequence of failure to deduct), p.51", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "If any employer fails to deduct the contributions of any of his workers ... or fails to make a deduction therefor at the time wages are paid for the next succeeding payroll period, he alone shall thereafter be liable", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-7(b)(4) (worker TDI/FLI wage base), p.~39", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "for purposes of contributions of workers to the State disability benefits fund, including the \"Family Temporary Disability Leave Account\" ... shall be 107 times the Statewide average weekly remuneration paid to workers by employers", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-7(e)(1) (employer TDI contribution is separate and may not be passed to workers), p.56", + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "Such contributions shall become due and be paid by the employer to the controller ... and shall not be deducted, in whole or in part, from the remuneration of individuals in his employ.", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-32(d) (private plans: cap on employee cost), NJDOL TDB Law compilation, July 2020, p.12", + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB Law July 2020.pdf", + "quote": "no greater amount is required to be paid by employees toward the cost of benefits than that prescribed by law as the amount of worker contribution to the State disability benefits fund for covered individuals under the State plan", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-33 (private plans: employer MAY collect employee contributions), TDB Law compilation p.13", + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB Law July 2020.pdf", + "quote": "the employer may during the continuance of the approved private plan collect the required contributions thereto by deduction from the wages paid to covered individuals under such plan, which deduction may be combined with that deduction required by Revised Statutes, section 43:21-7(d)(1)", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-27(a)(1) (FLI covered employer includes non-electing governmental employers; TDI excludes them unless elected)", + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB Law July 2020.pdf", + "quote": "including any governmental entity or instrumentality which is an employer under R.S.43:21-19(h)(5), notwithstanding that the governmental entity or instrumentality has not elected to be a covered employer", + "fetched_and_read": true + }, + { + "citation": "N.J.S.A. 43:21-31 (religious exemption)", + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB Law July 2020.pdf", + "quote": "Such person shall thereupon be exempt from the provisions of this act, including any obligation to make contributions hereunder and any right to receive benefits hereunder.", + "fetched_and_read": true + }, + { + "citation": "57 N.J.R. 1913(a), Sept 2, 2025 (proposed N.J.A.C. 12:15-1.3, 2026 taxable wage base)", + "url": "https://www.nj.gov/labor/assets/PDFs/Legal%20Notices/Notices%20of%20Proposal/57%20N.J.R.%201913_a_.pdf", + "quote": "The new taxable wage base would be $ 44,800 for contributions pursuant to the Unemployment Compensation Law and $ 171,100 for contributions to the State Disability Benefits Fund, including the Family Temporary Disability Leave Account.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.nj.gov/labor/myleavebenefits/employer/index.shtml", + "quote": "Employees: 0.19% on the first $171,100 in covered wages (max $325.09). [TDI] ... Employees: 0.23% on the first $171,100 in covered wages (max $393.53). [FLI] ... Employers: No contribution. [FLI]", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/tdi/", + "quote": "workers contribute 0.19% on the first $171,100 (wage cap) in covered wages earned during this calendar year. The maximum worker contribution for 2026 is $325.09.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/fli/", + "quote": "The family leave program is financed 100% by worker payroll deductions. Employers do not contribute to the program.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/employer-services/rate-info/", + "quote": "2026 Taxable Wage Base (TDI , FLI \u2013 workers only): $171,100", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/lwdhome/press/2025/20251229_newbenefitrates2026.shtml", + "quote": "The taxable wage base for workers covered under the Temporary Disability Insurance and Family Leave Insurance programs increases to $171,100 in 2026 from $165,400 in 2025.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/forms_pdfs/ea/Employer%20Informational%20Packet%20.pdf", + "quote": "Employers have the right to deduct worker contributions from wages paid, but the deduction must be made from the current or next succeeding payroll and noted on the payroll records.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/help/employer_handbook/income_security.shtml", + "quote": "Workers' contribution must not exceed those required under the State Plan", + "fetched_and_read": true + }, + { + "url": "https://business.nj.gov/recent/unemployment-and-disability-insurance-rates-increased-for-2026", + "quote": "The Temporary Disability and Family Leave Insurance taxable wage base increased to $171,100 (from $165,400)", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "Under the State plan: no. The statute grants no election. 43:21-7(d)(1)(G)(i)-(ii) imposes the contribution on each worker, and 43:21-7(d)(1)(E) says the employer \"shall ... withhold in trust\" the workers' contributions, \"notwithstanding any provision of law ... to the contrary\". In practice an employer can end up bearing the worker share: if it fails to deduct by the next succeeding payroll, \"he alone shall thereafter be liable\" and cannot recover the amount. The statute frames that as the consequence of a failure, not as an authorized option. NJDOL's employer packet (2012, revised 2019) does describe deduction as a \"right\". Under an approved private plan (TDI and/or FLI): yes. The employer funds the plan and \"may\" collect employee contributions by payroll deduction (43:21-33), capped at the State-plan worker rate (43:21-32(d)). It may therefore charge less or nothing (a non-contributory plan). Sources: N.J.S.A. 43:21-7(d)(1)(E), (G); 43:21-32(d); 43:21-33.", + "rate_2026": { + "total_rate": "Worker: TDI 0.19% and FLI 0.23%, total 0.42% of covered wages up to the wage base. The employer's separate TDI rate is 0.10% to 0.75% on the first $44,800 per employee (NJDOL employer page). There is no employer FLI contribution. Neither employer figure is part of the employee share.", + "max_employee_share": "State plan: TDI 0.19% x $171,100 = $325.09; FLI 0.23% x $171,100 = $393.53; combined $718.62 per worker per year. These are required amounts, not caps, under the State plan. Under an approved private plan, they cap what employees may be charged (43:21-32(d)).", + "wage_base_or_cap": "$171,100 (2026) for worker TDI and FLI contributions. Statutory formula, 43:21-7(b)(4): 107 x the Statewide average weekly remuneration, rounded up to the next $100, and never lower than the prior year. A worker who pays above the maximum across multiple employers can claim a refund (43:21-7(d)(3)(A)-(B)).", + "sources": [ + { + "url": "https://www.nj.gov/labor/assets/PDFs/Legal%20Notices/Notices%20of%20Proposal/57%20N.J.R.%201913_a_.pdf", + "published": "2025-09-02 (57 N.J.R. 1913(a), proposal of the $171,100 wage base under N.J.A.C. 12:15-1.3; comments due 2025-11-01; adoption notice not fetched)" + }, + { + "url": "https://www.nj.gov/labor/lwdhome/press/2025/20251229_newbenefitrates2026.shtml", + "published": "2025-12-29 (NJDOL press release, wage base $171,100; does not state the worker rates)" + }, + { + "url": "https://business.nj.gov/recent/unemployment-and-disability-insurance-rates-increased-for-2026", + "published": "2026-01-09 (NJ Business Action Center, wage base)" + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/employer/index.shtml", + "published": "undated NJDOL page (2026 worker rates 0.19% / 0.23% and maximums $325.09 / $393.53)" + }, + { + "url": "https://www.nj.gov/labor/ea/employer-services/rate-info/", + "published": "undated NJDOL page (TDI 0.0019, FLI 0.0023 for Jan 1 to Dec 31, 2026; wage base $171,100)" + }, + { + "url": "https://www.nfp.com/insights/nj-announces-2026-disability-and-family-leave-contribution-rates/", + "published": "2025-11-04 (secondary, broker article attributing 0.19% / 0.23% for 2026 to NJDOL; used only to date the rates' publication)" + } + ], + "published_before_2026_07_03": "Yes. The wage base was proposed in the NJ Register on 2025-09-02 and announced by NJDOL on 2025-12-29. The 0.19% and 0.23% worker rates appear on undated official NJDOL pages. A secondary source dated 2025-11-04 reports them. 43:21-7(d)(1)(G)(i)-(ii) requires the estimated rates to be posted on the department's website no later than 60 days after the end of the preceding fiscal year. All three figures predate 2026-07-03. I could not get a dated official NJDOL source for the rates themselves; the Wayback Machine was unreachable." + }, + "engine_amount_checks": [ + { + "scenario_id": "scenario_008 (TDI)", + "engine_amount": 50.92, + "law_amount_at_max_employee_share": 50.92, + "matches_within_1_dollar": true, + "note": "Law: 0.0019 x min($26,800, $171,100) = 0.0019 x 26,800 = $50.92. The engine returned 50.920002 when I ran it this session (NJ, age 40, employment_income $26,800, 2026, policyengine-us 2.15.17). It is the State-plan required worker contribution, so the difference is $0.00." + }, + { + "scenario_id": "scenario_008 (FLI)", + "engine_amount": 61.64, + "law_amount_at_max_employee_share": 61.64, + "matches_within_1_dollar": true, + "note": "Law: 0.0023 x min($26,800, $171,100) = 0.0023 x 26,800 = $61.64. The engine returned 61.640003, and nj_employee_state_payroll_tax returned 112.56 (= 50.92 + 61.64). The difference is $0.00. As the task notes, this output is already excluded from scoring because the engine has no NJ worker UI/WF/SWF variable. The nj/tax/payroll/unemployment directory contains only nj_employer_state_unemployment_tax and nj_taxable_earnings_for_state_unemployment_tax." + } + ], + "engine_documentation": "I read these files this session under .../policyengine_us/variables/gov/states/nj/tax/payroll/ and parameters/gov/states/nj/tax/payroll/.\n- nj_employee_state_payroll_tax (Person, YEAR, defined_for NJ): label \"New Jersey employee state payroll tax\". Its documentation reads \"Employee-side New Jersey payroll-funded contributions, including temporary disability insurance and family leave insurance.\" It adds the TDI and FLI contribution variables.\n- nj_employee_temporary_disability_insurance_contribution: documentation \"Employee-side New Jersey temporary disability insurance contribution.\" Formula: employee_rate x nj_temporary_disability_insurance_taxable_wages.\n- nj_temporary_disability_insurance_taxable_wages: min_(max_(0, employment_income), taxable_wage_base). Its documentation says the wages include federal pre-tax payroll deductions.\n- nj_employee_family_leave_insurance_contribution: documentation \"Employee-side New Jersey family leave insurance contribution.\" Formula: FLI employee_rate x nj_family_leave_insurance_taxable_wages. Those wages are computed the same way and use the TDI taxable_wage_base parameter.\n- Parameters: temporary_disability_insurance/employee_rate is 0 from 0000-01-01 and 0.0019 from 2026-01-01. family_leave_insurance/employee_rate is 0 from 0000-01-01 and 0.0023 from 2026-01-01. temporary_disability_insurance/taxable_wage_base (described as the base for both TDI and FLI) is 0 from 0000-01-01 and 171_100 from 2026-01-01.\n- References: NJ Unemployment Compensation Law R.S. 43:21-7 (nj.gov UI_statute.pdf, pages 46, 60 and 61) and the NJDOL TDI/FLI employer information page.\n- The NJ files contain no \"assuming the employer withholds the maximum permitted employee share\" language like Minnesota's. They model the State-plan rate as applying to every NJ wage earner, with no inputs for private plans, governmental employers, or exemptions.\n- The aggregator employee_state_payroll_tax (TaxUnit) lists nj_employee_state_payroll_tax among EMPLOYEE_STATE_PAYROLL_TAX_COMPONENTS. Its documentation reads \"Employee-side mandatory state payroll taxes and payroll-funded contributions\".", + "caveats": [ + "Approved private plans (43:21-32, -33): an employer may replace the State plan for TDI, FLI or both. Its workers then pay no State-fund contribution for that program ((G)(i), (G)(ii)). Instead, the employer may charge them up to the State-plan worker rate, or nothing for a non-contributory plan. For those workers the employee share is optional for the employer, which is the one pass-through-like feature. The engine applies State-plan rates regardless.", + "Collective bargaining: if employees under a collective bargaining agreement are to contribute to a private plan, a majority must agree by written election, unless the agreement expressly waives that right (43:21-33). Employees not covered by an agreement need no consent.", + "Governmental employers: the State of New Jersey, Rutgers and NJIT are TDI covered employers. Other political subdivisions and instrumentalities are not covered for TDI unless they elect coverage (43:21-27(a)(1)-(2)), so their workers pay no TDI contribution. FLI covers governmental employers that are UI employers even without an election (43:21-27(a)(1); (G)(ii)).", + "Religious exemption (43:21-31): a person who relies on spiritual healing and files a waiver is exempt from the TDB Law, including any obligation to make contributions.", + "Employer status: worker contributions accrue only after the employer becomes an 'employer' subject to the UC law (43:21-19(h)). I did not verify the size or wage threshold for that status this session.", + "Failure to deduct: if the employer misses the current and next payroll, it alone is liable for the worker contributions (43:21-7(d)(1)(E)). A worker whose employer never deducted would actually pay $0.", + "Multiple employers: the $171,100 base applies per employer for withholding. A worker can claim a refund of anything above the annual maximum (43:21-7(d)(3)). The engine caps total employment_income per person, which matches the net after that refund.", + "Statute text came from NJDOL's official compilations: the Unemployment Compensation Law (Dec 2024) and the TDB Law (July 2020 compilation, PDF created Aug 2022). I did not fetch the NJ Legislature's statute database to check for amendments after those compilations, and I did not read the N.J.A.C. 12:16/12:18 rules (both PDFs were over the fetch size limit).", + "The engine's taxable wages are employment_income. Whether NJ 'wages' for TDI/FLI purposes include every federal pre-tax deduction (for example, section 125) was not verified this session.", + "Because the parameter values are 0 before 2026-01-01 (0000-01-01: 0), the engine yields zero NJ TDI/FLI worker contributions for years before 2026.", + "The NJ Division of Taxation 2026 W-2 notice (m6025-2026.pdf) confirms the $171,100 DI/FLI base, but its PDF is dated Aug 10, 2026, after the 2026-07-03 cutoff, so it is not used as a timing source." + ], + "confidence": "high", + "summary": "New Jersey TDI and FLI worker contributions are mandatory and owed by the employee, and the employer must withhold them; they are not an optional employer pass-through.\n- N.J.S.A. 43:21-7(d)(1)(G)(i) says each worker \"shall ... make contributions\" to the State disability benefits fund (TDI). (G)(ii) says each worker \"shall contribute\" to the Family Temporary Disability Leave Account (FLI).\n- 43:21-7(d)(1)(E) requires each employer, \"notwithstanding any provision of law ... to the contrary\", to \"withhold in trust\" those contributions from wages. If it fails to deduct by the next payroll, the employer alone becomes liable.\n- The employer's own TDI contribution (0.10% to 0.75% on $44,800) is separate and may not be deducted from wages. FLI is 100% worker-funded.\n- The one exception that makes the employee share optional is an approved private plan (43:21-32, -33). There the employer \"may\" collect employee contributions, capped at the State-plan worker rate, or charge nothing.\n- Other narrower carve-outs: non-electing local governmental employers (TDI only), the religious exemption, and collective-bargaining election rules for contributory private plans.\n\n2026 figures:\n- Worker TDI is 0.19% and FLI is 0.23% of the first $171,100, for maximums of $325.09 and $393.53.\n- The wage base was proposed in 57 N.J.R. 1913(a) on 2025-09-02 and announced by NJDOL on 2025-12-29.\n- The rates appear on undated NJDOL pages and were reported by 2025-11-04. All of this predates 2026-07-03.\n\nEngine check (policyengine-us 2.15.17, run this session) for scenario_008 ($26,800 wages):\n- TDI: 0.0019 x 26,800 = $50.92.\n- FLI: 0.0023 x 26,800 = $61.64.\n- Both match the law exactly.\n\nThe engine models the State-plan default for every NJ wage earner, which is the correct default given household-only prompts. It does not model private-plan, governmental-employer or exemption cases. Separately, it has no NJ worker UI/WF/SWF contribution variable." + }, + "verdicts": [ + { + "lens": "statute", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "(E) Each employer shall, notwithstanding any provision of law in this State to the contrary, withhold in trust the amount of his workers' contributions from their wages at the time such wages are paid", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "unless the employer is covered by an approved private disability plan or is exempt ... shall, for calendar year 2012 and each subsequent calendar year, make contributions to the State disability benefits fund", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "(ii) Each worker shall contribute to the State disability benefits fund, in addition to any amount contributed pursuant to subparagraph (i) of this paragraph (1)(G)", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "he alone shall thereafter be liable for such contributions, and for the purpose of R.S.43:21-14, such contributions shall be treated as employer's contributions required from him.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "(F) As used in this chapter (R.S.43:21-1 et seq.), except when the context clearly requires otherwise, the term \"contributions\" shall include the contributions of workers pursuant to this section.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "including any contribution which the employer is required to collect from his employees to pay into the funds", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "and shall not be deducted, in whole or in part, from the remuneration of individuals in his employ.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "for purposes of contributions of workers to the State disability benefits fund, including the \"Family Temporary Disability Leave Account\" ... shall be 107 times the Statewide average weekly remuneration paid to workers by employers", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "which deduction may be combined with that deduction required by Revised Statutes, section 43 :21-7(d)(1)", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "Covered individuals so excluded shall be covered by the State plan and subject to the employee contribution required by law to be paid into the State disability benefits fund.", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "no greater amount is required to be paid by employees toward the cost of benefits than that prescribed by law as the amount of worker contribution to the State disability benefits fund for covered individuals under the State plan", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "Such person shall thereupon be exempt from the provisions of this act, including any obligation to make contributions hereunder and any right to receive benefits hereunder.", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "including any governmental entity or instrumentality which is an employer under R.S.43:21-19(h)(5), notwithstanding that the governmental entity or instrumentality has not elected to be a covered employer", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/forms_pdfs/ea/Employer%20Informational%20Packet%20.pdf", + "quote": "Employers have the right to deduct worker contributions from wages paid, but the deduction must be made from the current or next succeeding payroll and noted on the payroll records.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/employer/index.shtml", + "quote": "Employees: 0.19% on the first $171,100 in covered wages (max $325.09). ... Employers: No contribution. Employees: 0.23% on the first $171,100 in covered wages (max $393.53).", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/tdi/", + "quote": "The maximum worker contribution for 2026 is $325.09. This contribution is in the form of a salary deduction that your employer takes from your weekly wages.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/fli/", + "quote": "The family leave program is financed 100% by worker payroll deductions. Employers do not contribute to the program.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/employer-services/rate-info/", + "quote": "2026 Taxable Wage Base (TDI , FLI \u2013 workers only): $171,100", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/help/employer_handbook/taxes_wages.shtml", + "quote": "Worker contributions are to be deducted at the rates indicated [here], for the Health Care Subsidy Fund, Unemployment Insurance Trust Fund, Workforce Development Partnership Fund, and Family Leave Insurance", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/help/employer_handbook/income_security.shtml", + "quote": "Workers' contribution must not exceed those required under the State Plan.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/assets/PDFs/Legal%20Notices/Notices%20of%20Proposal/57%20N.J.R.%201913_a_.pdf", + "quote": "The new taxable wage base would be $ 44,800 for contributions pursuant to the Unemployment Compensation Law and $ 171,100 for contributions to the State Disability Benefits Fund, including the Family Temporary Disability Leave Account.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/lwdhome/press/2025/20251229_newbenefitrates2026.shtml", + "quote": "The taxable wage base for workers covered under the Temporary Disability Insurance and Family Leave Insurance programs increases to $171,100 in 2026 from $165,400 in 2025.", + "fetched_and_read": true + }, + { + "url": "https://business.nj.gov/recent/unemployment-and-disability-insurance-rates-increased-for-2026", + "quote": "The Temporary Disability and Family Leave Insurance taxable wage base increased to $171,100 (from $165,400).", + "fetched_and_read": true + }, + { + "url": "https://www.nfp.com/insights/nj-announces-2026-disability-and-family-leave-contribution-rates/", + "quote": "The NJ Department of Labor and Workforce Development (LWD) announced the 2026 Temporary Disability Insurance (TDI) and Family Leave Insurance (FLI) contribution rates", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None on the amounts. I ran policyengine-us 2.15.17 from the triage venv this session for NJ, age 40, $26,800 employment_income, 2026. It returned TDI 50.920002, FLI 61.640003, and nj_employee_state_payroll_tax and employee_state_payroll_tax 112.560005. These equal 0.0019 x 26,800 and 0.0023 x 26,800. The 26,800 is below the $171,100 base, so the cap does not bind.", + "Timing of the 0.19% and 0.23% rates: I also found no dated official NJDOL source. The pages that state them (myleavebenefits employer, TDI and FLI worker pages, EA rate-info) are undated. The Dec 29, 2025 press release states only the wage base. The 'published before 2026-07-03' finding therefore rests on two things. First, the statutory duty in 43:21-7(d)(1)(G)(i)-(ii) to post estimated rates within 60 days after the fiscal year ends (about Aug 29, 2025). Second, the secondary NFP article dated 2025-11-04. The Wayback availability API lists a rate-info snapshot dated 20260101080705, but WebFetch cannot read web.archive.org, so I could not check its content.", + "The NFP article the research cites to date the rates gives a 'Taxable wage base: $43,300'. That is wrong for worker TDI/FLI, which is $171,100. The rates it reports (0.19%, 0.23%) match NJDOL, but the error makes it weak dating evidence.", + "The $171,100 wage base source 57 N.J.R. 1913(a) (Sept 2, 2025) is a proposal (PRN 2025-113; comments due Nov 1, 2025). I did not verify the adoption notice. NJDOL's Dec 29, 2025 press release and the Jan 9, 2026 business.nj.gov page confirm the figure, and both predate the cutoff.", + "Engine citation nit, which does not affect the amounts: the parameter references use UI_statute.pdf PDF pages #page=60 (TDI rate), #page=61 (FLI rate) and #page=46 (wage base). In the Dec 2024 PDF, (G)(i) starts on PDF p.61, (G)(ii) on PDF p.62 and (b)(4) on PDF p.47. Each is one page off." + ], + "citation_problems": [ + "Every operative-text quote in the research appears verbatim at its URL. I checked them against the full text of the NJDOL UI law compilation (Dec 2024, 246 pp.) and the TDB Law compilation (July 2020; PDF created Aug 16, 2022), both extracted from the PDFs WebFetch saved.", + "Page number: the 43:21-7(e)(1) sentence 'Such contributions shall become due and be paid by the employer ... shall not be deducted' is on printed p.57 (PDF p.66), not p.56 as the research says.", + "Page number: 43:21-7(b)(4) is on printed p.38 (PDF p.47), not '~39'.", + "The TDI worker-page quote starts mid-sentence: the page reads 'For 2026, workers contribute 0.19% ...'. The wording is otherwise verbatim.", + "I could not check the NJ Legislature's official statute database: pub.njleg.gov, pub.njleg.state.nj.us and njleg.state.nj.us all refused connections (ECONNREFUSED), and Justia returned 403. So I could not independently rule out amendments after the compilations. Per the compilations' history lines, 43:21-7 was last amended by P.L.2020, c.150 (Dec 2024 compilation), and 43:21-32 and -33 by P.L.2019, c.37. A search summary describes P.L.2025, c.279 as a reinstatement and anti-retaliation act; I could not read its text.", + "The NJDOL employer handbook page (income_security.shtml) still says a contributory private plan needs a majority written election of employees. That predates P.L.2019, c.37, under which an election is needed only for employees under a collective bargaining agreement (43:21-33). The research's quote from that page is accurate, but the page is stale on this point.", + "I could not read N.J.A.C. 12:16 or 12:18: both NJDOL PDFs (NJAC.pdf, NJAdminCode 021825.pdf) exceed WebFetch's 10 MB limit. The research had the same gap." + ], + "reasoning": "I could not refute the research. The primary sources I read myself support 'mandatory_employee_withholding' for the State-plan TDI and FLI worker contributions the engine counts.\n\nStatute lens: I read all of 43:21-7(d)(1)(A)-(G), (d)(3)-(6), (e)(1)-(2) and (b)(4) in NJDOL's Dec 2024 compilation, plus 43:21-27(a)-(b), -31, -32, -33 and -34 in the TDB Law compilation. The mandatory language runs both ways:\n- Worker liability: (G)(i) says each worker 'shall ... make contributions'; (G)(ii) says each worker 'shall contribute'. (F) says 'contributions' in the chapter include workers' contributions.\n- Employer duty: (E) says each employer 'shall, notwithstanding any provision of law ... to the contrary, withhold in trust' the workers' contributions. It must also show the deduction on payroll records and transmit it.\n- Other sections confirm it. 43:21-14.4 (same compilation) refers to 'any contribution which the employer is required to collect from his employees' for the State disability benefits fund and the FLI account. 43:21-33 calls the State-plan withholding 'that deduction required by Revised Statutes, section 43:21-7(d)(1)'. 43:21-32 says workers excluded from a private plan are 'subject to the employee contribution required by law'.\n\nThe only 'may deduct' language in the chapter is in 43:21-33, and it applies only to an approved private plan. There the employer may collect contributions, capped at the State-plan worker rate by 43:21-32(d).\n\nThe two strongest refutation levers do not hold:\n- Failure to deduct: if the employer misses the current and next payroll, it 'alone shall thereafter be liable'. That is a sanction attached to a duty, not an election. The amount is then 'treated as employer's contributions' only for collection under 43:21-14.\n- The 'right to deduct' wording in NJDOL's 2012 employer packet (revised 2019): agency guidance phrasing cannot override the statute's 'shall withhold'. The same paragraph repeats the failure-to-deduct liability, and NJDOL's current worker pages describe the contribution as a salary deduction 'your employer takes'.\n\nThe employer's own TDI contribution under (e)(1) is separate and 'shall not be deducted' from wages. FLI has no employer contribution: the statute imposes (G)(ii) on workers only, and NJDOL confirms '100% by worker payroll deductions'.\n\nExceptions do not change the default for household-only prompts. These are approved private plans, non-electing local governmental employers (TDI only; FLI still applies under the second 'covered employer' definition in 43:21-27(a)(1)), the 43:21-31 religious waiver, and the collective bargaining election for contributory private plans. The research flagged each as a caveat accurately.\n\nEngine: I read every NJ payroll variable and parameter file and the employee_state_payroll_tax aggregator (TaxUnit; its documentation reads 'Employee-side mandatory state payroll taxes ...'). The NJ files contain no 'maximum permitted employee share' assumption; that wording appears only in the MA, MN and VT files. Running the engine reproduced $50.92 TDI and $61.64 FLI for scenario_008. Both match 0.19% and 0.23% of $26,800.\n\nRemaining weaknesses are documentary, not substantive:\n- There is no dated official source for the 2026 worker rates. The statutory posting deadline and a Nov 2025 secondary article support pre-cutoff publication.\n- The wage-base rule is cited as a proposal, and I did not verify its adoption.\n- The NJ Legislature's site was unreachable, so I could not rule out post-compilation amendments.\n- Two page cites are slightly off." + }, + { + "lens": "administration", + "refuted": false, + "your_classification": "mandatory_employee_withholding", + "evidence": [ + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "(E) Each employer shall, notwithstanding any provision of law in this State to the contrary, withhold in trust the amount of his workers' contributions from their wages at the time such wages are paid", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "If any employer fails to deduct the contributions of any of his workers at the time their wages are paid, or fails to make a deduction therefor at the time wages are paid for the next succeeding payroll period, he alone shall thereafter be liable for such contributions", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf", + "quote": "(ii) Each worker shall contribute to the State disability benefits fund, in addition to any amount contributed pursuant to subparagraph (i) of this paragraph (1)(G)", + "fetched_and_read": true + }, + { + "url": "https://www.law.cornell.edu/regulations/new-jersey/N-J-A-C-12-16-9-1", + "quote": "(a) Every employer shall withhold workers' contributions from their wages at each time of payment of such wages.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/forms_pdfs/tdi/Code.pdf", + "quote": "Employees excluded from a private plan shall be covered under the State plan and the employer shall be liable for the deduction and payment of workers' contributions and employer's contributions, as required by N.J.S.A. 43:21-7.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/employer/index.shtml", + "quote": "Employers must participate in these State public insurance programs and deduct payroll taxes for employees working in NJ, or employers may choose to provide employees coverage through a private insurance plan that meets NJ requirements.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/employer/index.shtml", + "quote": "Employees: 0.19% on the first $171,100 in covered wages (max $325.09). ... Employers: No contribution. Employees: 0.23% on the first $171,100 in covered wages (max $393.53).", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/tdi/", + "quote": "If your employer is covered for Temporary Disability Insurance, they are required to set up payroll contributions for you.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/myleavebenefits/worker/fli/", + "quote": "For 2026, workers contribute 0.23% on the first $171,100 (wage cap) in covered wages earned during this calendar year. The maximum worker contribution for 2026 is $393.53.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/forms_pdfs/ea/Employer%20Informational%20Packet%20.pdf", + "quote": "Employers have the right to deduct worker contributions from wages paid, but the deduction must be made from the current or next succeeding payroll and noted on the payroll records.", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "the employer may during the continuance of the approved private plan collect the required contributions thereto by deduction from the wages paid to covered individuals under such plan", + "fetched_and_read": true + }, + { + "url": "https://nj.gov/labor/myleavebenefits/assets/pdfs/TDB%20Law%20July%202020.pdf", + "quote": "(d) no greater amount is required to be paid by employees toward the cost of benefits than that prescribed by law as the amount of worker contribution to the State disability benefits fund for covered individuals under the State plan", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/employer-services/rate-info/", + "quote": "Worker U.I. D.I. W.F./S.W.F. F.L.I 0.003825 0.0019 0.000425 0.0023 January 1, 2026 to December 31, 2026 ... 2026 Taxable Wage Base (TDI , FLI \u2013 workers only): $171,100", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20251105152047/https://www.nj.gov/labor/ea/employer-services/rate-info/", + "quote": "Worker U.I. D.I. W.F./S.W.F. F.L.I 0.003825 0.0019 0.000425 0.0023 January 1, 2026 to December 31, 2026", + "fetched_and_read": true + }, + { + "url": "https://web.archive.org/web/20250926224746/https://www.nj.gov/labor/ea/employer-services/rate-info/", + "quote": "2026 rates will be available later this year. ... 2026 Taxable Wage Base (TDI , FLI \u2013 workers only): $171,100", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/assets/PDFs/Legal%20Notices/Notices%20of%20Proposal/57%20N.J.R.%201913_a_.pdf", + "quote": "The new taxable wage base would be $ 44,800 for contributions pursuant to the Unemployment Compensation Law and $ 171,100 for contributions to the State Disability Benefits Fund, including the Family Temporary Disability Leave Account.", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/lwdhome/press/2025/20251229_newbenefitrates2026.shtml", + "quote": "The benefit rates and taxable wage base for 2026 reflect the $1,598.66 average weekly wage for 2024", + "fetched_and_read": true + }, + { + "url": "https://www.nj.gov/labor/ea/help/employer_handbook/income_security.shtml", + "quote": "(3) Workers' contribution must not exceed those required under the State Plan.", + "fetched_and_read": true + } + ], + "rate_or_amount_issues": [ + "None material. I recomputed the scenario_008 amounts and they match the engine exactly. TDI: 0.0019 x min(26,800, 171,100) = $50.92, and the engine returned 50.920002. FLI: 0.0023 x 26,800 = $61.64, and the engine returned 61.640003. nj_employee_state_payroll_tax and employee_state_payroll_tax both returned 112.560005 (policyengine-us 2.15.17, run this session). The frozen payroll_tax reference of $2,162.76 equals 7.65% FICA on $26,800 ($2,050.20) plus $112.56.", + "The 2026 maximums check out: 0.0019 x 171,100 = $325.09 and 0.0023 x 171,100 = $393.53. The wage base follows the statutory formula: 107 x $1,598.66 (the 2024 statewide AWW, per the 2025-12-29 NJDOL press release) = $171,056.62, rounded up to $171,100.", + "Better dating than the researcher had: the official NJDOL rate-info page listed 2026 worker DI 0.0019 and FLI 0.0023 in a Wayback capture of 2025-11-05 (also in 2025-11-10 and 2026-01-01). The 2025-09-26 capture still said '2026 rates will be available later this year'. That capture already showed the $171,100 worker wage base. Both rates and the wage base were therefore public on an official page well before 2026-07-03.", + "Minor page-cite inaccuracies only. The researcher cited 43:21-7(b)(4) as 'p.~39'; in the Dec 2024 NJDOL compilation it is on printed p.38 (PDF p.47). The engine's taxable_wage_base parameter anchors UI_statute.pdf#page=46, one page early.", + "The engine's TDI/FLI parameters are 0 from 0000-01-01, so it computes zero for pre-2026 years. This does not affect tax year 2026." + ], + "citation_problems": [ + "Every researcher quote I checked against the downloaded PDFs and pages matches verbatim. Their ellipses do not change the meaning. One cosmetic difference: the TDB compilation prints 'section 43 :21-7(d)(1)' with a space, which the researcher rendered as '43:21-7(d)(1)'.", + "I read N.J.A.C. 12:16-9.1 on Cornell LII, which is not the official NJ OAL publication. I could not fetch an official NJ host of the current 12:16 text: NJDOL's NJAC.pdf compilations are 2003 scans with no text layer.", + "The N.J.A.C. 12:18-2.1(b) text comes from NJDOL's July 2011 compilation (Code.pdf), whose chapter expiration date was 2013-09-18. The rule may have been readopted or renumbered since.", + "The NJ Legislature servers (pub.njleg.state.nj.us and pub.njleg.gov) refused connections. So I could not check 43:21-7 against the official statute database for amendments after the Dec 2024 NJDOL compilation. I also could not read P.L.2025, c.279, signed in Jan 2026, which adds TDI/FLI job protection from 2026-07-17. Current NJDOL 2026 pages still say employers 'must ... deduct', which is consistent with no change.", + "I verified the business.nj.gov quote only through the WebFetch summarizer, not raw page text.", + "The researcher's agency-guidance quote attributed to the employer page adds bracketed labels ([TDI], [FLI]) that are not on the page. The underlying text matches." + ], + "reasoning": "I tried to refute the classification from the administration side and could not.\n\nSTATUTE (official NJDOL compilations, downloaded and text-extracted this session)\n- 43:21-7(d)(1)(G)(i): each worker of a covered employer 'shall ... make contributions' for TDI.\n- 43:21-7(d)(1)(G)(ii): each worker 'shall contribute' for FLI. FLI coverage includes governmental UI employers.\n- 43:21-7(d)(1)(E): each employer 'shall, notwithstanding any provision of law ... to the contrary, withhold in trust' those contributions.\n- 43:21-7(e)(1): the employer's own TDI contribution 'shall not be deducted' from wages.\n\nADMINISTRATIVE RULES\n- N.J.A.C. 12:16-9.1(a) (read via LII): 'Every employer shall withhold workers' contributions from their wages at each time of payment of such wages.' 12:16-9.3 requires the remittance to include contributions 'payable by the employer on behalf of its workers'.\n- N.J.A.C. 12:18-2.1(b)-(c) and 2.21(a) (NJDOL 2011 compilation): the employer 'shall be liable for the deduction' of workers' contributions whenever workers are on the State plan.\n\nAGENCY GUIDANCE FOR 2026\n- The current NJDOL employer page says employers 'must participate ... and deduct payroll taxes'. The only alternative it offers is a private plan.\n- The worker TDI and FLI pages say a covered employer is 'required to set up payroll contributions for you'.\n- The one 'may'-style phrasing is in the 2012 (rev. 2019) Employer Informational Packet: employers 'have the right to deduct'. In context that sentence is about timing: deduct in the current or next payroll or lose the ability. It does not grant an option to absorb the share.\n\nCAN THE EMPLOYER CHOOSE TO PAY THE EMPLOYEE SHARE?\n- Under the State plan, no statute, rule or page grants that election.\n- An employer that fails to deduct by the next payroll becomes 'alone' liable, and the amount is collected as an employer contribution. That is a de facto consequence of non-compliance with a 'shall withhold' duty, not an authorized pass-through.\n\nIS THE EMPLOYEE LIABLE IF THE AMOUNT IS NOT DEDUCTED?\n- Generally no. The statute puts sole liability on the employer.\n- The narrow exception is 43:21-7(d)(4). When wages are paid by a different employing unit, the employer is liable 'in the first instance'. It may then recover from that unit or, absent one, from the individual in a civil action within three months.\n- So in practice an employee whose employer never withheld pays $0. This matches the researcher's caveat and does not change the classification under the task's definition ('the employer must withhold it').\n\nWHERE THE SHARE IS GENUINELY OPTIONAL\nOnly under an approved private plan. Under 43:21-33 the employer 'may' collect contributions, capped at the State-plan rate by 43:21-32(d). It may therefore charge less or nothing. The researcher already identified this.\n\n2026 RATES, WAGE BASE AND DATES\n- Rates 0.19% (TDI) and 0.23% (FLI) and the $171,100 wage base are confirmed on current NJDOL pages and in 57 N.J.R. 1913(a) (2025-09-02).\n- The NJDOL press release of 2025-12-29 gives a 2024 AWW of $1,598.66; 107 x $1,598.66 = $171,056.62, which rounds up to $171,100.\n- Wayback captures show the official rate-info page listing the 2026 worker rates by 2025-11-05. All of this predates 2026-07-03.\n\nENGINE CHECK\n- I read the NJ variables and parameters this session. Each contribution is the rate times min(max(0, employment_income), $171,100). The NJ files carry no 'maximum permitted employee share' assumption.\n- For scenario_008 the engine gives $50.92 TDI and $61.64 FLI, which equals the law's State-plan required amounts exactly.\n\nRESIDUAL GAPS (none undercut the classification)\n- I could not reach the NJ Legislature site to rule out post-2024 amendments to 43:21-7.\n- The 12:16 rule text came from LII, not the official code.\n- Current 2026 agency pages remain consistent with mandatory withholding." + } + ] + }, + { + "key": "OTHER", + "research": { + "programs": [ + { + "program": "Washington Paid Family and Medical Leave (PFML) premium, employee share", + "state": "WA", + "engine_variables": [ + "wa_employee_paid_leave_contribution", + "wa_paid_leave_taxable_wages", + "wa_long_term_care_taxable_wages", + "gov.states.wa.tax.payroll.paid_leave.total_rate", + "gov.states.wa.tax.payroll.paid_leave.employer_share", + "gov.irs.payroll.social_security.cap" + ], + "premium_liability": "ESD assesses a premium on each individual's wages (RCW 50A.10.030(1)). The employer must collect it through payroll deduction and remit it, acting as the employees' agent ((7)(a)-(b)). But the deduction itself is permissive: the employer 'may deduct ... up to' the employee share ((3)(a)-(b)) and 'may elect to pay all or any portion' of it ((3)(c)). Under WAC 192-510-065(2), not deducting counts as electing to pay. So the employer remits the whole premium, and the employee bears the employee share only if the employer deducts it. Employers with fewer than 50 employees owe no employer portion ((5)(a)), but the employee portion must still be remitted.", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "mixed_or_unclear", + "operative_text": [ + { + "citation": "RCW 50A.10.030(3)(a) (version effective until Jan. 1, 2028)", + "url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "quote": "For medical leave premiums, an employer may deduct from the wages of each employee up to the full amount of the premium required.", + "fetched_and_read": true + }, + { + "citation": "RCW 50A.10.030(3)(c)", + "url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "quote": "An employer may elect to pay all or any portion of the employee's share of the premium for family leave or medical leave benefits, or both.", + "fetched_and_read": true + }, + { + "citation": "RCW 50A.10.030(7)(a)", + "url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "quote": "The employer must collect from the employees the premiums provided under this section through payroll deductions and remit the amounts collected to the department.", + "fetched_and_read": true + }, + { + "citation": "WAC 192-510-065(2)", + "url": "https://app.leg.wa.gov/WAC/default.aspx?cite=192-510&full=true", + "quote": "If an employer fails to deduct the maximum allowable employee share of the premium from wages paid for a pay period, the employer is considered to have elected to pay that portion of the employee share", + "fetched_and_read": true + }, + { + "citation": "RCW 50A.10.030(4) (wage cap)", + "url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "quote": "equal to the maximum wages subject to taxation for social security as determined by the social security administration.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://paidleave.wa.gov/employer-roles-responsibilities/", + "quote": "Employers are required to either withhold this amount ... or pay it on their employees\u2019 behalf.", + "fetched_and_read": true + }, + { + "url": "https://esd.wa.gov/about-us/news-release/2025/paid-family-medical-leave-premium-rate-increases-113-2026", + "quote": "Employers will pay 28.57% of the total premium and employees will pay 71.43%.", + "fetched_and_read": true + }, + { + "url": "https://paidleave.wa.gov/updates/", + "quote": "The premium rate will increase to 1.13%.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "yes: RCW 50A.10.030(3)(c) says so expressly, and WAC 192-510-065(2) treats a failure to deduct as an election to pay.", + "rate_2026": { + "total_rate": "1.13% of gross wages, not including tips", + "max_employee_share": "71.43% of the premium, about 0.8072% of wages; the employer pays 28.57% (employers with 50+ employees only)", + "wage_base_or_cap": "$184,500 (2026 Social Security wage base, per RCW 50A.10.030(4) and the ESD page)", + "sources": [ + { + "url": "https://esd.wa.gov/about-us/news-release/2025/paid-family-medical-leave-premium-rate-increases-113-2026", + "published": "2025-10-29" + }, + { + "url": "https://paidleave.wa.gov/employer-roles-responsibilities/", + "published": "undated agency page, read 2026-10-05; it states the 2026 rate and the $184,500 cap" + }, + { + "url": "https://app.leg.wa.gov/rcw/default.aspx?cite=50A.10.030", + "published": "statute; the cap rule is in force (history: 2026 c 26; 2025 c 304)" + } + ], + "published_before_2026_07_03": "yes: the 1.13% rate and the 71.43/28.57 split were announced 2025-10-29" + }, + "engine_amount_checks": [ + { + "scenario_id": "WA_single_age40_wages60000_2026", + "engine_amount": 484.3, + "law_amount_at_max_employee_share": 484.3, + "matches_within_1_dollar": true, + "note": "0.0113 x 0.7143 x 60,000. Wages have no tips and no pre-tax deductions." + }, + { + "scenario_id": "WA_single_age40_wages250000_2026", + "engine_amount": 1489.21, + "law_amount_at_max_employee_share": 1489.21, + "matches_within_1_dollar": true, + "note": "0.0113 x 0.7143 x 184,500 (Social Security cap)." + } + ], + "engine_documentation": "wa_employee_paid_leave_contribution has label 'Washington employee paid leave contribution' and documentation 'Employee-side Washington Paid Family and Medical Leave contribution.' Formula: total_rate x (1 - employer_share) x wa_paid_leave_taxable_wages. Unlike the DE, ME and VT variables, it has no 'assuming the employer withholds the maximum' note. Taxable wages = min(employment_income minus tip_income, Social Security cap). Parameters: total_rate is 0.0092 (2025) and 0.0113 (2026); employer_share is 0.2848 (2025) and 0.2857 (2026). Both cite RCW 50A.10.030, paidleave.wa.gov/updates and the 2025 employer mailer. The aggregate employee_state_payroll_tax is documented as 'Employee-side mandatory state payroll taxes and payroll-funded contributions'.", + "caveats": [ + "The statute pulls both ways. It assesses the premium on the individual and says the employer 'must collect from the employees' ((7)(a)). But the deduction is 'may ... up to', the employer may pay the employee share, and a missed deduction counts as an election to pay. The agency labels the employee premium 'mandatory for all employees' yet says employers may pay it on employees' behalf.", + "For a benchmark that counts only mandatory employee payroll taxes, the employee share is the statutory default but not guaranteed. Whether it is withheld depends on an employer choice the prompt never states.", + "The engine applies the rate by state of residence (defined_for WA). The law follows employment in Washington.", + "Conditional premium waivers (WAC 192-510-080/085) are not modeled." + ], + "confidence": "medium", + "summary": "WA PFML is mixed. The premium is assessed on each worker, and RCW 50A.10.030(7)(a) says the employer must collect it by payroll deduction. But (3) only lets the employer deduct 'up to' the 71.43% employee share and lets it pay that share instead, and WAC 192-510-065(2) treats a missed deduction as an election to pay. The engine's 2026 figure, 1.13% x 71.43% up to $184,500, matches the maximum permitted share exactly." + }, + { + "program": "Washington WA Cares Fund (long-term services and supports) premium", + "state": "WA", + "engine_variables": [ + "wa_employee_long_term_care_contribution", + "wa_long_term_care_taxable_wages", + "gov.states.wa.tax.payroll.long_term_care.employee_rate" + ], + "premium_liability": "Employee-only premium assessed on each individual's wages (RCW 50B.04.080(1)). The employer must collect it through payroll deductions as the employees' agent and remit it ((2)(a)-(b)). WAC 192-910-015(1) says employers must deduct premiums every pay period, and a noncompliant employer must pay past-due premiums ((2)). There is no employer premium.", + "employee_share_rule": "employer_must_withhold", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "RCW 50B.04.080(2)(a)", + "url": "https://app.leg.wa.gov/RCW/default.aspx?cite=50B.04.080", + "quote": "The employer must collect from the employees the premiums provided under this section through payroll deductions and remit the amounts collected to the employment security department.", + "fetched_and_read": true + }, + { + "citation": "RCW 50B.04.080(1)", + "url": "https://app.leg.wa.gov/RCW/default.aspx?cite=50B.04.080", + "quote": "The initial premium rate is .58 percent of the individual's wages. Beginning January 1, 2026, and biennially thereafter, the premium rate shall be set by the pension funding council at a rate no greater than .58 percent.", + "fetched_and_read": true + }, + { + "citation": "WAC 192-910-015(1)", + "url": "https://app.leg.wa.gov/WAC/default.aspx?cite=192-910&full=true", + "quote": "Employers must deduct premiums for each pay period in which the employee receives wages.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://wacaresfund.wa.gov/employers", + "quote": "The premium is 0.58% of an employee\u2019s gross wages", + "fetched_and_read": true + }, + { + "url": "https://wacaresfund.wa.gov/employers", + "quote": "you may, however, elect to pay some or all of your employees' share", + "fetched_and_read": true + }, + { + "url": "https://app.leg.wa.gov/committeeschedules/Home/Agenda/33473", + "quote": "Adoption of Long-Term Services and Supports Trust Program (WA Cares) premium rates.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "The agency page says yes, as a voluntary payment on the employee's behalf. Neither the statute nor WAC 192-910-015 provides an election; both require deduction.", + "rate_2026": { + "total_rate": "0.58% of gross wages, not including tips (statutory ceiling; the Pension Funding Council sets the rate from 2026 at no more than 0.58%)", + "max_employee_share": "100% (employee-only premium): 0.58%", + "wage_base_or_cap": "No cap ('not capped at the taxable maximum for social security', wacaresfund.wa.gov/employers)", + "sources": [ + { + "url": "https://app.leg.wa.gov/RCW/default.aspx?cite=50B.04.080", + "published": "statute as amended by 2025 c 380 s 10 (enacted 2025)" + }, + { + "url": "https://wacaresfund.wa.gov/employers", + "published": "undated agency page, read 2026-10-05" + }, + { + "url": "https://app.leg.wa.gov/committeeschedules/Home/Agenda/33473", + "published": "2025-10-08 Pension Funding Council agenda listing rate adoption (the adopted outcome was not read)" + } + ], + "published_before_2026_07_03": "yes: the 0.58% ceiling is in the statute, and the agency employer page states 0.58%. The Pension Funding Council's Oct. 8, 2025 adoption record itself was not read. The toolkit FAQ restating 0.58% is dated Aug. 18, 2026, after the cutoff, and is not relied on." + }, + "engine_amount_checks": [ + { + "scenario_id": "WA_single_age40_wages60000_2026", + "engine_amount": 348, + "law_amount_at_max_employee_share": 348, + "matches_within_1_dollar": true, + "note": "0.0058 x 60,000, no cap." + }, + { + "scenario_id": "WA_single_age40_wages250000_2026", + "engine_amount": 1450, + "law_amount_at_max_employee_share": 1450, + "matches_within_1_dollar": true, + "note": "0.0058 x 250,000, no cap." + } + ], + "engine_documentation": "wa_employee_long_term_care_contribution has label 'Washington employee long-term care contribution' and documentation 'Employee-side Washington WA Cares Fund contribution.' Formula: employee_rate x wa_long_term_care_taxable_wages. Taxable wages = max(0, employment_income - tip_income), documented as 'excluding tips and including federal pre-tax payroll deductions'. Parameter employee_rate is 0.0058 from 2023-07-01, citing RCW 50B.04.080 and wacaresfund.wa.gov/employers.", + "caveats": [ + "Approved individual exemptions are not modeled (WAC 192-910-015(3)(b)), nor is the nonimmigrant-visa carve-out ((3)(c)).", + "The agency's 'may elect to pay' statement is a voluntary employer payment on the employee's behalf. It does not change the statutory duty to collect from employees.", + "The engine applies the premium by state of residence. The law follows employment in Washington." + ], + "confidence": "high", + "summary": "WA Cares is a mandatory employee premium. RCW 50B.04.080(2)(a) requires employers to collect it from employees by payroll deduction, WAC 192-910-015(1) says employers must deduct it, and there is no employer share. The engine's 0.58% of uncapped non-tip wages matches the statute and the agency rate." + }, + { + "program": "Oregon Paid Leave Oregon (family and medical leave insurance), employee contribution", + "state": "OR", + "engine_variables": [ + "or_employee_paid_leave_contribution", + "or_paid_leave_taxable_wages", + "gov.states.or.tax.payroll.paid_leave.employee_rate", + "gov.irs.payroll.social_security.cap" + ], + "premium_liability": "ORS 657B.150(1)(a)-(b) requires employers and employees to contribute. The employee's 60% share must be deducted from wages by the employer ((2)(b)), which holds the money in trust and remits it ((11)-(12)). Employers with fewer than 25 employees pay no employer share ((4)(a)) but must still collect the employee contributions.", + "employee_share_rule": "employer_must_withhold", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "ORS 657B.150(2)(b)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html", + "quote": "An employer shall deduct employee contributions from the wages of each employee in an amount that is equal to 60 percent of the total rate determined by the director.", + "fetched_and_read": true + }, + { + "citation": "ORS 657B.150(1)(a)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html", + "quote": "all employers and employees shall contribute to the Paid Family and Medical Leave Insurance Fund established under ORS 657B.430.", + "fetched_and_read": true + }, + { + "citation": "ORS 657B.150(5)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html", + "quote": "Notwithstanding subsection (1) of this section, an employer may elect to pay the required employee contributions, in whole or in part, as an employer-offered benefit.", + "fetched_and_read": true + }, + { + "citation": "ORS 657B.150(1)(c)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors657B.html", + "quote": "The total rate may not exceed one percent of employee wages, up to a maximum amount in wages that is equivalent to the Social Security contribution and benefit base limit", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://paidleave.oregon.gov/employers/what-employers-need-to-do.html", + "quote": "You must withhold contributions from your employees\u2019 wages.", + "fetched_and_read": true + }, + { + "url": "https://www.oregon.gov/employ/NewsAndMedia/Documents/2025-11-18_Tax_Rate_2026_Press_Release.pdf", + "quote": "The Paid Leave Oregon contribution rate for 2026 will remain at 1%.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "yes, as an employer-offered benefit (ORS 657B.150(5)). The default rule is a mandatory deduction ((2)(b)).", + "rate_2026": { + "total_rate": "1% of wages", + "max_employee_share": "60% of the total: 0.6% of wages (statutorily fixed, not a ceiling)", + "wage_base_or_cap": "$184,500 (2026 Social Security base, per the OED release)", + "sources": [ + { + "url": "https://www.oregon.gov/employ/NewsAndMedia/Documents/2025-11-18_Tax_Rate_2026_Press_Release.pdf", + "published": "2025-11-18" + }, + { + "url": "https://paidleave.oregon.gov/employers/what-employers-need-to-do.html", + "published": "undated agency page, read 2026-10-05" + } + ], + "published_before_2026_07_03": "yes (2025-11-18)" + }, + "engine_amount_checks": [ + { + "scenario_id": "OR_single_age40_wages60000_2026", + "engine_amount": 360, + "law_amount_at_max_employee_share": 360, + "matches_within_1_dollar": true, + "note": "0.006 x 60,000." + }, + { + "scenario_id": "OR_single_age40_wages250000_2026", + "engine_amount": 1107, + "law_amount_at_max_employee_share": 1107, + "matches_within_1_dollar": true, + "note": "0.006 x 184,500 cap." + } + ], + "engine_documentation": "or_employee_paid_leave_contribution has label 'Oregon employee paid leave contribution' and documentation 'Employee-side Oregon Paid Leave contribution.' Formula: employee_rate x or_paid_leave_taxable_wages. Taxable wages = min(max(0, employment_income), Social Security cap), documented as 'including federal pre-tax payroll deductions'. Parameter employee_rate is 0.006 from 2023-01-01, citing ORS 657B.150 and the Paid Leave Oregon contributions calculator. The employer rate is 0.004, with a 25-employee threshold.", + "caveats": [ + "Employers with an approved equivalent plan (ORS 657B.210(4)) may follow different contribution arrangements. The engine does not model them.", + "Employers may voluntarily pay the employee share ((5)). The engine assumes the statutory default.", + "The engine applies the contribution by state of residence. The law follows where wages are earned (ORS 657B.175)." + ], + "confidence": "high", + "summary": "Paid Leave Oregon is a mandatory employee contribution. ORS 657B.150(2)(b) says the employer 'shall deduct' 60% of the total rate from wages. The employer may pay it as a benefit, but that is optional. The engine's 0.6% up to $184,500 matches the 2026 rate the OED announced on Nov. 18, 2025." + }, + { + "program": "Oregon statewide transit tax", + "state": "OR", + "engine_variables": [ + "or_employee_statewide_transit_tax", + "or_statewide_transit_taxable_wages", + "gov.states.or.tax.payroll.statewide_transit.employee_rate" + ], + "premium_liability": "Under ORS 320.550(2) the tax is imposed on the employee's wages. The employer must deduct and withhold it ((3)) and report and pay it ((4)), and is treated as a taxpayer ((5)). An employer that fails to withhold is deemed responsible for the tax and may be penalized ((9)). A resident working for an out-of-state employer reports and pays it themselves ((10)).", + "employee_share_rule": "employer_must_withhold", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "ORS 320.550(2)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors320.html", + "quote": "A tax is imposed at the rate of one-tenth of one percent of the wages of an employee who is:", + "fetched_and_read": true + }, + { + "citation": "ORS 320.550(3)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors320.html", + "quote": "Every employer at the time of the payment of wages shall deduct and withhold from the total amount of the wages paid", + "fetched_and_read": true + }, + { + "citation": "ORS 320.550(9)(a)", + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors320.html", + "quote": "An employer that fails to deduct and withhold the tax required under this section: (a) Is deemed responsible for the payment of the tax obligation", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.oregon.gov/dor/programs/businesses/pages/statewide-transit-tax.aspx", + "quote": "Please continue to withhold at the rate of one-tenth of 1 percent or .001.", + "fetched_and_read": true + }, + { + "url": "https://www.oregon.gov/dor/programs/businesses/pages/statewide-transit-tax.aspx", + "quote": "Measure 120 did not pass in the May 19, 2026 primary election.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "no: neither the statute nor the agency page provides for it. The tax is imposed on the employee's wages and withholding is required.", + "rate_2026": { + "total_rate": "0.1% of wages (ORS 316.162 definition)", + "max_employee_share": "100% (tax on the employee): 0.1%", + "wage_base_or_cap": "No cap", + "sources": [ + { + "url": "https://www.oregonlegislature.gov/bills_laws/ors/ors320.html", + "published": "ORS 320.550 (2017 c.750; 2018 c.93), 2025 edition. An editor's note says Enrolled HB 3991 (2025 special session) amends 320.550 subject to referendum." + }, + { + "url": "https://www.oregon.gov/dor/programs/businesses/pages/statewide-transit-tax.aspx", + "published": "undated DOR page, updated after the May 19, 2026 election; read 2026-10-05" + } + ], + "published_before_2026_07_03": "yes: the 0.1% rate has been in ORS 320.550 since 2018. The HB 3991 change was referred to voters as Measure 120, which failed on 2026-05-19, before the cutoff. DOR tells employers to continue at 0.1%." + }, + "engine_amount_checks": [ + { + "scenario_id": "OR_single_age40_wages60000_2026", + "engine_amount": 60, + "law_amount_at_max_employee_share": 60, + "matches_within_1_dollar": true, + "note": "0.001 x 60,000. No pre-tax deductions in the scenario." + }, + { + "scenario_id": "OR_single_age40_wages250000_2026", + "engine_amount": 250, + "law_amount_at_max_employee_share": 250, + "matches_within_1_dollar": true, + "note": "0.001 x 250,000, no cap." + } + ], + "engine_documentation": "or_employee_statewide_transit_tax has label 'Oregon employee statewide transit tax' and documentation 'Employee-side Oregon statewide transit tax withholding.' Formula: employee_rate x or_statewide_transit_taxable_wages. Taxable wages = irs_employment_income, documented as 'following the income tax withholding wage base'. irs_employment_income = max(0, employment_income - pre_tax_contributions). Parameter employee_rate is 0.001 from 2018-07-01, citing ORS 320.550 and the DOR Statewide Transit Tax page.", + "caveats": [ + "I did not read HB 3991's text or its proposed rate. The 0.1% rate for 2026 rests on the DOR page (Measure 120 failed) and the codified ORS 320.550.", + "ORS 316.162 excludes some kinds of pay from 'wages'. The engine uses federal wages net of pre-tax contributions.", + "The tax also applies to nonresidents for work in Oregon. The engine models Oregon residents only." + ], + "confidence": "high", + "summary": "The Oregon statewide transit tax is a mandatory employee tax. ORS 320.550 imposes 0.1% on the employee's wages and says every employer 'shall deduct and withhold' it. The engine's 0.1% uncapped matches the 2026 law, since the HB 3991 increase was referred as Measure 120 and failed on May 19, 2026." + }, + { + "program": "Rhode Island Temporary Disability Insurance (TDI/TCI) employee contribution", + "state": "RI", + "engine_variables": [ + "ri_employee_temporary_disability_insurance_contribution", + "ri_temporary_disability_insurance_taxable_wages", + "gov.states.ri.tax.payroll.temporary_disability_insurance.employee_rate", + "gov.states.ri.tax.payroll.temporary_disability_insurance.taxable_wage_base" + ], + "premium_liability": "Under R.I. Gen. Laws \u00a7 28-40-1(b), each employee owes the contribution (fund cost rate x wages up to the taxable wage base). Each employer must withhold it in trust and transmit it (\u00a7 28-40-3). If the employer fails to deduct it, the employer alone becomes liable and it is treated as an employer contribution (\u00a7 28-40-4). There is no employer TDI contribution.", + "employee_share_rule": "employee_must_contribute", + "classification": "mandatory_employee_withholding", + "operative_text": [ + { + "citation": "R.I. Gen. Laws \u00a7 28-40-1(b) (version effective Jan. 1, 2026)", + "url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-1_28-40-1.htm", + "quote": "Each employee shall contribute ... an amount equal to the fund cost rate times the wages paid by the employer to the employee up to the taxable wage base", + "fetched_and_read": true + }, + { + "citation": "R.I. Gen. Laws \u00a7 28-40-1(a) (effective Jan. 1, 2026; P.L. 2025, ch. 235 and ch. 237)", + "url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-1_28-40-1.htm", + "quote": "The taxable wage base under this chapter for each calendar year shall be equal to the greater of one hundred thousand dollars ($100,000) or the annual earnings needed by an individual to qualify for the maximum weekly benefit amount", + "fetched_and_read": true + }, + { + "citation": "R.I. Gen. Laws \u00a7 28-40-3", + "url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-3.htm", + "quote": "Each employer shall withhold in trust contributions from the wages of his or her employees at the time those wages are earned or paid", + "fetched_and_read": true + }, + { + "citation": "R.I. Gen. Laws \u00a7 28-40-4", + "url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-4.htm", + "quote": "If any employer fails to deduct the contributions of any of the employer\u2019s employees at the time their wages are paid ... the employer alone shall subsequently be liable for those contributions", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://dlt.ri.gov/sites/g/files/xkgbur571/files/2026-06/7.1.26%20UI%20%26%20TDI%20Quick%20Reference.pdf", + "quote": "1.1% of first $100,000 earned", + "fetched_and_read": true + }, + { + "url": "https://dlt.ri.gov/sites/g/files/xkgbur571/files/2026-06/7.1.26%20UI%20%26%20TDI%20Quick%20Reference.pdf", + "quote": "None (Employee Tax)", + "fetched_and_read": true + }, + { + "url": "https://dlt.ri.gov/press-releases/2026-tax-rates-unemployment-insurance-and-temporary-disability-insurance", + "quote": "NOT READ: HTTP 403 (Cloudflare) on both WebFetch and curl; the engine cites this release.", + "fetched_and_read": false + } + ], + "can_employer_elect_to_pay_employee_share": "No election exists in the statute. \u00a7 28-40-4 makes the employer alone liable if it fails to deduct, framed as a failure rather than a permitted choice.", + "rate_2026": { + "total_rate": "1.1% (employee-only)", + "max_employee_share": "100%: 1.1% of wages", + "wage_base_or_cap": "$100,000 (statutory floor effective 2026-01-01)", + "sources": [ + { + "url": "https://dlt.ri.gov/sites/g/files/xkgbur571/files/2026-06/7.1.26%20UI%20%26%20TDI%20Quick%20Reference.pdf", + "published": "2026-06-15 (PDF creation date; document headed 'Effective July 1, 2026')" + }, + { + "url": "https://webserver.rilegislature.gov/Statutes/TITLE28/28-40/28-40-1_28-40-1.htm", + "published": "P.L. 2025, ch. 235 and ch. 237 (enacted 2025, effective 2026-01-01) set the $100,000 wage-base floor" + } + ], + "published_before_2026_07_03": "yes: the $100,000 wage base was enacted in 2025, and the DLT quick reference with 1.1% was created 2026-06-15. The DLT press release the engine cites (late 2025) could not be read (403)." + }, + "engine_amount_checks": [ + { + "scenario_id": "RI_single_age40_wages60000_2026", + "engine_amount": 660, + "law_amount_at_max_employee_share": 660, + "matches_within_1_dollar": true, + "note": "0.011 x 60,000." + }, + { + "scenario_id": "RI_single_age40_wages250000_2026", + "engine_amount": 1100, + "law_amount_at_max_employee_share": 1100, + "matches_within_1_dollar": true, + "note": "0.011 x 100,000 wage base." + } + ], + "engine_documentation": "ri_employee_temporary_disability_insurance_contribution has label 'Rhode Island employee temporary disability insurance contribution' and documentation 'Employee-side Rhode Island TDI contribution, which also funds TCI.' Formula: employee_rate x ri_temporary_disability_insurance_taxable_wages. Taxable wages = min(irs_employment_income, taxable_wage_base), documented as 'following the income tax wage base'. Parameters: employee_rate is 0.011 from 2026-01-01 and 0 before; taxable_wage_base is 100,000 from 2026-01-01 and 0 before. Both cite R.I. Gen. Laws \u00a7 28-40-1 and the DLT 2026 rates press release.", + "caveats": [ + "Statutory exemptions are not modeled: spiritual-healing dependents (\u00a7 28-40-2), minors aged 14-15 (\u00a7 28-40-2.1), and certain persons with disabilities (\u00a7 28-40-2.2).", + "The engine's parameters are 0 before 2026. That is outside the 2026 benchmark year and not assessed here.", + "The engine uses wages net of pre-tax contributions (irs_employment_income). I did not verify the RI TDI wage definition." + ], + "confidence": "high", + "summary": "Rhode Island TDI is a mandatory employee contribution. \u00a7 28-40-1(b) says each employee 'shall contribute' and \u00a7 28-40-3 says each employer 'shall withhold'. The engine's 1.1% of the first $100,000 matches the 2026 statutory wage base and the DLT quick reference (June 15, 2026)." + }, + { + "program": "Delaware Paid Leave (family and medical leave insurance) contributions, employee share", + "state": "DE", + "engine_variables": [ + "de_employee_paid_leave_contribution", + "de_paid_leave_contribution_rate", + "taxable_earnings_for_social_security", + "employer_headcount", + "gov.states.de.tax.payroll.paid_leave.employee_share", + "gov.states.de.tax.payroll.paid_leave.medical_rate", + "gov.states.de.tax.payroll.paid_leave.family_caregiver_rate", + "gov.states.de.tax.payroll.paid_leave.parental_rate", + "gov.states.de.tax.payroll.paid_leave.small_employer_threshold", + "gov.states.de.tax.payroll.paid_leave.full_coverage_threshold" + ], + "premium_liability": "The employer is liable. Under 19 Del. C. \u00a7 3705(b) the employer remits contributions for each employee, and under \u00a7 3705(g) it is liable for the contribution when wages are paid and must remit the total. If it fails to deduct, it remains liable for the full amount. The employee share is only what the employer chooses to deduct, capped at 50% of each component (\u00a7 3705(d)-(f)).", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "19 Del. C. \u00a7 3705(d) (same wording in (e) and (f))", + "url": "https://delcode.delaware.gov/title19/c037/index.html", + "quote": "From the wages of an employee for whom a contribution is required under paragraph (b)(1) of this section, an employer shall deduct not greater than 50% of the contribution required for the employee.", + "fetched_and_read": true + }, + { + "citation": "19 Del. C. \u00a7 3705(d)", + "url": "https://delcode.delaware.gov/title19/c037/index.html", + "quote": "However, an employer may elect to pay all or any portion of the employee\u2019s share of the contribution listed in paragraph (b)(1) of this section.", + "fetched_and_read": true + }, + { + "citation": "19 Del. C. \u00a7 3705(g)", + "url": "https://delcode.delaware.gov/title19/c037/index.html", + "quote": "The employer is liable for the contribution at the time that wages are paid and shall remit the total contribution required under subsection (b) of this section to the Fund.", + "fetched_and_read": true + }, + { + "citation": "19 Del. C. \u00a7 3705(b)(1)a. (also (b)(2)a. 0.08% and (b)(3)a. 0.32%)", + "url": "https://delcode.delaware.gov/title19/c037/index.html", + "quote": "For 2025 and 2026, the contribution rate for medical leave benefits as a percentage of wages is 0.4%.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://labor.delaware.gov/delaware-paid-leave/", + "quote": "Employers can require employees to contribute up to half the cost.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "yes: \u00a7 3705(d)-(f) let the employer pay all or any portion, and \u00a7 3705(g) keeps the employer liable for any amount not deducted.", + "rate_2026": { + "total_rate": "0.8% of wages for employers with 25+ employees (medical 0.40% + family caregiving 0.08% + parental 0.32%). Employers with 10-24 employees owe the parental component only (0.32%). Under 10 employees: not covered.", + "max_employee_share": "50% of each component: 0.40% of wages (25+ employees) or 0.16% (10-24 employees)", + "wage_base_or_cap": "Wages are FICA OASDI remuneration (\u00a7 3701(24)), so effectively the $184,500 Social Security base", + "sources": [ + { + "url": "https://delcode.delaware.gov/title19/c037/index.html", + "published": "Statute: rates fixed 'For 2025 and 2026' in \u00a7 3705(b) (83 Del. Laws c. 301, 2022, as amended)" + }, + { + "url": "https://labor.delaware.gov/delaware-paid-leave/", + "published": "undated agency page, read 2026-10-05" + } + ], + "published_before_2026_07_03": "yes: the 2026 rates are fixed in the statute" + }, + "engine_amount_checks": [ + { + "scenario_id": "DE_single_age40_wages60000_2026", + "engine_amount": 240, + "law_amount_at_max_employee_share": 240, + "matches_within_1_dollar": true, + "note": "0.008 x 0.5 x 60,000. The engine's employer_headcount defaults to 100, which selects full coverage. The law amount at the minimum employee share is $0." + }, + { + "scenario_id": "DE_single_age40_wages250000_2026", + "engine_amount": 738, + "law_amount_at_max_employee_share": 738, + "matches_within_1_dollar": true, + "note": "0.008 x 0.5 x 184,500. The law amount at the minimum employee share is $0." + } + ], + "engine_documentation": "de_employee_paid_leave_contribution has label 'Delaware employee paid leave contribution' and documentation 'Employee-side Delaware paid leave contribution, assuming the employer withholds the maximum permitted employee share.' Formula: de_paid_leave_contribution_rate x employee_share x taxable_earnings_for_social_security. de_paid_leave_contribution_rate selects 0 if headcount < 10, the parental rate if < 25, else the sum of the three rates. Default headcount is 100. Parameters: employee_share is 0.5 from 2025, described as 'Default employee share of Delaware paid leave contributions'. Rates from 2025 are medical 0.004, family caregiver 0.0008 and parental 0.0032, all citing 19 Del. C. \u00a7\u00a7 3701/3705. de_employer_paid_leave_contribution is documented as 'assuming the employer contributes only the minimum required share.'", + "caveats": [ + "The prompt never gives employer size, but the law's rate depends on it (0%, 0.32% or 0.8% total). The engine silently assumes 100 employees.", + "Employers with an approved private plan (\u00a7 3705(i), \u00a7 3716) need not remit, and the engine does not model this.", + "The engine counts this permissive deduction in employee_state_payroll_tax, which is documented as 'mandatory'. That mismatch is the same pattern as Minnesota." + ], + "confidence": "high", + "summary": "Delaware Paid Leave is an optional employer pass-through. Under 19 Del. C. \u00a7 3705(g) the employer is liable for and remits the whole contribution. It may deduct no more than 50% from wages and may pay the employee's share instead (\u00a7 3705(d)-(f)). The engine counts the 50% maximum (0.4% of OASDI wages, assuming 25+ employees), which equals the permitted maximum, but the legally required employee amount is $0." + }, + { + "program": "Maine Paid Family and Medical Leave premium, employee share", + "state": "ME", + "engine_variables": [ + "me_employee_paid_leave_contribution", + "me_paid_leave_taxable_wages", + "gov.states.me.tax.payroll.paid_leave.employee_rate", + "gov.states.me.tax.payroll.paid_leave.employer_rate", + "gov.states.me.tax.payroll.paid_leave.employer_headcount_threshold", + "gov.irs.payroll.social_security.cap" + ], + "premium_liability": "The employer remits premiums for each employee (26 M.R.S. \u00a7 850-F(2)): 100% of the 1% premium with 15+ employees, 50% with fewer (\u00a7 850-F(5)). Either way it 'may deduct up to 50%' of the premium from wages. Rule 12-702 ch. 1 \u00a7 X(L) treats a failure to deduct as an election to pay. A non-paying employer faces assessments (\u00a7 850-F(9)).", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "26 M.R.S. \u00a7 850-F(5)(A)", + "url": "https://legislature.maine.gov/legis/statutes/26/title26sec850-F.html", + "quote": "An employer with 15 or more employees may deduct up to 50% of the premium required for an employee by subsection 3 from that employee's wages and shall remit 100% of the combined premium contribution", + "fetched_and_read": true + }, + { + "citation": "26 M.R.S. \u00a7 850-F(3)(A)", + "url": "https://legislature.maine.gov/legis/statutes/26/title26sec850-F.html", + "quote": "Beginning January 1, 2025, the premium amount may not be more than a combined rate of 1.0% of wages.", + "fetched_and_read": true + }, + { + "citation": "26 M.R.S. \u00a7 850-F(6)", + "url": "https://legislature.maine.gov/legis/statutes/26/title26sec850-F.html", + "quote": "Premiums are required up to the contribution and benefit base limit established annually by the federal Social Security Administration", + "fetched_and_read": true + }, + { + "citation": "12-702 C.M.R. ch. 1, \u00a7 X(L) (Maine Department of Labor PFML rule, effective Jan. 1, 2025)", + "url": "https://www1.maine.gov/sos/sites/maine.gov.sos/files/content/assets/702c001.pdf", + "quote": "If an employer fails to deduct the required employee share of the premium from wages paid during a pay period, the employer is considered to have elected to pay that portion of the employee share.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://www.maine.gov/paidleave/docs/2024/EmployersGuidetoMainePFMLContributions.pdf", + "quote": "may deduct up to fifty percent (50%) of the premium (.5% of wages)", + "fetched_and_read": true + }, + { + "url": "https://www.maine.gov/paidleave/docs/2026/employers/faq/employerFAQenglish.pdf", + "quote": "No more than 0.5 percent can come from the employee.", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "yes: the deduction is 'may deduct up to 50%' (\u00a7 850-F(5)), and the rule deems a non-deduction an election to pay (\u00a7 X(L)). The rule also requires one deduction choice for all employees (\u00a7 X(I)).", + "rate_2026": { + "total_rate": "1.0% of wages for employers with 15+ employees; 0.5% remitted by employers with fewer than 15", + "max_employee_share": "0.5% of wages (50% of the 1% premium), at any employer size", + "wage_base_or_cap": "Social Security contribution and benefit base: $184,500 in 2026", + "sources": [ + { + "url": "https://legislature.maine.gov/legis/statutes/26/title26sec850-F.html", + "published": "PL 2023, c. 412 (1.0% ceiling and 50% deduction rule), amended by PL 2025, c. 277" + }, + { + "url": "https://www.maine.gov/paidleave/docs/2024/EmployersGuidetoMainePFMLContributions.pdf", + "published": "2025-01-13 (PDF creation date)" + }, + { + "url": "https://www1.maine.gov/sos/sites/maine.gov.sos/files/content/assets/702c001.pdf", + "published": "rule effective 2025-01-01 (PDF dated 2024-12-02)" + } + ], + "published_before_2026_07_03": "yes: the statute and the Jan. 2025 employer guide state 1% of wages, with the employee share up to 0.5%. The department sets the rate itself only from 2028 under \u00a7 850-F(3)(B). The 2026 FAQ restating '2025-2027' rates was created 2026-09-04 and is not relied on for the cutoff." + }, + "engine_amount_checks": [ + { + "scenario_id": "ME_single_age40_wages60000_2026", + "engine_amount": 300, + "law_amount_at_max_employee_share": 300, + "matches_within_1_dollar": true, + "note": "0.005 x 60,000. The law amount at the minimum employee share is $0." + }, + { + "scenario_id": "ME_single_age40_wages250000_2026", + "engine_amount": 922.5, + "law_amount_at_max_employee_share": 922.5, + "matches_within_1_dollar": true, + "note": "0.005 x 184,500 cap. The law amount at the minimum employee share is $0." + } + ], + "engine_documentation": "me_employee_paid_leave_contribution has label 'Maine employee paid leave contribution' and documentation 'Employee-side Maine paid leave contribution, assuming the employer withholds the maximum permitted employee share.' Formula: employee_rate x me_paid_leave_taxable_wages. Taxable wages = min(max(0, employment_income), Social Security cap). Parameter employee_rate is 0.005 from 2025, citing 26 M.R.S. \u00a7 850-F and the Employers Guide (page 11). The employer rate is 0.005, with a 15-employee threshold.", + "caveats": [ + "Employers with an approved private plan (\u00a7 850-F(8)) do not remit. The engine does not model this.", + "Small employers (<15) remit only 0.5% and may deduct all of it from wages. The employee maximum is still 0.5%, so the engine's figure is the cap at any size.", + "The engine counts this permissive deduction in employee_state_payroll_tax, which is documented as 'mandatory'.", + "The Jan. 2025 guide says the department sets the rate 'Beginning in 2027'. The current statute \u00a7 850-F(3)(B) says 2028. Neither affects 2026." + ], + "confidence": "high", + "summary": "Maine PFML is an optional employer pass-through. The employer remits the premium, and 26 M.R.S. \u00a7 850-F(5) only lets it 'deduct up to 50%' from wages. Under the rule, a missed deduction is an election to pay. The engine's 0.5% up to the $184,500 Social Security base equals the maximum permitted employee share, not a mandatory amount." + }, + { + "program": "Vermont Child Care Contribution, employee share", + "state": "VT", + "engine_variables": [ + "vt_employee_child_care_contribution", + "vt_child_care_contribution_wages", + "gov.states.vt.tax.payroll.child_care.employee_rate", + "gov.states.vt.tax.payroll.child_care.employer_rate" + ], + "premium_liability": "The employer is liable. Under 32 V.S.A. \u00a7 10553(a)(1) each employer pays the 0.44% contribution on covered wages and remits it. Under \u00a7 10553(c)(2) employers are responsible for the full amount of any unpaid contribution. The employer may withhold up to one quarter (0.11% of wages) from the employee.", + "employee_share_rule": "employer_may_deduct_up_to", + "classification": "optional_employer_pass_through", + "operative_text": [ + { + "citation": "32 V.S.A. \u00a7 10553(a)(1)", + "url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "quote": "Each employer shall pay the Child Care Contribution on all covered wages paid to each of the employer\u2019s employees", + "fetched_and_read": true + }, + { + "citation": "32 V.S.A. \u00a7 10553(a)(1)", + "url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "quote": "An employer may deduct and withhold from an employee\u2019s covered wages an amount equal to not more than one quarter of the contribution required pursuant to subsection (b) of this section.", + "fetched_and_read": true + }, + { + "citation": "32 V.S.A. \u00a7 10553(b)", + "url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "quote": "The contribution rate shall be 0.44 percent of each employee\u2019s covered wages", + "fetched_and_read": true + }, + { + "citation": "32 V.S.A. \u00a7 10553(c)(2)", + "url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "quote": "Employers shall be responsible for the full amount of any unpaid contributions due pursuant to subdivision (a)(1) of this section.", + "fetched_and_read": true + } + ], + "agency_guidance": [ + { + "url": "https://tax.vermont.gov/sites/tax/files/documents/GB-1326.pdf", + "quote": "Employees are not required to file or pay the CCC", + "fetched_and_read": true + }, + { + "url": "https://tax.vermont.gov/sites/tax/files/documents/GB-1326.pdf", + "quote": "choose not to withhold any amount from employees", + "fetched_and_read": true + } + ], + "can_employer_elect_to_pay_employee_share": "yes: the employer is the payer, and withholding up to 0.11% is optional (\u00a7 10553(a)(1); GB-1326 'Employer Election to Withhold from Employees').", + "rate_2026": { + "total_rate": "0.44% of covered wages (IRC \u00a7 3401 wages subject to Vermont withholding)", + "max_employee_share": "one quarter: 0.11% of wages", + "wage_base_or_cap": "No cap", + "sources": [ + { + "url": "https://legislature.vermont.gov/statutes/fullchapter/32/246", + "published": "2023 Act 76, \u00a7 24, effective 2024-07-01" + }, + { + "url": "https://tax.vermont.gov/sites/tax/files/documents/GB-1326.pdf", + "published": "2024-10-02 (PDF creation date)" + } + ], + "published_before_2026_07_03": "yes (statutory rate since 2024-07-01)" + }, + "engine_amount_checks": [ + { + "scenario_id": "VT_single_age40_wages60000_2026", + "engine_amount": 66, + "law_amount_at_max_employee_share": 66, + "matches_within_1_dollar": true, + "note": "0.0011 x 60,000. The law amount at the minimum employee share is $0." + }, + { + "scenario_id": "VT_single_age40_wages250000_2026", + "engine_amount": 275, + "law_amount_at_max_employee_share": 275, + "matches_within_1_dollar": true, + "note": "0.0011 x 250,000, no cap. The law amount at the minimum employee share is $0." + } + ], + "engine_documentation": "vt_employee_child_care_contribution has label 'Vermont employee child care contribution' and documentation 'Employee-side Vermont Child Care Contribution, assuming the employer withholds the maximum permitted employee share.' Formula: employee_rate x vt_child_care_contribution_wages. Wages = irs_employment_income, documented as 'following the income tax withholding wage base'. Parameter employee_rate is 0.0011 and employer_rate is 0.0033, both from 2024-07-01, citing 32 V.S.A. \u00a7 10553 and GB-1326 (page 3). vt_employer_child_care_contribution is documented as 'assuming the employer contributes only the minimum required employer share.'", + "caveats": [ + "The statute names the employer as payer and treats withholding as optional, and the Tax Department says employees are not required to pay. Counting 0.11% as a mandatory employee tax therefore contradicts the law.", + "Only wages subject to Vermont income tax withholding are covered (work physically performed in Vermont). The engine keys on Vermont residence.", + "The engine counts this permissive deduction in employee_state_payroll_tax, which is documented as 'mandatory'." + ], + "confidence": "high", + "summary": "The Vermont Child Care Contribution is an optional employer pass-through. Under 32 V.S.A. \u00a7 10553 the employer pays the 0.44% and is responsible for any unpaid amount, and may withhold up to a quarter (0.11%) from the employee. GB-1326 says employees are not required to pay. The engine's 0.11% of uncapped wages equals the maximum permitted withholding, but the legally required employee amount is $0." + } + ] + }, + "verdicts": [] + } + ] +} \ No newline at end of file diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact.json b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact.json new file mode 100644 index 00000000..7806e4e6 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact.json @@ -0,0 +1,1054 @@ +{ + "published_reproduced": true, + "salt_commit": "8af912a062dd7ae1373de4c043ae2727d3406930", + "salt": { + "changed_outputs": [ + "scenario_022/federal_income_tax_before_refundable_credits", + "scenario_081/federal_income_tax_before_refundable_credits", + "scenario_114/federal_income_tax_before_refundable_credits" + ], + "scored_outputs": { + "published": 1928, + "proposed": 1925 + }, + "exact_delta_pp": { + "min": 0.41983119224074983, + "max": 0.6110860229821071, + "mean": 0.5239506378619773 + }, + "score_delta": { + "min": -0.0017724385748465465, + "max": 0.555965927062843 + }, + "exact_rank_changes": { + "gpt-6-luna": { + "published": 4, + "proposed": 5 + }, + "claude-sonnet-5.5": { + "published": 5, + "proposed": 4 + } + }, + "within1pct_rank_changes": { + "gpt-6.1-sol": { + "published": 9, + "proposed": 10 + }, + "grok-4.7": { + "published": 10, + "proposed": 9 + }, + "gemini-3.5-flash-lite": { + "published": 27, + "proposed": 26 + }, + "glm-5.3": { + "published": 26, + "proposed": 27 + }, + "claude-opus-4.8": { + "published": 40, + "proposed": 39 + }, + "minimax-m3": { + "published": 39, + "proposed": 40 + } + }, + "score_rank_changes": { + "gpt-5.5": { + "published": 14, + "proposed": 13 + }, + "gpt-5.6-terra": { + "published": 13, + "proposed": 14 + }, + "claude-opus-5": { + "published": 24, + "proposed": 23 + }, + "deepseek-v4-flash-0731": { + "published": 23, + "proposed": 24 + }, + "minimax-m3": { + "published": 43, + "proposed": 42 + }, + "claude-sonnet-5": { + "published": 42, + "proposed": 43 + } + }, + "always_zero_baseline": { + "published": { + "exact": 69.97959457368849, + "within1pct": 69.97959457368849 + }, + "proposed": { + "exact": 70.39464856524052, + "within1pct": 70.39464856524052 + } + }, + "leader_margin_over_always_zero_exact": { + "model": "gpt-6-sol", + "published": 25.023904562030225, + "proposed": 25.219936593460304 + }, + "top5_exact_published": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "gpt-6-luna", + "claude-sonnet-5.5" + ], + "top5_exact_proposed": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "payroll_exact_delta_by_model": {}, + "payroll_program": {}, + "measured_against": "published" + }, + "exclude": { + "changed_outputs": [ + "scenario_032/payroll_tax", + "scenario_043/payroll_tax", + "scenario_081/payroll_tax", + "scenario_082/payroll_tax" + ], + "scored_outputs": { + "published": 1928, + "proposed": 1924 + }, + "exact_delta_pp": { + "min": -0.06670007462854244, + "max": 0.6197044822258704, + "mean": 0.43651111567473166 + }, + "score_delta": { + "min": -0.10686930776678594, + "max": 0.24838814821841027 + }, + "exact_rank_changes": { + "gpt-6-luna": { + "published": 4, + "proposed": 5 + }, + "claude-sonnet-5.5": { + "published": 5, + "proposed": 4 + }, + "gpt-6.1-sol": { + "published": 8, + "proposed": 9 + }, + "kimi-k3": { + "published": 9, + "proposed": 8 + }, + "grok-4.7": { + "published": 12, + "proposed": 13 + }, + "inkling": { + "published": 13, + "proposed": 12 + }, + "claude-opus-4.7": { + "published": 29, + "proposed": 30 + }, + "gemini-3-flash-preview": { + "published": 30, + "proposed": 29 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 36, + "proposed": 35 + } + }, + "within1pct_rank_changes": { + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-6.1-sol": { + "published": 9, + "proposed": 10 + }, + "grok-4.7": { + "published": 10, + "proposed": 9 + }, + "gemini-3-flash-preview": { + "published": 31, + "proposed": 32 + }, + "grok-build-0.1": { + "published": 34, + "proposed": 31 + }, + "claude-sonnet-4.6": { + "published": 32, + "proposed": 34 + }, + "gemini-3.5-flash": { + "published": 36, + "proposed": 35 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + } + }, + "score_rank_changes": { + "kimi-k3": { + "published": 19, + "proposed": 18 + }, + "gpt-5.6-luna": { + "published": 18, + "proposed": 19 + }, + "grok-4.3": { + "published": 36, + "proposed": 35 + }, + "gemini-3.1-flash-lite-preview": { + "published": 35, + "proposed": 36 + } + }, + "always_zero_baseline": { + "published": { + "exact": 69.97959457368849, + "within1pct": 69.97959457368849 + }, + "proposed": { + "exact": 70.52527771951148, + "within1pct": 70.52527771951148 + } + }, + "leader_margin_over_always_zero_exact": { + "model": "gpt-6-sol", + "published": 25.023904562030225, + "proposed": 24.56800728567967 + }, + "top5_exact_published": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "gpt-6-luna", + "claude-sonnet-5.5" + ], + "top5_exact_proposed": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "payroll_exact_delta_by_model": { + "gpt-6-sol": -3, + "gpt-5.6-sol": -3, + "gpt-6-luna": -2, + "claude-sonnet-5.5": -1, + "gpt-6-astra": -3, + "gpt-6.1-sol": -4, + "kimi-k3": -3, + "gpt-5.6-luna": -2, + "grok-4.7": -2, + "inkling": -1, + "gpt-5.5": -2, + "deepseek-v4.1-flash": -2, + "grok-4.6": -1, + "gemini-3.7-flash": -1, + "gemini-3.6-flash": -1, + "gemini-3.1-pro-preview": -1, + "deepseek-v4-pro": -1, + "glm-5.2": -1, + "kimi-k2.6": -1 + }, + "payroll_program": { + "payroll_tax": { + "exact_published": 78.5024154589372, + "exact_proposed": 81.00686498855836, + "exact_delta": 2.5044495296211693, + "score_published": 94.95457150909301, + "score_proposed": 95.12885899304094, + "score_delta": 0.17428748394792137, + "n_published": 4554, + "n_proposed": 4370, + "n_delta": -184 + } + }, + "measured_against": "published" + }, + "regenerate": { + "changed_outputs": [ + "scenario_032/payroll_tax", + "scenario_043/payroll_tax", + "scenario_081/payroll_tax", + "scenario_082/payroll_tax" + ], + "scored_outputs": { + "published": 1928, + "proposed": 1928 + }, + "exact_delta_pp": { + "min": -0.6879552856780577, + "max": 0.687955285678072, + "mean": 0.16946573844677584 + }, + "score_delta": { + "min": -0.04009914519481583, + "max": 0.037873732016365125 + }, + "exact_rank_changes": { + "gpt-6-luna": { + "published": 4, + "proposed": 5 + }, + "claude-sonnet-5.5": { + "published": 5, + "proposed": 4 + }, + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-6.1-sol": { + "published": 8, + "proposed": 9 + }, + "kimi-k3": { + "published": 9, + "proposed": 8 + }, + "gpt-5.6-luna": { + "published": 10, + "proposed": 11 + }, + "ox-alpha": { + "published": 11, + "proposed": 10 + }, + "grok-4.7": { + "published": 12, + "proposed": 13 + }, + "inkling": { + "published": 13, + "proposed": 12 + }, + "gpt-5.5": { + "published": 14, + "proposed": 15 + }, + "gpt-5.6-terra": { + "published": 15, + "proposed": 14 + }, + "grok-4.6": { + "published": 17, + "proposed": 18 + }, + "gemini-3.8-flash": { + "published": 18, + "proposed": 17 + }, + "deepseek-v4-pro-0813": { + "published": 22, + "proposed": 23 + }, + "deepseek-v4-flash-0731": { + "published": 23, + "proposed": 22 + }, + "gemini-3-flash-preview": { + "published": 30, + "proposed": 31 + }, + "grok-4.3": { + "published": 31, + "proposed": 30 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 36, + "proposed": 35 + } + }, + "within1pct_rank_changes": { + "gpt-6-sol": { + "published": 1, + "proposed": 2 + }, + "claude-opus-5.5": { + "published": 2, + "proposed": 1 + }, + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-6.1-sol": { + "published": 9, + "proposed": 12 + }, + "gpt-5.6-luna": { + "published": 11, + "proposed": 10 + }, + "ox-alpha": { + "published": 13, + "proposed": 11 + }, + "grok-4.7": { + "published": 10, + "proposed": 9 + }, + "inkling": { + "published": 12, + "proposed": 13 + }, + "claude-fable-5": { + "published": 23, + "proposed": 22 + }, + "deepseek-v4-pro-0813": { + "published": 21, + "proposed": 23 + }, + "deepseek-v4-flash-0731": { + "published": 22, + "proposed": 21 + }, + "gemini-3.7-flash": { + "published": 25, + "proposed": 24 + }, + "gemini-3.6-flash": { + "published": 24, + "proposed": 25 + }, + "gemini-3-flash-preview": { + "published": 31, + "proposed": 34 + }, + "grok-build-0.1": { + "published": 34, + "proposed": 32 + }, + "claude-sonnet-4.6": { + "published": 32, + "proposed": 33 + }, + "gemini-3.5-flash": { + "published": 36, + "proposed": 35 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 33, + "proposed": 31 + }, + "qwen3.8-max": { + "published": 42, + "proposed": 43 + }, + "gpt-5.4-mini": { + "published": 43, + "proposed": 42 + } + }, + "score_rank_changes": { + "deepseek-v4-pro-0813": { + "published": 21, + "proposed": 22 + }, + "gemini-3.7-flash": { + "published": 22, + "proposed": 21 + } + }, + "always_zero_baseline": { + "published": { + "exact": 69.97959457368849, + "within1pct": 69.97959457368849 + }, + "proposed": { + "exact": 69.97959457368849, + "within1pct": 69.97959457368849 + } + }, + "leader_margin_over_always_zero_exact": { + "model": "gpt-6-sol", + "published": 25.023904562030225, + "proposed": 24.64892116455411 + }, + "top5_exact_published": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "gpt-6-luna", + "claude-sonnet-5.5" + ], + "top5_exact_proposed": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "payroll_exact_delta_by_model": { + "gpt-6-sol": -2, + "claude-opus-5.5": 4, + "gpt-5.6-sol": -2, + "claude-sonnet-5.5": 2, + "gpt-6-astra": -3, + "claude-fable-5.1": 4, + "gpt-6.1-sol": -4, + "kimi-k3": -3, + "gpt-5.6-luna": -1, + "ox-alpha": 4, + "inkling": 1, + "gpt-5.5": -1, + "gpt-5.6-terra": 2, + "deepseek-v4.1-flash": -2, + "grok-4.6": 1, + "gemini-3.8-flash": 4, + "grok-4.5": 2, + "claude-opus-5": 3, + "claude-fable-5": 2, + "deepseek-v4-flash-0731": 3, + "gemini-3.5-flash-lite": 3, + "glm-5.3": 3, + "claude-opus-4.7": 2, + "gemini-3-flash-preview": 1, + "grok-4.3": 3, + "grok-build-0.1": 3, + "claude-sonnet-4.6": 2, + "gemini-3.5-flash": 2, + "deepseek-v4-pro": -1, + "gemini-3.1-flash-lite-preview": 3, + "qwen-3.7-max": 2, + "glm-5.2": -1, + "claude-opus-4.8": 3, + "minimax-m3": 3, + "qwen3.8-max": 2, + "claude-haiku-4.5": 1, + "gpt-5.4-mini": 1, + "claude-sonnet-5": 2, + "kimi-k2.6": -1 + }, + "payroll_program": { + "payroll_tax": { + "exact_published": 78.5024154589372, + "exact_proposed": 79.53447518664908, + "exact_delta": 1.0320597277118821, + "score_published": 94.95457150909301, + "score_proposed": 95.01545619845788, + "score_delta": 0.060884689364868905, + "n_published": 4554, + "n_proposed": 4554, + "n_delta": 0 + } + }, + "measured_against": "published" + }, + "exclude+salt": { + "changed_outputs": [ + "scenario_032/payroll_tax", + "scenario_043/payroll_tax", + "scenario_081/payroll_tax", + "scenario_082/payroll_tax" + ], + "scored_outputs": { + "published": 1925, + "proposed": 1921 + }, + "exact_delta_pp": { + "min": -0.021612722479446234, + "max": 0.7121113961713235, + "mean": 0.5194277429690684 + }, + "score_delta": { + "min": -0.08929133009138468, + "max": 0.3020164401803527 + }, + "exact_rank_changes": { + "grok-4.7": { + "published": 12, + "proposed": 13 + }, + "inkling": { + "published": 13, + "proposed": 12 + }, + "claude-opus-4.7": { + "published": 29, + "proposed": 30 + }, + "gemini-3-flash-preview": { + "published": 30, + "proposed": 29 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 36, + "proposed": 35 + } + }, + "within1pct_rank_changes": { + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-6.1-sol": { + "published": 10, + "proposed": 11 + }, + "gpt-5.6-luna": { + "published": 11, + "proposed": 10 + }, + "gemini-3.5-flash-lite": { + "published": 26, + "proposed": 27 + }, + "glm-5.3": { + "published": 27, + "proposed": 26 + }, + "gemini-3-flash-preview": { + "published": 31, + "proposed": 32 + }, + "grok-build-0.1": { + "published": 34, + "proposed": 31 + }, + "claude-sonnet-4.6": { + "published": 32, + "proposed": 34 + }, + "gemini-3.5-flash": { + "published": 36, + "proposed": 35 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + } + }, + "score_rank_changes": { + "kimi-k3": { + "published": 19, + "proposed": 18 + }, + "gpt-5.6-luna": { + "published": 18, + "proposed": 19 + }, + "claude-opus-5": { + "published": 23, + "proposed": 24 + }, + "deepseek-v4-flash-0731": { + "published": 24, + "proposed": 23 + }, + "grok-4.3": { + "published": 36, + "proposed": 35 + }, + "gemini-3.1-flash-lite-preview": { + "published": 35, + "proposed": 36 + } + }, + "always_zero_baseline": { + "published": { + "exact": 70.39464856524052, + "within1pct": 70.39464856524052 + }, + "proposed": { + "exact": 71.03273862500897, + "within1pct": 71.03273862500897 + } + }, + "leader_margin_over_always_zero_exact": { + "model": "gpt-6-sol", + "published": 25.219936593460304, + "proposed": 24.716719755313363 + }, + "top5_exact_published": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "top5_exact_proposed": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "payroll_exact_delta_by_model": { + "gpt-6-sol": -3, + "gpt-5.6-sol": -3, + "claude-sonnet-5.5": -1, + "gpt-6-luna": -2, + "gpt-6-astra": -3, + "gpt-6.1-sol": -4, + "kimi-k3": -3, + "gpt-5.6-luna": -2, + "grok-4.7": -2, + "inkling": -1, + "gpt-5.5": -2, + "deepseek-v4.1-flash": -2, + "grok-4.6": -1, + "gemini-3.7-flash": -1, + "gemini-3.6-flash": -1, + "gemini-3.1-pro-preview": -1, + "deepseek-v4-pro": -1, + "glm-5.2": -1, + "kimi-k2.6": -1 + }, + "payroll_program": { + "payroll_tax": { + "exact_published": 78.5024154589372, + "exact_proposed": 81.00686498855836, + "exact_delta": 2.5044495296211693, + "score_published": 94.95457150909301, + "score_proposed": 95.12885899304094, + "score_delta": 0.17428748394792137, + "n_published": 4554, + "n_proposed": 4370, + "n_delta": -184 + } + }, + "measured_against": "salt" + }, + "regenerate+salt": { + "changed_outputs": [ + "scenario_032/payroll_tax", + "scenario_043/payroll_tax", + "scenario_081/payroll_tax", + "scenario_082/payroll_tax" + ], + "scored_outputs": { + "published": 1925, + "proposed": 1925 + }, + "exact_delta_pp": { + "min": -0.735274847474443, + "max": 0.735274847474443, + "mean": 0.16946573844677676 + }, + "score_delta": { + "min": -0.04294449211639062, + "max": 0.040557703662329914 + }, + "exact_rank_changes": { + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-5.6-luna": { + "published": 10, + "proposed": 11 + }, + "ox-alpha": { + "published": 11, + "proposed": 10 + }, + "grok-4.7": { + "published": 12, + "proposed": 13 + }, + "inkling": { + "published": 13, + "proposed": 12 + }, + "gpt-5.5": { + "published": 14, + "proposed": 15 + }, + "gpt-5.6-terra": { + "published": 15, + "proposed": 14 + }, + "grok-4.6": { + "published": 17, + "proposed": 18 + }, + "gemini-3.8-flash": { + "published": 18, + "proposed": 17 + }, + "deepseek-v4-pro-0813": { + "published": 22, + "proposed": 23 + }, + "deepseek-v4-flash-0731": { + "published": 23, + "proposed": 22 + }, + "gemini-3.1-pro-preview": { + "published": 27, + "proposed": 28 + }, + "glm-5.3": { + "published": 28, + "proposed": 27 + }, + "gemini-3-flash-preview": { + "published": 30, + "proposed": 31 + }, + "grok-4.3": { + "published": 31, + "proposed": 30 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 36, + "proposed": 35 + } + }, + "within1pct_rank_changes": { + "gpt-6-sol": { + "published": 1, + "proposed": 2 + }, + "claude-opus-5.5": { + "published": 2, + "proposed": 1 + }, + "gpt-6-astra": { + "published": 6, + "proposed": 7 + }, + "claude-fable-5.1": { + "published": 7, + "proposed": 6 + }, + "gpt-6.1-sol": { + "published": 10, + "proposed": 12 + }, + "gpt-5.6-luna": { + "published": 11, + "proposed": 10 + }, + "ox-alpha": { + "published": 13, + "proposed": 11 + }, + "inkling": { + "published": 12, + "proposed": 13 + }, + "claude-fable-5": { + "published": 23, + "proposed": 22 + }, + "deepseek-v4-pro-0813": { + "published": 21, + "proposed": 23 + }, + "deepseek-v4-flash-0731": { + "published": 22, + "proposed": 21 + }, + "gemini-3.7-flash": { + "published": 25, + "proposed": 24 + }, + "gemini-3.6-flash": { + "published": 24, + "proposed": 25 + }, + "gemini-3.5-flash-lite": { + "published": 26, + "proposed": 27 + }, + "glm-5.3": { + "published": 27, + "proposed": 26 + }, + "gemini-3-flash-preview": { + "published": 31, + "proposed": 34 + }, + "grok-build-0.1": { + "published": 34, + "proposed": 32 + }, + "claude-sonnet-4.6": { + "published": 32, + "proposed": 33 + }, + "gemini-3.5-flash": { + "published": 36, + "proposed": 35 + }, + "deepseek-v4-pro": { + "published": 35, + "proposed": 36 + }, + "gemini-3.1-flash-lite-preview": { + "published": 33, + "proposed": 31 + }, + "qwen3.8-max": { + "published": 42, + "proposed": 43 + }, + "gpt-5.4-mini": { + "published": 43, + "proposed": 42 + } + }, + "score_rank_changes": { + "gpt-5.5": { + "published": 13, + "proposed": 14 + }, + "gpt-5.6-terra": { + "published": 14, + "proposed": 13 + } + }, + "always_zero_baseline": { + "published": { + "exact": 70.39464856524052, + "within1pct": 70.39464856524052 + }, + "proposed": { + "exact": 70.39464856524052, + "within1pct": 70.39464856524052 + } + }, + "leader_margin_over_always_zero_exact": { + "model": "gpt-6-sol", + "published": 25.219936593460304, + "proposed": 24.797633634187804 + }, + "top5_exact_published": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "top5_exact_proposed": [ + "gpt-6-sol", + "claude-opus-5.5", + "gpt-5.6-sol", + "claude-sonnet-5.5", + "gpt-6-luna" + ], + "payroll_exact_delta_by_model": { + "gpt-6-sol": -2, + "claude-opus-5.5": 4, + "gpt-5.6-sol": -2, + "claude-sonnet-5.5": 2, + "gpt-6-astra": -3, + "claude-fable-5.1": 4, + "gpt-6.1-sol": -4, + "kimi-k3": -3, + "gpt-5.6-luna": -1, + "ox-alpha": 4, + "inkling": 1, + "gpt-5.5": -1, + "gpt-5.6-terra": 2, + "deepseek-v4.1-flash": -2, + "grok-4.6": 1, + "gemini-3.8-flash": 4, + "grok-4.5": 2, + "claude-opus-5": 3, + "claude-fable-5": 2, + "deepseek-v4-flash-0731": 3, + "gemini-3.5-flash-lite": 3, + "glm-5.3": 3, + "claude-opus-4.7": 2, + "gemini-3-flash-preview": 1, + "grok-4.3": 3, + "grok-build-0.1": 3, + "claude-sonnet-4.6": 2, + "gemini-3.5-flash": 2, + "deepseek-v4-pro": -1, + "gemini-3.1-flash-lite-preview": 3, + "qwen-3.7-max": 2, + "glm-5.2": -1, + "claude-opus-4.8": 3, + "minimax-m3": 3, + "qwen3.8-max": 2, + "claude-haiku-4.5": 1, + "gpt-5.4-mini": 1, + "claude-sonnet-5": 2, + "kimi-k2.6": -1 + }, + "payroll_program": { + "payroll_tax": { + "exact_published": 78.5024154589372, + "exact_proposed": 79.53447518664908, + "exact_delta": 1.0320597277118821, + "score_published": 94.95457150909301, + "score_proposed": 95.01545619845788, + "score_delta": 0.060884689364868905, + "n_published": 4554, + "n_proposed": 4554, + "n_delta": 0 + } + }, + "measured_against": "salt" + } +} diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_models.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_models.csv new file mode 100644 index 00000000..5fe358a2 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_models.csv @@ -0,0 +1,47 @@ +model,exact_published,exact_proposed,exact_delta,within1pct_published,within1pct_proposed,within1pct_delta,within5pct_published,within5pct_proposed,within5pct_delta,within10pct_published,within10pct_proposed,within10pct_delta,score_published,score_proposed,score_delta,outputGroupScore_published,outputGroupScore_proposed,outputGroupScore_delta,exact_rank_published,exact_rank_proposed,exact_rank_change,within1pct_rank_published,within1pct_rank_proposed,within1pct_rank_change,score_rank_published,score_rank_proposed,score_rank_change,n_published,n_proposed,payroll_exact_published,payroll_exact_proposed,payroll_exact_delta +gpt-6-sol,95.00349913571871,95.09328500519115,0.08978586947243627,95.86029750969453,96.01298708979411,0.152689580099576,96.42213904089867,96.57482862099825,0.152689580099576,97.06811228110645,97.06431591710506,-0.003796364001388497,98.0299198559808,98.03504158003801,0.005121724057218557,98.14179607937497,98.14484909641024,0.0030530170352705,1,1,0,1,1,0,1,1,0,1928,1924,98,95,-3 +claude-opus-5.5,93.69749019238289,94.31719467460876,0.6197044822258704,94.88831885276893,95.55311068714387,0.6647918343749382,95.74625086459541,96.41206858714703,0.6658177225516226,96.87372623411001,96.86940502946163,-0.004321204648377375,97.6099738003582,97.64347517956308,0.03350137920487839,98.14448136513198,98.15664873185219,0.012167366720206019,2,2,0,2,2,0,2,2,0,1928,1924,92,92,0 +gpt-5.6-sol,93.57174672125771,93.6985789524106,0.1268322311528891,94.26182283296569,94.43374241626766,0.1719195833019711,95.5545604572212,95.74429639900123,0.18973594178002884,95.87353434639427,95.86973798239286,-0.003796364001402708,96.85553786104374,96.86318125024968,0.007643389205938433,97.66275018846758,97.6644066756765,0.0016564872089190885,3,3,0,3,3,0,6,6,0,1928,1924,94,91,-3 +gpt-6-luna,92.09699160500828,92.34119957341971,0.24420796841143044,93.81308317526569,94.1023784958262,0.28929532056051244,95.73666125898856,96.04377293802713,0.3071116790385702,96.6927665361552,96.66895327729827,-0.023813258856932862,97.41637983000545,97.41074111490151,-0.005638715103941649,97.70199089499401,97.70809592756942,0.006105032575405289,4,5,-1,4,4,0,3,3,0,1928,1924,91,89,-2 +claude-sonnet-5.5,92.08077168426426,92.51851195845713,0.437740274192862,93.10611423025681,93.58894185659874,0.48282762634192977,94.71230799077293,95.21295197559293,0.5006439848200017,96.13462720532719,96.11081394647026,-0.023813258856932862,97.01245986422123,97.01753073179823,0.0050708675770039235,97.03463446957241,97.0434062481639,0.008771778591494694,5,4,1,5,5,0,4,4,0,1928,1924,92,91,-1 +gpt-6-astra,91.67398399934551,91.80081623049841,0.12683223115290332,92.67041936953811,92.64880664705865,-0.021612722479460444,93.91637338255535,93.89578654825257,-0.02058683430277597,94.53781568309596,94.5172288487932,-0.02058683430276176,96.98446045414371,96.98084376082548,-0.0036166933182357752,98.11217521873846,98.1096064140764,-0.0025688046620615523,6,6,0,6,7,-1,5,5,0,1928,1924,86,83,-3 +claude-fable-5.1,90.82809795026888,91.38279160660294,0.5546936563340523,92.24870514513555,92.84848615361868,0.5997810084831343,93.90559636414038,94.52319373110157,0.617597366961192,95.06434828354557,94.9939903648287,-0.0703579187168657,96.61083374877116,96.5795461871227,-0.0312875616484547,97.81482986920358,97.82560139488984,0.010771525686266159,7,7,0,7,6,1,8,8,0,1928,1924,88,88,0 +gpt-6.1-sol,90.5950906002756,90.52839052564705,-0.06670007462854244,90.97537892746472,90.95376620498527,-0.021612722479446234,92.20411569594783,92.18352886164506,-0.02058683430277597,93.44346489403966,93.42287805973689,-0.02058683430277597,95.58118540969375,95.5766704736719,-0.0045149360218488255,97.87878895026749,97.87512381930755,-0.0036651309599449178,8,9,-1,9,10,-1,11,11,0,1928,1924,86,82,-4 +kimi-k3,90.43471486856777,90.5409030589101,0.10618819034233695,91.48262667645122,91.58881486679356,0.10618819034233695,93.03337615499089,92.98294171001676,-0.05043444497412963,93.46669844422065,93.46135135139559,-0.00534709282506185,94.58318717784636,94.57988451887755,-0.003302658968806327,94.53459258075978,94.52890686042959,-0.005685720330191657,9,8,1,8,8,0,19,18,1,1928,1924,92,89,-3 +gpt-5.6-luna,89.30789108165669,89.55261702024342,0.24472593858672553,90.53600129749496,90.78072723608169,0.24472593858672553,91.70421014385452,91.94893608244124,0.24472593858671132,93.43828154470901,93.37807658556241,-0.06020495914660273,94.59529089279916,94.56119845965107,-0.034092433148089185,95.84490092723847,95.84550069771542,0.000599770476952699,10,10,0,11,11,0,18,19,-1,1928,1924,89,87,-2 +ox-alpha,88.49353035198622,89.08789686107778,0.5943665090915573,89.73663128671126,90.3760851479519,0.6394538612406393,91.41211107760786,92.05156493884849,0.6394538612406251,93.00044543189705,92.96976036593769,-0.030685065959360713,94.86340123268947,94.87533163680841,0.011930404118942306,95.98755058341617,95.99893921667221,0.011388633256046887,11,11,0,13,13,0,17,17,0,1928,1924,87,87,0 +grok-4.7,88.3352356001671,88.61013123949486,0.2748956393277666,90.8559540301881,91.1377214766183,0.28176744643019447,93.57550903955742,93.85727648598761,0.28176744643019447,94.84097272845779,94.83562563563274,-0.005347092825047639,96.76800515903547,96.77771950680615,0.009714347770682252,97.36508321195915,97.37107959809688,0.005996386137738341,12,13,-1,10,9,1,7,7,0,1928,1924,92,90,-2 +inkling,88.22006716270263,88.64921673631044,0.4291495736078019,90.22048592027716,90.53434607873793,0.31386015846076987,91.63940759009171,91.95326774855248,0.31386015846076987,93.53534600182131,93.52922217417998,-0.006123827641332014,96.39903404563158,96.41171736276094,0.012683317129358329,96.79351511049066,96.79712317180483,0.0036080613141677986,13,12,1,12,12,0,9,9,0,1928,1924,81,80,-1 +gpt-5.5,88.15164989242267,88.41498228783898,0.26333239541631315,88.86576150317778,88.95337795129073,0.08761644811295355,91.5182660691513,91.61378021254336,0.09551414339206588,93.06851538766645,93.04561884120648,-0.022896546459975298,95.12547300857929,95.10866628305416,-0.01680672552512874,96.46404648991201,96.46644137533842,0.0023948854264119745,14,14,0,15,15,0,14,14,0,1928,1924,88,86,-2 +gpt-5.6-terra,87.69497703300591,88.28934354209746,0.5943665090915431,89.24375535140979,89.88320921265041,0.6394538612406251,91.33116919368196,91.80305586823049,0.4718866745485286,93.25789584167555,93.23408258281862,-0.023813258856932862,95.21005308051073,95.21541394777249,0.005360867261757107,96.38139933140464,96.38865878251823,0.007259451113597493,15,15,0,14,14,0,13,13,0,1928,1924,83,83,0 +deepseek-v4.1-flash,87.0552377499924,87.32115290007705,0.26591515008465194,88.63835381257343,88.589852388136,-0.04850142443743266,91.50223230715568,91.47154724119632,-0.030685065959360713,92.57265838043338,92.54197331447402,-0.030685065959360713,95.05626593089697,95.03193410447226,-0.024331826424713654,96.58652674175693,96.58604069613398,-0.0004860456229494048,16,16,0,16,16,0,15,15,0,1928,1924,88,86,-2 +grok-4.6,86.40208736627577,86.83201367469984,0.4299263084240721,87.74845427809858,88.00266463921929,0.2542103611207125,90.03659610470429,90.2992738387298,0.262677734025516,91.51445377859353,91.50223487866604,-0.012218899927489701,94.910144673076,94.91956500546911,0.009420332393105468,96.76153897191449,96.76794904140074,0.006410069486250336,17,17,0,17,17,0,16,16,0,1928,1924,88,87,-1 +gemini-3.8-flash,85.94579492214984,86.53158488255684,0.5857899604069985,86.74176419758159,87.3275541579886,0.5857899604070127,91.27675751184425,91.93391855578322,0.6571610439389701,93.17056158315293,93.13976734141384,-0.030794241739087624,96.29217728031497,96.30276065524018,0.010583374925204225,97.16992672276967,97.17989683338192,0.009970110612243843,18,18,0,18,18,0,10,10,0,1928,1924,81,81,0 +grok-4.5,85.08110483198445,85.60602251123046,0.5249176792460162,86.00850651724336,86.53342419648939,0.5249176792460304,88.14557170517973,88.50451776353603,0.35894605835629534,90.07512942777474,90.15917885217803,0.08404942440328966,93.72547899612037,93.70181668232198,-0.023662313798396895,95.98818070767912,96.00773227175668,0.019551564077559647,19,19,0,19,19,0,20,20,0,1928,1924,85,85,0 +claude-opus-5,84.02988376350788,84.54307990366988,0.5131961401619947,85.00690856768351,85.52113059602219,0.5142220283386791,86.77312547350387,87.29421930894499,0.5210938354411212,89.09939384862093,88.95034875686206,-0.1490450917588788,93.10434478976686,93.03466098123779,-0.06968380852906364,95.47065392942848,95.48119511878467,0.010541189356189307,20,20,0,20,20,0,24,24,0,1928,1924,85,85,0 +claude-fable-5,83.6253944438549,84.2128443088653,0.5874498650103988,84.0419404286506,84.63626210076343,0.5943216721128266,86.7052616423924,87.162369003887,0.4571073614945931,88.92460130189772,88.90336466004115,-0.021236641856575034,92.87276020256175,92.8861435126481,0.013383310086354072,95.45713913114773,95.46609537150533,0.008956240357605338,21,21,0,23,23,0,26,26,0,1928,1924,82,82,0 +deepseek-v4-pro-0813,83.37167580453175,83.95912566954215,0.5874498650103988,84.82718523955461,85.23891915726165,0.41173391770703915,87.42991217590283,87.40077783876711,-0.029134337135715782,88.68234431644125,88.65320997930554,-0.02913433713570157,93.52207817631968,93.51217107539274,-0.009907100926938028,96.2099699376765,96.21014978190249,0.00017984422598260608,22,22,0,21,21,0,21,21,0,1928,1924,79,79,0 +deepseek-v4-flash-0731,83.2758619160859,83.83963091780217,0.5637690017162669,84.18326968295833,84.74703868467459,0.5637690017162669,88.38514671885079,89.00934225272339,0.6241955338725944,89.46053404560782,89.39677429380235,-0.06375975180546334,93.13916790103261,93.12301846727361,-0.016149433758997134,94.92567743190693,94.93230704722824,0.006629615321301685,23,23,0,22,22,0,23,23,0,1928,1924,79,79,0 +gemini-3.7-flash,83.11782882466474,83.53078723370932,0.4129584090445775,83.63691536248353,84.04987377152811,0.4129584090445775,86.61828353374163,87.03970931569103,0.4214257819493952,88.37484809941563,88.51043629561859,0.13558819620295992,93.49128429689975,93.50787909853628,0.016594801636529155,96.02906927657445,96.03507005087066,0.006000774296211375,24,24,0,25,25,0,22,22,0,1928,1924,71,70,-1 +gemini-3.6-flash,83.09499152818556,83.51894335119485,0.4239518230092898,83.77568744351451,84.19963926652379,0.4239518230092756,87.90786893770674,88.38537548576991,0.4775065480631753,89.93339257104228,90.07997418120993,0.14658161016765803,95.2495232591938,95.29039788633139,0.040874627137597486,96.81412429532105,96.82827606472799,0.01415176940693641,25,25,0,24,24,0,12,12,0,1928,1924,70,69,-1 +gemini-3.5-flash-lite,81.9592849674438,82.5116748044665,0.5523898370226874,82.84338141389416,83.22133225712233,0.3779508432281631,84.18154889958008,84.55949974280823,0.37795084322814887,85.9229953561171,85.8052462659398,-0.1177490901772984,90.03121813140012,89.98577227455431,-0.04544585684581648,92.99058422606034,92.99074876524875,0.00016453918841818904,26,26,0,27,27,0,32,32,0,1928,1924,78,78,0 +gemini-3.1-pro-preview,81.63648613293833,82.07890412197949,0.4424179890411608,82.21503833486231,82.65745632390346,0.4424179890411466,84.53467776508941,85.03065047918444,0.4959727140950321,86.46018765972694,86.62523543592648,0.16504777619954325,92.95579714889485,93.00654592486063,0.05074877596578631,95.53168217701608,95.53675063505584,0.005068458039758639,27,27,0,29,29,0,25,25,0,1928,1924,66,65,-1 +glm-5.3,81.09010878646016,81.65730686694565,0.5671980804854826,82.887622207975,83.49990764060955,0.6122854326345504,85.34735655714336,85.78144886400374,0.4340923068603786,86.74625194707212,86.68592127403586,-0.06033067303626183,89.61270934577402,89.60276220122839,-0.009947144545634501,91.60488818087755,91.60338725757387,-0.001500923303680679,28,28,0,26,26,0,34,34,0,1928,1924,81,81,0 +claude-opus-4.7,80.9638962030392,81.47596154710394,0.5120653440647374,81.70908369740485,82.22114904146959,0.5120653440647374,84.00944767428736,84.33587840784979,0.32643073356243235,85.62462327242545,85.64888637836385,0.02426310593840242,89.90261973339878,89.84331118647528,-0.05930854692350351,92.94155966652754,92.95615219125017,0.014592524722630174,29,30,-1,30,30,0,33,33,0,1928,1924,83,83,0 +gemini-3-flash-preview,80.95099070665046,81.55613000287646,0.6051392962259996,81.61251554501574,82.02412253546032,0.41160699044458227,84.75197587267989,85.01706880123542,0.2650929285555321,86.62149249317326,86.73009947762782,0.10860698445455341,92.48205252928467,92.64909727546453,0.16704474617985454,95.89663313888457,95.94749157260887,0.05085843372430077,30,29,1,31,32,-1,27,27,0,1928,1924,65,65,0 +grok-4.3,80.62160266220387,81.20105473311281,0.5794520709089426,82.53905982363676,83.1185118945457,0.5794520709089426,83.55781152859795,84.15507995798497,0.5972684293870145,85.89631293707023,86.0251524267228,0.1288394896525631,89.13971012353002,89.14261270821132,0.002902584681294229,93.6220341272469,93.63129974816486,0.009265620917958017,31,31,0,28,28,0,36,35,1,1928,1924,77,77,0 +grok-build-0.1,80.36084468856437,80.9458591275279,0.5850144389635261,81.44544593251806,82.03892774438641,0.593481811868358,83.45594821451222,84.04943002638056,0.5934818118683438,85.21073113533913,85.30978996731085,0.09905883197171761,91.1324834497806,91.15626086524543,0.023777415464834917,94.36565089664984,94.38346971517979,0.017818818529946157,32,32,0,34,31,3,30,30,0,1928,1924,71,71,0 +claude-sonnet-4.6,80.27617496987095,80.8171304062289,0.5409554363579474,81.52873822877525,81.8830331664542,0.3542949376789437,86.84827007162907,87.05440974689635,0.2061396752672806,88.42639688767113,88.3576399289854,-0.06875695868572507,90.93225103176982,90.8829068519237,-0.049344179846116276,94.43492800125435,94.44181959195672,0.006891590702366557,33,33,0,32,34,-2,31,31,0,1928,1924,83,83,0 +gemini-3.5-flash,80.04238564541743,80.64752494164344,0.6051392962260138,80.65165534394116,81.25679464016716,0.6051392962259996,82.65249080337206,83.26609747250288,0.6136066691308173,85.23510538068777,85.35428906992196,0.11918368923419109,91.9574985410053,92.01209706872874,0.0545985277234422,95.30733194933845,95.31277954548074,0.0054475961422895125,34,34,0,36,35,1,28,28,0,1928,1924,61,61,0 +deepseek-v4-pro,79.75124065440951,80.14392968627774,0.3926890318682297,80.91779413249449,81.13604417056818,0.21825003807369114,85.4731858125084,85.54341727938595,0.07023146687754434,87.93991141377664,87.81661057487277,-0.123300838903873,91.625686267379,91.57708413326735,-0.04860213411164693,94.88443394488235,94.87720068071265,-0.007233264169698828,35,36,-1,35,36,-1,29,29,0,1928,1924,68,67,-1 +gemini-3.1-flash-lite-preview,79.73940156545854,80.25324819869412,0.5138466332355875,81.48438903762849,81.99823567086408,0.5138466332355875,83.00632269033785,83.52016932357343,0.5138466332355733,84.74813485700159,84.74846519835366,0.0003303413520683307,89.14415485236766,89.08550001914202,-0.058654833225645575,93.26084731439737,93.27495116508896,0.014103850691597586,36,35,1,33,33,0,35,36,-1,1928,1924,76,76,0 +qwen-3.7-max,77.38613945075842,77.92699392302774,0.5408544722693165,78.79437682154989,79.33523129381922,0.5408544722693307,81.27196352984511,81.6383790083199,0.36641547847479217,83.17596242907331,83.02886161566458,-0.14710081340872705,87.33527967112641,87.27738734121208,-0.05789232991433835,91.48372111324807,91.48973885180864,0.0060177385605726386,37,37,0,37,37,0,37,37,0,1928,1924,73,73,0 +glm-5.2,77.04754617804436,77.4198964918887,0.37235031384433626,78.40294653618636,78.40732555045473,0.004379014268366177,82.5432767994509,82.55612318662408,0.01284638717318387,84.06122691696918,84.11916065629143,0.057933739322251654,86.86613898600048,86.92388246055846,0.05774347455798079,88.41384713439248,88.44801656967346,0.03416943528097249,38,38,0,38,38,0,38,38,0,1928,1924,73,72,-1 +claude-opus-4.8,76.17545674851087,76.69124583755817,0.5157890890473027,77.25613559407958,77.77192468312688,0.5157890890473027,78.75534238142073,79.27113147046805,0.5157890890473169,81.00663384747759,81.02672300311944,0.020089155641855427,84.91925054467029,84.85959798304238,-0.05965256162791377,91.72770115914611,91.73856418807011,0.010863028924006812,39,39,0,40,40,0,39,39,0,1928,1924,81,81,0 +minimax-m3,75.74649533851503,76.26703183213327,0.5205364936182377,77.2712752290567,77.79181172267494,0.5205364936182377,79.9814357124164,80.50197220603464,0.5205364936182377,80.53641549589192,80.54343569762663,0.007020201734718512,82.10743147259882,82.14698561242889,0.03955413983007361,90.96878161865847,91.00015327669213,0.03137165803366315,40,40,0,39,39,0,43,43,0,1928,1924,63,63,0 +qwen3.8-max,75.22482335536425,75.73021241592843,0.505389060564184,75.83105699989459,76.33644606045875,0.5053890605641698,77.57329061246871,77.72852473193498,0.15523411946627164,78.73445137394236,78.56970150730655,-0.16474986663581603,82.95047048737395,82.85843586080266,-0.0920346265712908,89.7743259143331,89.76779424990302,-0.006531664430085016,41,41,0,42,42,0,41,41,0,1928,1924,71,71,0 +claude-haiku-4.5,74.6603649485753,75.20116798615173,0.5408030375764241,75.97216713709038,76.53078653314486,0.5586193960544819,77.83789532532154,78.2029824155946,0.3650870902730645,79.78305248301713,79.65371659339357,-0.12933588962356168,83.64631494260415,83.59630764933144,-0.05000729327271358,89.75967813092348,89.74955534070962,-0.010122790213856092,42,42,0,41,41,0,40,40,0,1928,1924,65,65,0 +gpt-5.4-mini,73.66669485735765,74.12760463451117,0.46090977715351755,75.63642913518649,76.09733891234,0.46090977715351755,75.92267879361472,76.38358857076824,0.46090977715351755,77.68933140410081,77.63658819815531,-0.052743205945503746,81.29939112383417,81.29807008847426,-0.0013210353599077962,88.88801014329086,88.90920896418984,0.021198820898987947,43,43,0,43,43,0,44,44,0,1928,1924,54,54,0 +claude-sonnet-5,72.64249589279979,73.11445634825245,0.47196045545265974,74.6630018321248,75.13496228757747,0.47196045545265974,75.83771649556213,76.11614464523336,0.2784281496712282,77.42378744351629,77.38223160708543,-0.04155583643085947,82.49049611863575,82.38362681086896,-0.10686930776678594,91.63640300577666,91.64924026741339,0.012837261636732933,44,44,0,44,44,0,42,42,0,1928,1924,76,76,0 +kimi-k2.6,67.40417281119927,67.55572682652985,0.151554015330575,67.42572172627803,67.5772757416086,0.151554015330575,68.53579711586619,68.68735113119676,0.151554015330575,68.99989280107147,69.15144681640204,0.151554015330575,69.45103956892189,69.60279079216527,0.1517512232433802,77.0569946963265,77.11182202724605,0.05482733091955083,45,45,0,45,45,0,46,46,0,1928,1924,46,45,-1 +gpt-5.4-nano,65.33891134939903,65.9065229052872,0.5676115558881776,66.37027185509079,66.93788341097897,0.5676115558881776,68.91289387798616,69.48050543387433,0.5676115558881634,70.82365628740529,71.19773553751203,0.37407925010674603,75.66934446405278,75.9177326122712,0.24838814821841027,88.25864207573021,88.29017748769306,0.03153541196284948,46,46,0,46,46,0,45,45,0,1928,1924,17,17,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_models.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_models.csv new file mode 100644 index 00000000..e34d0696 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_models.csv @@ -0,0 +1,47 @@ +model,exact_published,exact_proposed,exact_delta,within1pct_published,within1pct_proposed,within1pct_delta,within5pct_published,within5pct_proposed,within5pct_delta,within10pct_published,within10pct_proposed,within10pct_delta,score_published,score_proposed,score_delta,outputGroupScore_published,outputGroupScore_proposed,outputGroupScore_delta,exact_rank_published,exact_rank_proposed,exact_rank_change,within1pct_rank_published,within1pct_rank_proposed,within1pct_rank_change,score_rank_published,score_rank_proposed,score_rank_change,n_published,n_proposed,payroll_exact_published,payroll_exact_proposed,payroll_exact_delta +gpt-6-sol,95.61458515870082,95.74945838032234,0.13487322162151827,96.25800028047114,96.41068986057071,0.152689580099576,96.81984181167527,96.97253139177485,0.152689580099576,97.25243179967755,97.24863543567616,-0.003796364001388497,98.06615198900951,98.07133567949359,0.005183690484074077,98.1491468296578,98.15219984669307,0.0030530170352705,1,1,0,1,1,0,1,1,0,1925,1921,98,95,-3 +claude-opus-5.5,94.26125665356861,94.97336804973993,0.7121113961713235,95.23870206174915,95.95081345792048,0.7121113961713235,95.91231455500453,96.62545183935254,0.7131372843480079,97.08710948631551,97.08278828166713,-0.004321204648377375,97.67971469666311,97.71600114901679,0.036286452353678555,98.15499994008667,98.16716730680687,0.012167366720206019,2,2,0,2,2,0,2,2,0,1925,1921,92,92,0 +gpt-5.6-sol,94.1828327442398,94.35475232754177,0.1719195833019711,94.6595256037423,94.83144518704427,0.1719195833019711,95.9522632279978,96.14199916977783,0.18973594178002884,96.27123711717086,96.26744075316947,-0.003796364001388497,96.94152074410239,96.94926511637277,0.007744372270380495,97.68164048821839,97.6832969754273,0.0016564872089048777,3,3,0,3,3,0,6,6,0,1925,1921,94,91,-3 +claude-sonnet-5.5,92.64453814544999,93.17468533358831,0.5301471881383293,93.45649743923701,93.98664462737534,0.5301471881383293,95.07837136687458,95.62633491349098,0.5479635466164012,96.3480104575327,96.32419719867576,-0.023813258856932862,97.08748179841196,97.09531989321862,0.007838094806658091,97.04971451456683,97.05848629315832,0.008771778591494694,4,4,0,5,5,0,4,4,0,1925,1921,92,91,-1 +gpt-6-luna,92.60798390084705,92.94459878320394,0.33661488235689774,94.11069221889896,94.44730710125584,0.3366148823568835,96.09727003153967,96.45170127237462,0.3544312408349555,97.10069487050269,97.07688161164576,-0.023813258856932862,97.46791128693393,97.46503528392695,-0.00287600300697477,97.71185743132558,97.71796246390097,0.006105032575391078,5,5,0,4,4,0,3,3,0,1925,1921,91,89,-2 +gpt-6-astra,92.28507002232762,92.45698960562959,0.1719195833019711,93.06812214031471,93.04650941783525,-0.021612722479460444,94.3297563204534,94.30916948615062,-0.02058683430277597,94.75119893530146,94.7306121009987,-0.02058683430276176,97.06194584714922,97.05843130876629,-0.0035145383829302546,98.12774864615544,98.12517984149338,-0.0025688046620615523,6,6,0,6,7,-1,5,5,0,1925,1921,86,83,-3 +claude-fable-5.1,91.34454484965822,91.99164541993773,0.6471005702795054,92.56744895944082,93.21454952972034,0.6471005702795196,94.02434049275311,94.68925742151069,0.6649169287575774,95.23041197395469,95.16005405523782,-0.0703579187168657,96.68102037595823,96.65252180485545,-0.028498571102772985,97.82847553728683,97.8392470629731,0.010771525686266159,7,7,0,7,6,1,8,8,0,1925,1921,88,88,0 +gpt-6.1-sol,91.20617662325769,91.18456390077824,-0.021612722479446234,91.37308169824132,91.35146897576186,-0.021612722479460444,92.61749863384588,92.59691179954311,-0.02058683430277597,93.65684814624517,93.63626131194239,-0.02058683430277597,95.65866652129478,95.65425374020823,-0.004412781086543305,97.89121325216055,97.8875481212006,-0.0036651309599591286,8,8,0,10,11,-1,11,11,0,1925,1921,86,82,-4 +kimi-k3,90.95116176795709,91.1497568722449,0.19859510428780425,91.99907357584055,92.19766868012834,0.19859510428779004,93.644462177973,93.63911508514794,-0.00534709282506185,93.65101796279176,93.6456708699667,-0.00534709282506185,94.63367736734034,94.63497385038956,0.0012964830492165902,94.54317322730626,94.53748750697605,-0.005685720330205868,9,9,0,8,8,0,19,18,1,1925,1921,92,89,-3 +gpt-5.6-luna,89.91352250108824,90.20333579182405,0.2898132907358075,91.14163271692651,91.4314460076623,0.2898132907357933,92.30984156328607,92.59965485402186,0.2898132907357933,93.61714645972954,93.55694150058295,-0.06020495914658852,94.64256693018902,94.61184437303882,-0.03072255715019878,95.84658947315877,95.84718924363574,0.0005997704769669099,10,10,0,11,10,1,18,19,-1,1925,1921,89,87,-2 +ox-alpha,89.00997725137556,89.69675067441257,0.6867734230370104,90.03969493389509,90.7264683569321,0.6867734230370104,91.76249428658808,92.44926770962509,0.6867734230370104,93.39814820267365,93.36746313671429,-0.030685065959360713,94.94265921491042,94.95729247183509,0.014633256924668103,95.99218122244062,96.0035698556967,0.011388633256075309,11,11,0,13,13,0,17,17,0,1925,1921,87,87,0 +grok-4.7,88.89900206135282,89.21898505282965,0.3199829914768344,91.41972049137382,91.7465752899531,0.32685479857927646,94.18659506253951,94.51344986111879,0.32685479857927646,95.02529224702889,95.01994515420384,-0.005347092825047639,96.83372616430886,96.84755332807796,0.013827163769107642,97.36808696377673,97.37408334991446,0.00599638613772413,12,13,-1,9,9,0,7,7,0,1925,1921,92,90,-2 +inkling,88.7783790203378,89.25261594609468,0.4742369257568839,90.5654145257068,90.87927468416757,0.31386015846076987,92.04733592443918,92.36119608289995,0.31386015846076987,93.72989108396328,93.72376725632196,-0.006123827641317803,96.46751410286626,96.4804056773429,0.012891574476640244,96.8039513725253,96.80755943383947,0.0036080613141677986,13,12,1,12,12,0,9,9,0,1925,1921,81,80,-1 +gpt-5.5,88.71541635360839,89.02383610117379,0.30841974756539514,89.4295279643635,89.56223176462552,0.13270380026202133,92.14503225925485,92.285633754796,0.14060149554114787,93.4818983255645,93.45900177910453,-0.022896546459975298,95.25511312953918,95.24246622877507,-0.012646900764110569,96.48422201987421,96.48661690530064,0.0023948854264261854,14,14,0,15,15,0,13,13,0,1925,1921,88,86,-2 +gpt-5.6-terra,88.20596932884467,88.89274275188168,0.6867734230370104,89.54136439504306,90.22813781808007,0.6867734230370104,91.67609779911162,92.14798447366014,0.4718866745485286,93.40282476141265,93.37901150255573,-0.02381325885691865,95.23345324999597,95.23939604207486,0.005942792078883485,96.37966628568644,96.38692573680004,0.007259451113597493,15,15,0,14,14,0,14,14,0,1925,1921,83,83,0 +deepseek-v4.1-flash,87.57168464938174,87.93000671341184,0.358322064030105,88.98873702155365,88.94023559711621,-0.04850142443743266,91.89993507793228,91.86925001197291,-0.030685065959374924,92.97036115120999,92.93967608525062,-0.030685065959360713,95.12537671317271,95.10154352724643,-0.02383318592627859,96.58602749925093,96.58554145362798,-0.0004860456229494048,16,16,0,16,16,0,15,15,0,1925,1921,88,86,-2 +grok-4.6,86.96585382746149,87.44086748803464,0.4750136605731541,88.32790090640574,88.62719861967554,0.2992977132697945,90.66336229480784,90.97112738098244,0.30776508617459797,91.71445346428608,91.70223456435859,-0.012218899927489701,95.00688197277564,95.02056690980179,0.013684937026141597,96.75952479388886,96.76593486337511,0.006410069486250336,17,17,0,17,17,0,16,16,0,1925,1921,88,87,-1 +gemini-3.8-flash,86.46224182153918,87.14043869589165,0.6781968743524658,87.25821109697092,87.93640797132339,0.6781968743524658,91.59550132614953,92.29998193188487,0.7044806057353412,93.5366249592546,93.50583071751551,-0.030794241739087624,96.40412149584576,96.41926003811668,0.015138542270918265,97.18834204181844,97.19831215243069,0.009970110612243843,18,18,0,18,18,0,10,10,0,1925,1921,81,81,0 +grok-4.5,85.59755173137378,86.21487632456527,0.6173245931914835,86.5249534166327,87.1422780098242,0.6173245931914977,88.77233789528329,89.17637130578866,0.40403341050537733,90.27512911346729,90.35917853787058,0.08404942440328966,93.82543336108115,93.80626090987407,-0.01917245120708344,95.98327882340698,96.00283038748452,0.019551564077545436,19,19,0,19,19,0,20,20,0,1925,1921,85,85,0 +claude-opus-5,84.54087605934664,85.1464791134541,0.605603054107462,85.51790086352227,86.12452980580642,0.6066289422841464,87.28411776934264,87.89761851872922,0.6135007493865743,89.67338587337753,89.56942813376773,-0.10395773960979682,93.24580317131411,93.18671012989586,-0.05909304141825089,95.49057382466182,95.50111501401804,0.010541189356217728,20,20,0,20,20,0,23,24,-1,1925,1921,85,85,0 +claude-fable-5,84.13638673969366,84.81624351864953,0.679856778955866,84.55293272448937,85.23966131054767,0.6867285860583081,87.26357350002756,87.81308777546761,0.5495142754400462,89.31684946912377,89.29561282726719,-0.021236641856575034,92.93410509695849,92.95467508911356,0.0205699921550746,95.45433567003552,95.46329191039314,0.00895624035761955,21,21,0,23,23,0,26,26,0,1925,1921,82,82,0 +deepseek-v4-pro-0813,83.88812270392108,84.56797948287695,0.679856778955866,85.35931230606539,85.8634531377179,0.5041408316525064,87.84329511380088,87.81416077666516,-0.029134337135715782,89.0957272543393,89.0665929172036,-0.02913433713570157,93.62186976023973,93.61506864925198,-0.006801110987751713,96.19919277500318,96.19937261922917,0.00017984422598260608,22,22,0,21,21,0,21,21,0,1925,1921,79,79,0 +deepseek-v4-flash-0731,83.78685421192468,84.4430301275864,0.65617591566172,84.69426197879709,85.35043789445882,0.6561759156617342,88.68275576248406,89.35427085815303,0.6715150956689655,89.80546265103746,89.741702899232,-0.06375975180546334,93.21799737712904,93.20628424952827,-0.011713127600771145,94.92720162977261,94.93383124509391,0.006629615321301685,23,23,0,22,22,0,24,23,1,1925,1921,79,79,0 +gemini-3.7-flash,83.63427572405408,84.13964104704412,0.5053653229900448,84.15336226187287,84.65872758486292,0.5053653229900448,87.1977301620488,87.71156285794365,0.5138326958948483,88.78823103731368,88.92381923351664,0.13558819620295992,93.58208064897043,93.60456206752549,0.022481418555059918,96.02186358030215,96.02786435459838,0.006000774296225586,24,24,0,25,25,0,22,22,0,1925,1921,71,70,-1 +gemini-3.6-flash,83.6114384275749,84.12779716452964,0.5163587369547429,84.29213434290385,84.8084930798586,0.5163587369547429,88.2739323138084,88.79875842366796,0.5248261098595606,90.34677550894033,90.49335711910798,0.14658161016765803,95.32055269959673,95.36754565898414,0.04699295938740988,96.81374451621733,96.82789628562425,0.014151769406922199,25,25,0,24,24,0,12,12,0,1925,1921,70,69,-1 +gemini-3.5-flash-lite,82.47573186683313,83.1205286178013,0.6447967509681689,83.40714787507989,83.83018607045712,0.42303819537723086,84.7453153607658,85.16835355614303,0.42303819537723086,86.48676181730282,86.41410007927459,-0.07266173802823062,90.10722551090474,90.06655769731897,-0.04066781358577032,92.97351203496137,92.9736765741498,0.0001645391884323999,26,26,0,26,27,-1,32,32,0,1925,1921,78,78,0 +gemini-3.1-pro-preview,82.15293303232767,82.68775793531428,0.5348249029866139,82.73148523425164,83.26631013723826,0.5348249029866281,84.88506097406963,85.42835324996105,0.5432922758914174,86.66018734541949,86.82523512161903,0.16504777619954325,93.05098119993194,93.10637290130332,0.05539170137137717,95.52190116745766,95.52696962549739,0.005068458039730217,27,27,0,29,29,0,25,25,0,1925,1921,66,65,-1 +glm-5.3,81.6065556858495,82.26616068028044,0.6596049944309357,83.19068585515883,83.85029084958977,0.6596049944309357,85.65042020432719,86.13183207298394,0.4814118686567497,87.09663515605234,87.03630448301608,-0.06033067303626183,89.79913811903863,89.79190109289807,-0.0072370261405581005,91.63377955359329,91.6322786302896,-0.001500923303680679,28,28,0,27,26,1,34,34,0,1925,1921,81,81,0 +claude-opus-4.7,81.48034310242853,82.08481536043874,0.6044722580102047,82.22553059679419,82.83000285480439,0.6044722580102047,84.57321413547308,84.99205178298098,0.41883764750789965,86.18838973361117,86.30505975349504,0.11667001988386971,90.07301887909907,90.02737571089244,-0.04564316820662384,92.97624792044954,92.99084044517217,0.014592524722630174,29,30,-1,30,30,0,33,33,0,1925,1921,83,83,0 +gemini-3-flash-preview,81.46198300248922,82.1595292126607,0.6975462101714811,82.12350784085451,82.62752174524456,0.5040139043900496,85.35760729211142,85.66778757281604,0.3101802807046141,87.24280407972626,87.39649841632989,0.1536943366036354,92.57686469893929,92.74984410955548,0.17297941061619326,95.88728363804293,95.93814207176723,0.05085843372430077,30,29,1,31,32,-1,27,27,0,1925,1921,65,65,0 +grok-4.3,81.13259495804265,81.80445394289706,0.6718589848544099,83.05005211947554,83.72191110432993,0.6718589848543957,83.86880413874418,84.55847948207665,0.6896753433324676,85.99392229501095,86.1700813464599,0.1761590514489484,89.17131673925229,89.18337630075544,0.012059561503150462,93.59652582567767,93.60579144659566,0.009265620917986439,31,31,0,28,28,0,36,35,1,1925,1921,77,77,0 +grok-build-0.1,80.86761629047841,81.5450376433874,0.6774213529089934,81.9522175344321,82.6381062602459,0.6858887258138111,84.01003937822264,84.69592810403645,0.6858887258138111,85.81214186084594,85.95628804496674,0.1441461841207996,91.25170127374535,91.28837473606599,0.03667346232063551,94.35643149634075,94.3742503148707,0.017818818529946157,32,32,0,34,31,3,30,30,0,1925,1921,71,71,0 +claude-sonnet-4.6,80.78716726570971,81.42052961601313,0.6333623503034147,82.03973052461401,82.48643237623844,0.4467018516244252,87.4539014910606,87.70512851847698,0.2512270274163768,88.61864536920461,88.5498884105189,-0.06875695868571086,90.98708283257356,90.9424070472249,-0.04467578534865879,94.43506195597807,94.44195354668045,0.006891590702380768,33,33,0,32,34,-2,31,31,0,1925,1921,83,83,0 +gemini-3.5-flash,80.55883254480676,81.25637875497824,0.6975462101714811,81.16810224333048,81.86564845350196,0.6975462101714811,83.21625726455777,83.92227084763405,0.7060135830762846,85.86187157079132,86.0261426121746,0.1642710413832731,92.07228767657818,92.13551934848763,0.063231671909449,95.29947401779633,95.3049216139386,0.005447596142275302,34,34,0,36,35,1,28,28,0,1925,1921,61,61,0 +deepseek-v4-pro,80.26223295024828,80.74732889606199,0.4850959458137112,81.47610599012964,81.73944338035243,0.26333739022278735,86.09449739906138,86.20981621808801,0.11531881902662633,88.56122300032963,88.48300951357483,-0.07821348675480522,91.7086754644478,91.66711271365338,-0.041562750794412295,94.87130101083795,94.86406774666824,-0.007233264169713038,35,36,-1,35,36,-1,29,29,0,1925,1921,68,67,-1 +gemini-3.1-flash-lite-preview,80.25584846484787,80.86210201202893,0.6062535471810548,82.00083593701783,82.60708948419887,0.6062535471810406,83.30938633752167,83.91563988470273,0.6062535471810548,85.05119850418542,85.14393575948294,0.09273725529752141,89.22491184785987,89.18143146727195,-0.043480380587922696,93.231555966066,93.24565981675761,0.014103850691611797,36,35,1,33,33,0,35,36,-1,1925,1921,76,76,0 +qwen-3.7-max,77.89713174659718,78.53039313281197,0.6332613862147838,79.30536911738866,79.93863050360345,0.6332613862147838,81.87759494927666,82.28909777990052,0.41150283062385995,83.56821059629935,83.46619713503969,-0.10201346125965927,87.53257387229348,87.49737893663996,-0.0351949356535215,91.49129327238187,91.49731101094244,0.0060177385605726386,37,37,0,37,37,0,37,37,0,1925,1921,73,73,0 +glm-5.2,77.4682400104827,77.9329972382725,0.46475722778980355,78.87095993042108,78.92042629683853,0.04946636641744817,83.05860975548201,83.11654349480428,0.057933739322265865,84.36317662079479,84.42111036011704,0.057933739322251654,86.86436654742563,86.92617697814903,0.06181043072339776,88.41147135632066,88.44564079160162,0.03416943528095828,38,38,0,38,38,0,38,38,0,1925,1921,73,72,-1 +claude-opus-4.8,76.69190364790019,77.30009965089297,0.6081960029927842,77.77258249346892,78.38077849646169,0.60819600299277,79.07178959511752,79.6799855981103,0.6081960029927842,81.32308106117438,81.4355771307617,0.11249606958732272,85.02995146614548,84.98266109277971,-0.0472903733657688,91.73059547308952,91.74145850201356,0.010863028924035234,39,39,0,39,39,0,39,39,0,1925,1921,81,81,0 +minimax-m3,76.22206780769882,76.83501121526253,0.612943407563705,77.74684769824049,78.35979110580419,0.612943407563705,80.49788261180574,81.11082601936944,0.612943407563705,81.05286239528125,81.15228951096144,0.0994271156801858,82.66339739966166,82.7953584534372,0.1319610537755409,91.10339700696609,91.13476866499978,0.031371658033691574,40,40,0,40,40,0,42,42,0,1925,1921,63,63,0 +qwen3.8-max,75.73581565120303,76.33361162571266,0.5977959745096371,76.34204929573335,76.93984527024298,0.5977959745096371,78.13160247010387,78.33192394171923,0.20032147161535363,79.29276323157752,79.17310071709079,-0.11966251448673404,83.27224694624788,83.18631905350495,-0.08592789274293011,89.81424379049652,89.80771212606643,-0.006531664430085016,41,41,0,42,42,0,41,41,0,1925,1921,71,71,0 +claude-haiku-4.5,75.17135724441408,75.80456719593597,0.6332099515218914,76.48315943292914,77.1341857429291,0.6510263099999634,77.93550468326225,78.39299868748078,0.4574940042185318,79.92798140275424,79.84373286527975,-0.0842485374744939,83.65101496555216,83.60854850583878,-0.042466459713381255,89.71726861613516,89.70714582592129,-0.010122790213870303,42,42,0,41,41,0,40,40,0,1925,1921,65,65,0 +gpt-5.4-mini,74.13681272299088,74.69012941408987,0.5533166910989848,76.10654700081972,76.6598636919187,0.5533166910989848,76.39279665924795,76.94611335034693,0.5533166910989848,78.20676883153043,78.199112977734,-0.007655853796421752,81.67682653034329,81.68568751976989,0.008860989426594301,88.96879391106151,88.98999273196053,0.02119882089901637,43,43,0,43,43,0,44,44,0,1925,1921,54,54,0 +claude-sonnet-5,73.15348818863855,73.71785555803667,0.564367369398127,75.17399412796357,75.7383614973617,0.564367369398127,76.34870879140091,76.7195438550176,0.3708350636166955,77.93477973935505,77.98563081686966,0.05085107751460782,82.63641365513257,82.54712232504119,-0.08929133009138468,91.64434375730706,91.65718101894379,0.012837261636732933,44,44,0,44,44,0,43,43,0,1925,1921,76,76,0 +kimi-k2.6,67.82400400344002,68.05728780293161,0.23328379949158773,67.84555291851878,68.07883671801036,0.23328379949158773,68.95562830810694,69.18891210759853,0.23328379949158773,69.41972399331222,69.6530077928038,0.23328379949158773,69.87087076116264,70.10435176856703,0.23348100740439293,77.16794591473611,77.22277324565565,0.05482733091953662,45,45,0,45,45,0,46,46,0,1925,1921,46,45,-1 +gpt-5.4-nano,65.81448381858281,66.47450228841646,0.6600184698336449,66.8458443242746,67.50586279410822,0.6600184698336307,69.38846634716995,70.0484848170036,0.6600184698336449,71.29922875658909,71.76571492064129,0.4664861640521991,76.10932834773762,76.41134478791797,0.3020164401803527,88.35205513270202,88.38359054466487,0.03153541196284948,46,46,0,46,46,0,45,45,0,1925,1921,17,17,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_programs.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_programs.csv new file mode 100644 index 00000000..78887b72 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_plus_salt_programs.csv @@ -0,0 +1,2 @@ +variable,exact_published,exact_proposed,exact_delta,score_published,score_proposed,score_delta,n_published,n_proposed,n_delta +payroll_tax,78.5024154589372,81.00686498855836,2.5044495296211693,94.95457150909301,95.12885899304094,0.17428748394792137,4554,4370,-184 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_programs.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_programs.csv new file mode 100644 index 00000000..78887b72 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_exclude_programs.csv @@ -0,0 +1,2 @@ +variable,exact_published,exact_proposed,exact_delta,score_published,score_proposed,score_delta,n_published,n_proposed,n_delta +payroll_tax,78.5024154589372,81.00686498855836,2.5044495296211693,94.95457150909301,95.12885899304094,0.17428748394792137,4554,4370,-184 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_models.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_models.csv new file mode 100644 index 00000000..2e2c210c --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_models.csv @@ -0,0 +1,47 @@ +model,exact_published,exact_proposed,exact_delta,within1pct_published,within1pct_proposed,within1pct_delta,within5pct_published,within5pct_proposed,within5pct_delta,within10pct_published,within10pct_proposed,within10pct_delta,score_published,score_proposed,score_delta,outputGroupScore_published,outputGroupScore_proposed,outputGroupScore_delta,exact_rank_published,exact_rank_proposed,exact_rank_change,within1pct_rank_published,within1pct_rank_proposed,within1pct_rank_change,score_rank_published,score_rank_proposed,score_rank_change,n_published,n_proposed,payroll_exact_published,payroll_exact_proposed,payroll_exact_delta +gpt-6-sol,95.00349913571871,94.6285157382426,-0.37498339747611453,95.86029750969453,95.4853141122184,-0.37498339747612874,96.42213904089867,96.04715564342254,-0.37498339747612874,97.06811228110645,97.06811228110645,0.0,98.0299198559808,98.00736101506496,-0.02255884091583482,98.14179607937497,98.13500594574762,-0.006790133627347927,1,1,0,1,2,-1,1,1,0,1928,1928,98,96,-2 +claude-opus-5.5,93.69749019238289,94.38544547806096,0.687955285678072,94.88831885276893,95.57627413844699,0.6879552856780577,95.74625086459541,96.43420615027347,0.6879552856780577,96.87372623411001,96.87372623411001,0.0,97.6099738003582,97.64784753237457,0.037873732016365125,98.14448136513198,98.15683514070405,0.012353775572066183,2,2,0,2,1,1,2,2,0,1928,1928,92,96,4 +gpt-5.6-sol,93.57174672125771,93.2708560471425,-0.30089067411520887,94.26182283296569,93.9609321588505,-0.30089067411519466,95.5545604572212,95.253669783106,-0.30089067411519466,95.87353434639427,95.87353434639427,0.0,96.85553786104374,96.83720968021761,-0.0183281808261313,97.66275018846758,97.65598350067222,-0.006766687795362714,3,3,0,3,3,0,6,6,0,1928,1928,94,92,-2 +gpt-6-luna,92.09699160500828,92.07088619512123,-0.026105409887051678,93.81308317526569,93.78697776537864,-0.026105409887051678,95.73666125898856,95.71055584910151,-0.026105409887051678,96.6927665361552,96.6927665361552,0.0,97.41637983000545,97.4141955402267,-0.002184289778753623,97.70199089499401,97.70175521924975,-0.00023567574426408555,4,5,-1,4,4,0,3,3,0,1928,1928,91,91,0 +claude-sonnet-5.5,92.08077168426426,92.44173088594007,0.3609592016758114,93.10611423025681,93.46707343193262,0.3609592016758114,94.71230799077293,95.07326719244874,0.3609592016758114,96.13462720532719,96.13462720532719,0.0,97.01245986422123,97.03204907069008,0.01958920646885076,97.03463446957241,97.04071252686177,0.00607805728935773,5,4,1,5,5,0,4,4,0,1928,1928,92,94,2 +gpt-6-astra,91.67398399934551,91.17956101944888,-0.4944229798966262,92.67041936953811,91.98246408386005,-0.6879552856780577,93.91637338255535,93.22841809687729,-0.6879552856780577,94.53781568309596,94.53781568309596,0.0,96.98446045414371,96.9443613089489,-0.04009914519481583,98.11217521873846,98.09909701579804,-0.01307820294042017,6,7,-1,6,7,-1,5,5,0,1928,1928,86,83,-3 +claude-fable-5.1,90.82809795026888,91.51605323594694,0.6879552856780577,92.24870514513555,92.9366604308136,0.6879552856780577,93.90559636414038,94.59355164981844,0.6879552856780577,95.06434828354557,95.06434828354557,0.0,96.61083374877116,96.64870748078752,0.037873732016365125,97.81482986920358,97.82718364477564,0.012353775572066183,7,6,1,7,6,1,8,8,0,1928,1928,88,92,4 +gpt-6.1-sol,90.5950906002756,89.90713531459754,-0.6879552856780577,90.97537892746472,90.28742364178666,-0.6879552856780577,92.20411569594783,91.51616041026976,-0.687955285678072,93.44346489403966,93.44346489403966,0.0,95.58118540969375,95.54111514544583,-0.04007026424791604,97.87878895026749,97.86571912166066,-0.013069828606830924,8,9,-1,9,12,-3,11,11,0,1928,1928,86,82,-4 +kimi-k3,90.43471486856777,89.92119857668425,-0.5135162918835192,91.48262667645122,90.9691103845677,-0.5135162918835192,93.03337615499089,92.51985986310737,-0.5135162918835192,93.46669844422065,93.46669844422065,0.0,94.58318717784636,94.55213609572529,-0.031051082121066997,94.53459258075978,94.52442364676217,-0.010168933997604768,9,8,1,8,8,0,19,19,0,1928,1928,92,89,-3 +gpt-5.6-luna,89.30789108165669,89.12643998996201,-0.181451091694683,90.53600129749496,90.54808251158171,0.01208121408674856,91.70421014385452,91.71629135794126,0.012081214086734349,93.43828154470901,93.43828154470901,0.0,94.59529089279916,94.5929413938478,-0.002349498951360829,95.84490092723847,95.84396949203065,-0.000931435207817799,10,11,-1,11,10,1,18,18,0,1928,1928,89,88,-1 +ox-alpha,88.49353035198622,89.18148563766428,0.6879552856780577,89.73663128671126,90.42458657238932,0.6879552856780577,91.41211107760786,92.10006636328592,0.6879552856780577,93.00044543189705,93.00044543189705,0.0,94.86340123268947,94.90127103523966,0.03786980255019046,95.98755058341617,95.99990321959828,0.012352636182114907,11,10,1,13,11,2,17,17,0,1928,1928,87,91,4 +grok-4.7,88.3352356001671,88.34731681425383,0.012081214086734349,90.8559540301881,90.86803524427484,0.012081214086734349,93.57550903955742,93.58759025364415,0.012081214086734349,94.84097272845779,94.84097272845779,0.0,96.76800515903547,96.76720713723584,-0.0007980217996248484,97.36508321195915,97.36460285786667,-0.0004803540924740446,12,13,-1,10,9,1,7,7,0,1928,1928,92,92,0 +inkling,88.22006716270263,88.36561215501018,0.14554499230754914,90.22048592027716,90.17249860680327,-0.04798731347388241,91.63940759009171,91.59142027661782,-0.04798731347388241,93.53534600182131,93.53534600182131,0.0,96.39903404563158,96.39706938872773,-0.0019646569038513917,96.79351511049066,96.79367039692947,0.0001552864388116859,13,12,1,12,13,-1,9,9,0,1928,1928,81,82,1 +gpt-5.5,88.15164989242267,87.98422299652829,-0.16742689589437987,88.86576150317778,88.50480230150197,-0.3609592016758114,91.5182660691513,91.15730686747548,-0.3609592016758114,93.06851538766645,93.06851538766645,0.0,95.12547300857929,95.10650555268117,-0.01896745589812099,96.46404648991201,96.45806957025852,-0.005976919653491564,14,15,-1,15,15,0,14,14,0,1928,1928,88,87,-1 +gpt-5.6-terra,87.69497703300591,88.014961019108,0.31998398610208767,89.24375535140979,89.75727164329331,0.5135162918835192,91.33116919368196,91.67024649177094,0.3390772980889807,93.25789584167555,93.25789584167555,0.0,95.21005308051073,95.22734355793717,0.017290477426442408,96.38139933140464,96.3871294524498,0.0057301210451612405,15,14,1,14,14,0,13,13,0,1928,1928,83,85,2 +deepseek-v4.1-flash,87.0552377499924,86.73525376389031,-0.31998398610208767,88.63835381257343,87.95039852689537,-0.6879552856780577,91.50223230715568,90.81427702147762,-0.6879552856780577,92.57265838043338,92.57265838043338,0.0,95.05626593089697,95.01705300773143,-0.03921292316553604,96.58652674175693,96.57373363491149,-0.012793106845435886,16,16,0,16,16,0,15,15,0,1928,1928,88,86,-2 +grok-4.6,86.40208736627577,86.54763235858331,0.14554499230754914,87.74845427809858,87.7004669646247,-0.04798731347388241,90.03659610470429,90.18214109701184,0.14554499230754914,91.51445377859353,91.51445377859353,0.0,94.910144673076,94.9096044476573,-0.000540225418703244,96.76153897191449,96.76210728720889,0.0005683152944016001,17,18,-1,17,17,0,16,16,0,1928,1928,88,89,1 +gemini-3.8-flash,85.94579492214984,86.6337502078279,0.6879552856780577,86.74176419758159,87.42971948325965,0.6879552856780577,91.27675751184425,91.96471279752231,0.6879552856780577,93.17056158315293,93.17056158315293,0.0,96.29217728031497,96.33005101233132,0.037873732016350914,97.16992672276967,97.18228049834175,0.012353775572080394,18,17,1,18,18,0,10,10,0,1928,1928,81,85,4 +grok-4.5,85.08110483198445,85.40108881808654,0.31998398610208767,86.00850651724336,86.32849050334546,0.3199839861021019,88.14557170517973,88.46555569128182,0.31998398610208767,90.07512942777474,90.07512942777474,0.0,93.72547899612037,93.74702634488862,0.021547348768251595,95.98818070767912,95.99535338477394,0.007172677094814617,19,19,0,19,19,0,20,20,0,1928,1928,85,87,2 +claude-opus-5,84.02988376350788,84.5434000553914,0.5135162918835192,85.00690856768351,85.69486385336157,0.6879552856780577,86.77312547350387,87.46108075918193,0.6879552856780577,89.09939384862093,89.09939384862093,0.0,93.10434478976686,93.1419518337653,0.03760704399844883,95.47065392942848,95.48291642229549,0.0122624928670092,20,20,0,20,20,0,24,24,0,1928,1928,85,88,3 +claude-fable-5,83.6253944438549,83.9754126937373,0.35001824988239605,84.0419404286506,84.72989571432866,0.6879552856780577,86.7052616423924,87.22971888606934,0.5244572436769346,88.92460130189772,88.92460130189772,0.0,92.87276020256175,92.90810854382819,0.03534834126644171,95.45713913114773,95.46862802030576,0.011488889158030702,21,21,0,23,22,1,26,26,0,1928,1928,82,84,2 +deepseek-v4-pro-0813,83.37167580453175,83.37167580453175,0.0,84.82718523955461,84.63365293377319,-0.19353230578141734,87.42991217590283,87.2664141339017,-0.16349804200112317,88.68234431644125,88.68234431644125,0.0,93.52207817631968,93.50417291567614,-0.017905260643544807,96.2099699376765,96.2040440448067,-0.005925892869811378,22,23,-1,21,23,-2,21,22,-1,1928,1928,79,79,0 +deepseek-v4-flash-0731,83.2758619160859,83.77028489598253,0.4944229798966262,84.18326968295833,84.87122496863638,0.6879552856780577,88.38514671885079,89.07310200452885,0.6879552856780577,89.46053404560782,89.46053404560782,0.0,93.13916790103261,93.17702255231228,0.037854651279673135,94.92567743190693,94.93802567481919,0.012348242912253227,23,22,1,22,21,1,23,23,0,1928,1928,79,82,3 +gemini-3.7-flash,83.11782882466474,83.1108167267646,-0.007012097900144454,83.63691536248353,83.80434225837791,0.16742689589437987,86.61828353374163,86.78571042963601,0.16742689589437987,88.37484809941563,88.37484809941563,0.0,93.49128429689975,93.5076457039952,0.016361407095445202,96.02906927657445,96.03416491307308,0.005095636498637646,24,24,0,25,24,1,22,21,1,1928,1928,71,71,0 +gemini-3.6-flash,83.09499152818556,83.08797943028542,-0.007012097900144454,83.77568744351451,83.76867534561437,-0.007012097900144454,87.90786893770674,88.07529583360113,0.16742689589439408,89.93339257104228,89.93339257104228,0.0,95.2495232591938,95.26680673953881,0.017283480345014368,96.81412429532105,96.81951932023836,0.005395024917305591,25,25,0,24,25,-1,12,12,0,1928,1928,70,70,0 +gemini-3.5-flash-lite,81.9592849674438,82.47280125932733,0.5135162918835192,82.84338141389416,83.18245871198314,0.3390772980889807,84.18154889958008,84.52062619766906,0.3390772980889807,85.9229953561171,85.9229953561171,0.0,90.03121813140012,90.05021919336465,0.019001061964530663,92.99058422606034,92.99686669629843,0.006282470238090809,26,26,0,27,27,0,32,32,0,1928,1928,78,81,3 +gemini-3.1-pro-preview,81.63648613293833,81.62947403503819,-0.007012097900144454,82.21503833486231,82.3824652307567,0.16742689589437987,84.53467776508941,84.70210466098379,0.16742689589437987,86.46018765972694,86.46018765972694,0.0,92.95579714889485,92.97321143872439,0.017414289829545737,95.53168217701608,95.53711685997351,0.005434682957428549,27,27,0,29,29,0,25,25,0,1928,1928,66,66,0 +glm-5.3,81.09010878646016,81.58453176635679,0.4944229798966262,82.887622207975,83.38204518787163,0.4944229798966262,85.34735655714336,85.84177953703998,0.4944229798966262,86.74625194707212,86.74625194707212,0.0,89.61270934577402,89.6454065155593,0.03269716978527981,91.60488818087755,91.6157270663557,0.010838885478150928,28,28,0,26,26,0,34,34,0,1928,1928,81,84,3 +claude-opus-4.7,80.9638962030392,81.2838801891413,0.3199839861021019,81.70908369740485,82.02906768350694,0.31998398610208767,84.00944767428736,84.32943166038946,0.3199839861021019,85.62462327242545,85.79906226621999,0.17443899379453853,89.90261973339878,89.93575117599013,0.0331314425913547,92.94155966652754,92.95251336335807,0.010953696830526383,29,29,0,30,30,0,33,33,0,1928,1928,83,85,2 +gemini-3-flash-preview,80.95099070665046,81.10747665075142,0.1564859441009645,81.61251554501574,81.57546918333529,-0.03704636168045283,84.75197587267989,84.90846181678086,0.1564859441009645,86.62149249317326,86.62149249317326,0.0,92.48205252928467,92.47906015716882,-0.0029923721158553462,95.89663313888457,95.89621650500315,-0.000416633881414441,30,31,-1,31,34,-3,27,27,0,1928,1928,65,66,1 +grok-4.3,80.62160266220387,81.13511895408739,0.5135162918835192,82.53905982363676,83.05257611552028,0.5135162918835192,83.55781152859795,84.07132782048149,0.5135162918835334,85.89631293707023,85.89631293707023,0.0,89.13971012353002,89.15565474273987,0.015944619209847133,93.6220341272469,93.62732785550321,0.00529372825630503,31,30,1,28,28,0,36,36,0,1928,1928,77,80,3 +grok-build-0.1,80.36084468856437,80.855267668461,0.4944229798966262,81.44544593251806,81.93986891241468,0.4944229798966262,83.45594821451222,83.95037119440885,0.4944229798966262,85.21073113533913,85.21073113533913,0.0,91.1324834497806,91.16783032996587,0.03534688018527277,94.36565089664984,94.377257814133,0.01160691748316367,32,32,0,34,32,2,30,30,0,1928,1928,71,74,3 +claude-sonnet-4.6,80.27617496987095,80.59615895597304,0.31998398610208767,81.52873822877525,81.65518990909591,0.12645168032065612,86.84827007162907,86.99381506393662,0.14554499230754914,88.42639688767113,88.42639688767113,0.0,90.93225103176982,90.93099581152732,-0.001255220242498467,94.43492800125435,94.43528309337377,0.0003550921194204193,33,33,0,32,33,-1,31,31,0,1928,1928,83,85,2 +gemini-3.5-flash,80.04238564541743,80.36236963151953,0.3199839861021019,80.65165534394116,80.97163933004325,0.31998398610208767,82.65249080337206,83.14691378326869,0.4944229798966262,85.23510538068777,85.23510538068777,0.0,91.9574985410053,91.99355666270452,0.036058121699227286,95.30733194933845,95.3191554462275,0.011823496889050489,34,34,0,36,35,1,28,28,0,1928,1928,61,63,2 +deepseek-v4-pro,79.75124065440951,79.58774261240839,-0.16349804200112317,80.91779413249449,80.77338940248026,-0.14440473001423015,85.4731858125084,85.50322007628871,0.03003426378030838,87.93991141377664,87.93991141377664,0.0,91.625686267379,91.61294376683507,-0.01274250054392212,94.88443394488235,94.87964499646573,-0.0047889484166177,35,36,-1,35,36,-1,29,29,0,1928,1928,68,67,-1 +gemini-3.1-flash-lite-preview,79.73940156545854,80.25291785734207,0.5135162918835334,81.48438903762849,81.99790532951201,0.5135162918835192,83.00632269033785,83.51983898222137,0.5135162918835192,84.74813485700159,84.74813485700159,0.0,89.14415485236766,89.18081445583242,0.03665960346475572,93.26084731439737,93.27280809910063,0.011960784703262561,36,35,1,33,31,2,35,35,0,1928,1928,76,79,3 +qwen-3.7-max,77.38613945075842,77.70612343686051,0.31998398610208767,78.79437682154989,79.11436080765199,0.3199839861021019,81.27196352984511,81.61104082793409,0.3390772980889807,83.17596242907331,83.17596242907331,0.0,87.33527967112641,87.35217897566432,0.016899304537901116,91.48372111324807,91.48933369991357,0.005612586665506569,37,37,0,37,37,0,37,37,0,1928,1928,73,75,2 +glm-5.2,77.04754617804436,76.88404813604323,-0.16349804200112317,78.40294653618636,77.87147719460927,-0.5314693415770932,82.5432767994509,82.0118074578738,-0.5314693415770932,84.06122691696918,84.06122691696918,0.0,86.86613898600048,86.83685147484766,-0.029287511152816137,88.41384713439248,88.40451754770329,-0.009329586689190705,38,38,0,38,38,0,38,38,0,1928,1928,73,72,-1 +claude-opus-4.8,76.17545674851087,76.68897304039439,0.5135162918835192,77.25613559407958,77.7696518859631,0.5135162918835192,78.75534238142073,79.26885867330425,0.5135162918835192,81.00663384747759,81.18107284127213,0.17443899379453853,84.91925054467029,84.95662908803797,0.037378543367680095,91.72770115914611,91.73989563338583,0.012194474239720421,39,39,0,40,40,0,39,39,0,1928,1928,81,84,3 +minimax-m3,75.74649533851503,76.26001163039855,0.5135162918835192,77.2712752290567,77.78479152094022,0.5135162918835192,79.9814357124164,80.49495200429992,0.5135162918835192,80.53641549589192,80.53641549589192,0.0,82.10743147259882,82.13507765423368,0.02764618163486432,90.96878161865847,90.97783718679567,0.009055568137199543,40,40,0,39,39,0,43,43,0,1928,1928,63,66,3 +qwen3.8-max,75.22482335536425,75.54480734146634,0.31998398610208767,75.83105699989459,76.15104098599667,0.31998398610208767,77.57329061246871,77.8932745985708,0.31998398610208767,78.73445137394236,78.73445137394236,0.0,82.95047048737395,82.97051634852454,0.02004586115059226,89.7743259143331,89.78108089898979,0.006754984656680563,41,41,0,42,43,-1,41,41,0,1928,1928,71,73,2 +claude-haiku-4.5,74.6603649485753,74.82386299057643,0.16349804200112317,75.97216713709038,76.31010417288604,0.3379370357956617,77.83789532532154,78.13878599943673,0.30089067411519466,79.78305248301713,79.78305248301713,0.0,83.64631494260415,83.66172493142547,0.015409988821318166,89.75967813092348,89.76549309882526,0.005814967901784485,42,42,0,41,41,0,40,40,0,1928,1928,65,66,1 +gpt-5.4-mini,73.66669485735765,73.83019289935876,0.16349804200110896,75.63642913518649,76.16789847676357,0.531469341577079,75.92267879361472,76.45414813519182,0.5314693415770932,77.68933140410081,77.68933140410081,0.0,81.29939112383417,81.31169321902841,0.012302095194243634,88.88801014329086,88.89119617047707,0.0031860271862171885,43,43,0,43,42,1,44,44,0,1928,1928,54,55,1 +claude-sonnet-5,72.64249589279979,72.96247987890187,0.31998398610208767,74.6630018321248,74.9829858182269,0.31998398610208767,75.83771649556213,76.15770048166424,0.3199839861021019,77.42378744351629,77.59822643731083,0.17443899379453853,82.49049611863575,82.52122438111202,0.03072826247627347,91.63640300577666,91.64665980307208,0.010256797295426168,44,44,0,44,44,0,42,42,0,1928,1928,76,78,2 +kimi-k2.6,67.40417281119927,67.24067476919815,-0.16349804200112317,67.42572172627803,67.2622236842769,-0.16349804200112317,68.53579711586619,68.37229907386507,-0.16349804200112317,68.99989280107147,68.99989280107147,0.0,69.45103956892189,69.44162441290798,-0.009415156013901083,77.0569946963265,77.05376317322367,-0.0032315231028263725,45,45,0,45,45,0,46,46,0,1928,1928,46,45,-1 +gpt-5.4-nano,65.33891134939903,65.33891134939903,0.0,66.37027185509079,66.56380416087222,0.19353230578143155,68.91289387798616,69.1064261837676,0.19353230578143155,70.82365628740529,70.82365628740529,0.0,75.66934446405278,75.68974994231168,0.02040547825889405,88.25864207573021,88.26508627790362,0.00644420217341235,46,46,0,46,46,0,45,45,0,1928,1928,17,17,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_models.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_models.csv new file mode 100644 index 00000000..a6fa770a --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_models.csv @@ -0,0 +1,47 @@ +model,exact_published,exact_proposed,exact_delta,within1pct_published,within1pct_proposed,within1pct_delta,within5pct_published,within5pct_proposed,within5pct_delta,within10pct_published,within10pct_proposed,within10pct_delta,score_published,score_proposed,score_delta,outputGroupScore_published,outputGroupScore_proposed,outputGroupScore_delta,exact_rank_published,exact_rank_proposed,exact_rank_change,within1pct_rank_published,within1pct_rank_proposed,within1pct_rank_change,score_rank_published,score_rank_proposed,score_rank_change,n_published,n_proposed,payroll_exact_published,payroll_exact_proposed,payroll_exact_delta +gpt-6-sol,95.61458515870082,95.19228219942832,-0.42230295927249983,96.25800028047114,95.83569732119862,-0.42230295927251404,96.81984181167527,96.39753885240276,-0.42230295927251404,97.25243179967755,97.25243179967755,0.0,98.06615198900951,98.04074778666538,-0.02540420234413432,98.1491468296578,98.14235669603045,-0.006790133627347927,1,1,0,1,2,-1,1,1,0,1925,1925,98,96,-2 +claude-opus-5.5,94.26125665356861,94.99653150104305,0.735274847474443,95.23870206174915,95.9739769092236,0.735274847474443,95.91231455500453,96.64758940247897,0.735274847474443,97.08710948631551,97.08710948631551,0.0,97.67971469666311,97.72027240032544,0.040557703662329914,98.15499994008667,98.16735371565873,0.012353775572066183,2,2,0,2,1,1,2,2,0,1925,1925,92,96,4 +gpt-5.6-sol,94.1828327442398,93.83462250832822,-0.34821023591157996,94.6595256037423,94.3113153678307,-0.34821023591159417,95.9522632279978,95.60405299208622,-0.34821023591157996,96.27123711717086,96.27123711717086,0.0,96.94152074410239,96.92034721635468,-0.02117352774770609,97.68164048821839,97.67487380042303,-0.006766687795362714,3,3,0,3,3,0,6,6,0,1925,1925,94,92,-2 +claude-sonnet-5.5,92.64453814544999,93.05281690892218,0.4082787634721967,93.45649743923701,93.86477620270922,0.4082787634722109,95.07837136687458,95.48665013034679,0.4082787634722109,96.3480104575327,96.3480104575327,0.0,97.08748179841196,97.10975497652677,0.02227317811481555,97.04971451456683,97.0557925718562,0.006078057289371941,4,4,0,5,5,0,4,4,0,1925,1925,92,94,2 +gpt-6-luna,92.60798390084705,92.6291980527564,0.021214151909347834,94.11069221889896,94.1319063708083,0.021214151909333623,96.09727003153967,96.118484183449,0.021214151909333623,97.10069487050269,97.10069487050269,0.0,97.46791128693393,97.46840858134054,0.0004972944066139462,97.71185743132558,97.71162175558132,-0.00023567574426408555,5,5,0,4,4,0,3,3,0,1925,1925,91,91,0 +gpt-6-astra,92.28507002232762,91.7433274806346,-0.5417425416930115,93.06812214031471,92.33284729284027,-0.735274847474443,94.3297563204534,93.59448147297896,-0.735274847474443,94.75119893530146,94.75119893530146,0.0,97.06194584714922,97.01900135503283,-0.04294449211639062,98.12774864615544,98.11467044321502,-0.01307820294042017,6,7,-1,6,7,-1,5,5,0,1925,1925,86,83,-3 +claude-fable-5.1,91.34454484965822,92.07981969713266,0.735274847474443,92.56744895944082,93.30272380691527,0.735274847474443,94.02434049275311,94.75961534022755,0.735274847474443,95.23041197395469,95.23041197395469,0.0,96.68102037595823,96.72157807962054,0.0405577036623157,97.82847553728683,97.84082931285893,0.012353775572094605,7,6,1,7,6,1,8,8,0,1925,1925,88,92,4 +gpt-6.1-sol,91.20617662325769,90.47090177578325,-0.735274847474443,91.37308169824132,90.63780685076688,-0.735274847474443,92.61749863384588,91.88222378637143,-0.7352748474744573,93.65684814624517,93.65684814624517,0.0,95.65866652129478,95.6157509101253,-0.04291561116947662,97.89121325216055,97.87814342355374,-0.013069828606816714,8,8,0,10,12,-2,11,11,0,1925,1925,86,82,-4 +kimi-k3,90.95116176795709,90.43764547607357,-0.5135162918835192,91.99907357584055,91.48555728395702,-0.5135162918835334,93.644462177973,93.13094588608948,-0.5135162918835192,93.65101796279176,93.65101796279176,0.0,94.63367736734034,94.6022330477665,-0.03144431957383631,94.54317322730626,94.53300429330864,-0.010168933997618979,9,9,0,8,8,0,19,19,0,1925,1925,92,89,-3 +gpt-5.6-luna,89.91352250108824,89.68475184759717,-0.2287706534910683,91.14163271692651,91.10639436921686,-0.03523834770965095,92.30984156328607,92.27460321557642,-0.03523834770965095,93.61714645972954,93.61714645972954,0.0,94.64256693018902,94.63737200208682,-0.005194928102199015,95.84658947315877,95.84565803795093,-0.0009314352078462207,10,11,-1,11,10,1,18,18,0,1925,1925,89,88,-1 +ox-alpha,89.00997725137556,89.74525209885,0.735274847474443,90.03969493389509,90.77496978136953,0.735274847474443,91.76249428658808,92.49776913406252,0.735274847474443,93.39814820267365,93.39814820267365,0.0,94.94265921491042,94.98321298910659,0.040553774196169456,95.99218122244062,96.00453385862276,0.012352636182143328,11,10,1,13,11,2,17,17,0,1925,1925,87,91,4 +grok-4.7,88.89900206135282,88.86376371364317,-0.03523834770965095,91.41972049137382,91.38448214366417,-0.03523834770965095,94.18659506253951,94.15135671482987,-0.03523834770963674,95.02529224702889,95.02529224702889,0.0,96.83372616430886,96.83008279558767,-0.003643368721185425,97.36808696377673,97.36760660968426,-0.0004803540924740446,12,13,-1,9,9,0,7,7,0,1925,1925,92,92,0 +inkling,88.7783790203378,88.87660445084896,0.09822543051116384,90.5654145257068,90.47010765043655,-0.0953068752702535,92.04733592443918,91.95202904916893,-0.0953068752702535,93.72989108396328,93.72989108396328,0.0,96.46751410286626,96.46270401681157,-0.004810086054689577,96.8039513725253,96.80410665896413,0.00015528643882589677,13,12,1,12,13,-1,9,9,0,1925,1925,81,82,1 +gpt-5.5,88.71541635360839,88.50066989591762,-0.21474645769076517,89.4295279643635,89.0212492008913,-0.4082787634721967,92.14503225925485,91.73675349578265,-0.4082787634721967,93.4818983255645,93.4818983255645,0.0,95.25511312953918,95.2333003267195,-0.021812802819681565,96.48422201987421,96.4782451002207,-0.0059769196535057745,14,15,-1,15,15,0,13,14,-1,1925,1925,88,87,-1 +gpt-5.6-terra,88.20596932884467,88.52595331494676,0.31998398610208767,89.54136439504306,90.05488068692658,0.5135162918835192,91.67609779911162,91.96785553540421,0.2917577362925954,93.40282476141265,93.40282476141265,0.0,95.23345324999597,95.24786679065195,0.014413540655979773,96.37966628568644,96.38539640673162,0.005730121045175451,15,14,1,14,14,0,14,13,1,1925,1925,83,85,2 +deepseek-v4.1-flash,87.57168464938174,87.25170066327965,-0.31998398610208767,88.98873702155365,88.2534621740792,-0.735274847474443,91.89993507793228,91.16466023045784,-0.735274847474443,92.97036115120999,92.97036115120999,0.0,95.12537671317271,95.08355884597471,-0.04181786719800584,96.58602749925093,96.57323439240548,-0.012793106845450097,16,16,0,16,16,0,15,15,0,1925,1925,88,86,-2 +grok-4.6,86.96585382746149,87.06407925797265,0.09822543051116384,88.32790090640574,88.23259403113548,-0.09530687527026771,90.66336229480784,90.761587725319,0.09822543051116384,91.71445346428608,91.71445346428608,0.0,95.00688197277564,95.00349631820612,-0.003385654569527219,96.75952479388886,96.76009310918326,0.0005683152944016001,17,18,-1,17,17,0,16,16,0,1925,1925,88,89,1 +gemini-3.8-flash,86.46224182153918,87.19751666901362,0.735274847474443,87.25821109697092,87.99348594444537,0.735274847474443,91.59550132614953,92.33077617362397,0.735274847474443,93.5366249592546,93.5366249592546,0.0,96.40412149584576,96.44467919950809,0.040557703662329914,97.18834204181844,97.20069581739052,0.012353775572080394,18,17,1,18,18,0,10,10,0,1925,1925,81,85,4 +grok-4.5,85.59755173137378,85.91753571747587,0.31998398610208767,86.5249534166327,86.84493740273479,0.31998398610208767,88.77233789528329,89.09232188138537,0.31998398610208767,90.27512911346729,90.27512911346729,0.0,93.82543336108115,93.84584332242508,0.020409961343929695,95.98327882340698,95.99045150050178,0.007172677094800406,19,19,0,19,19,0,20,20,0,1925,1925,85,87,2 +claude-opus-5,84.54087605934664,85.05439235123018,0.5135162918835334,85.51790086352227,86.25317571099671,0.735274847474443,87.28411776934264,88.01939261681709,0.735274847474443,89.67338587337753,89.67338587337753,0.0,93.24580317131411,93.28609365568344,0.040290484369336355,95.49057382466182,95.50283631752883,0.0122624928670092,20,20,0,20,20,0,23,23,0,1925,1925,85,88,3 +claude-fable-5,84.13638673969366,84.48640498957606,0.35001824988239605,84.55293272448937,85.28820757196381,0.735274847474443,87.26357350002756,87.83535030550088,0.5717768054733199,89.31684946912377,89.31684946912377,0.0,92.93410509695849,92.97211356263654,0.03800846567804683,95.45433567003552,95.46582455919356,0.011488889158044913,21,21,0,23,22,1,26,26,0,1925,1925,82,84,2 +deepseek-v4-pro-0813,83.88812270392108,83.88812270392108,0.0,85.35931230606539,85.16578000028396,-0.19353230578143155,87.84329511380088,87.67979707179975,-0.16349804200112317,89.0957272543393,89.0957272543393,0.0,93.62186976023973,93.60523519356869,-0.016634566671044126,96.19919277500318,96.19326688213339,-0.005925892869797167,22,23,-1,21,23,-2,21,21,0,1925,1925,79,79,0 +deepseek-v4-flash-0731,83.78685421192468,84.32859675361769,0.5417425416930115,84.69426197879709,85.42953682627153,0.735274847474443,88.68275576248406,89.4180306099585,0.735274847474443,89.80546265103746,89.80546265103746,0.0,93.21799737712904,93.25853600005468,0.040538622925637924,94.92720162977261,94.93954987268489,0.012348242912281648,23,22,1,22,21,1,24,24,0,1925,1925,79,82,3 +gemini-3.7-flash,83.63427572405408,83.62726362615392,-0.007012097900158665,84.15336226187287,84.36810871956364,0.21474645769076517,87.1977301620488,87.41247661973956,0.21474645769076517,88.78823103731368,88.78823103731368,0.0,93.58208064897043,93.60073899562869,0.018658346658256164,96.02186358030215,96.02695921680079,0.005095636498637646,24,24,0,25,24,1,22,22,0,1925,1925,71,71,0 +gemini-3.6-flash,83.6114384275749,83.60442632967475,-0.007012097900144454,84.29213434290385,84.28512224500369,-0.007012097900158665,88.2739323138084,88.48867877149918,0.21474645769077938,90.34677550894033,90.34677550894033,0.0,95.32055269959673,95.34040682961015,0.01985413001341385,96.81374451621733,96.81913954113462,0.00539502491729138,25,25,0,24,25,-1,12,12,0,1925,1925,70,70,0 +gemini-3.5-flash-lite,82.47573186683313,82.98924815871666,0.5135162918835334,83.40714787507989,83.69890561137248,0.2917577362925954,84.7453153607658,85.0370730970584,0.2917577362925954,86.48676181730282,86.48676181730282,0.0,90.10722551090474,90.1237910098438,0.016565498939058898,92.97351203496137,92.97979450519946,0.006282470238090809,26,26,0,26,27,-1,32,32,0,1925,1925,78,81,3 +gemini-3.1-pro-preview,82.15293303232767,82.14592093442752,-0.007012097900144454,82.73148523425164,82.94623169194242,0.21474645769077938,84.88506097406963,85.0998074317604,0.21474645769076517,86.66018734541949,86.66018734541949,0.0,93.05098119993194,93.0709809127912,0.019999712859259944,95.52190116745766,95.52733585041506,0.005434682957400128,27,28,-1,29,29,0,25,25,0,1925,1925,66,66,0 +glm-5.3,81.6065556858495,82.14829822754251,0.5417425416930115,83.19068585515883,83.73242839685184,0.5417425416930115,85.65042020432719,86.1921627460202,0.5417425416930115,87.09663515605234,87.09663515605234,0.0,89.79913811903863,89.83451926046988,0.0353811414312446,91.63377955359329,91.64461843907144,0.010838885478150928,28,27,1,27,26,1,34,34,0,1925,1925,81,84,3 +claude-opus-4.7,81.48034310242853,81.80032708853062,0.31998398610208767,82.22553059679419,82.54551458289627,0.31998398610208767,84.57321413547308,84.89319812157517,0.31998398610208767,86.18838973361117,86.4101482892021,0.22175855559092383,90.07301887909907,90.1086205964548,0.03560171735573192,92.97624792044954,92.98720161728006,0.010953696830526383,29,29,0,30,30,0,33,33,0,1925,1925,83,85,2 +gemini-3-flash-preview,81.46198300248922,81.6184689465902,0.1564859441009787,82.12350784085451,82.08646147917405,-0.037046361680467044,85.35760729211142,85.5140932362124,0.1564859441009787,87.24280407972626,87.24280407972626,0.0,92.57686469893929,92.57273384041447,-0.004130858524817427,95.88728363804293,95.88686700416153,-0.00041663388140023017,30,31,-1,31,34,-3,27,27,0,1925,1925,65,66,1 +grok-4.3,81.13259495804265,81.64611124992616,0.5135162918835192,83.05005211947554,83.56356841135906,0.5135162918835192,83.86880413874418,84.3823204306277,0.5135162918835192,85.99392229501095,85.99392229501095,0.0,89.17131673925229,89.18406849545528,0.012751756202987963,93.59652582567767,93.60181955393398,0.00529372825630503,31,30,1,28,28,0,36,36,0,1925,1925,77,80,3 +grok-build-0.1,80.86761629047841,81.40935883217142,0.5417425416930115,81.9522175344321,82.4939600761251,0.5417425416930115,84.01003937822264,84.55178191991565,0.5417425416930115,85.81214186084594,85.81214186084594,0.0,91.25170127374535,91.289729738116,0.038028464370640336,94.35643149634075,94.36803841382392,0.011606917483177881,32,32,0,34,32,2,30,30,0,1925,1925,71,74,3 +claude-sonnet-4.6,80.78716726570971,81.10715125181181,0.3199839861021019,82.03973052461401,82.16618220493469,0.12645168032067033,87.4539014910606,87.55212692157178,0.09822543051117805,88.61864536920461,88.61864536920461,0.0,90.98708283257356,90.98293173009067,-0.004151102482893521,94.43506195597807,94.43541704809749,0.0003550921194204193,33,33,0,32,33,-1,31,31,0,1925,1925,83,85,2 +gemini-3.5-flash,80.55883254480676,80.87881653090885,0.31998398610208767,81.16810224333048,81.48808622943258,0.3199839861021019,83.21625726455777,83.7579998062508,0.5417425416930257,85.86187157079132,85.86187157079132,0.0,92.07228767657818,92.11099696118099,0.03870928460280254,95.29947401779633,95.31129751468538,0.011823496889050489,34,34,0,36,35,1,28,28,0,1925,1925,61,63,2 +deepseek-v4-pro,80.26223295024828,80.09873490824715,-0.16349804200112317,81.47610599012964,81.28438169831902,-0.19172429181061545,86.09449739906138,86.12453166284169,0.03003426378030838,88.56122300032963,88.56122300032963,0.0,91.7086754644478,91.69404689319636,-0.014628571251435574,94.87130101083795,94.86651206242135,-0.004788948416603489,35,36,-1,35,36,-1,29,29,0,1925,1925,68,67,-1 +gemini-3.1-flash-lite-preview,80.25584846484787,80.76936475673139,0.5135162918835192,82.00083593701783,82.51435222890134,0.5135162918835192,83.30938633752167,83.82290262940519,0.5135162918835192,85.05119850418542,85.05119850418542,0.0,89.22491184785987,89.2639457497417,0.03903390188182243,93.231555966066,93.24351675076925,0.01196078470324835,36,35,1,33,31,2,35,35,0,1925,1925,76,79,3 +qwen-3.7-max,77.89713174659718,78.21711573269927,0.31998398610208767,79.30536911738866,79.62535310349075,0.31998398610208767,81.87759494927666,82.16935268556925,0.2917577362925954,83.56821059629935,83.56821059629935,0.0,87.53257387229348,87.54656113981883,0.013987267525351399,91.49129327238187,91.49690585904735,0.0056125866654781476,37,37,0,37,37,0,37,37,0,1925,1925,73,75,2 +glm-5.2,77.4682400104827,77.30474196848158,-0.16349804200112317,78.87095993042108,78.29217102704762,-0.5787889033734643,83.05860975548201,82.47982085210853,-0.5787889033734785,84.36317662079479,84.36317662079479,0.0,86.86436654742563,86.8322083464954,-0.032158200930226144,88.41147135632066,88.40214176963147,-0.009329586689190705,38,38,0,38,38,0,38,38,0,1925,1925,73,72,-1 +claude-opus-4.8,76.69190364790019,77.20541993978372,0.5135162918835334,77.77258249346892,78.28609878535244,0.5135162918835192,79.07178959511752,79.58530588700104,0.5135162918835192,81.32308106117438,81.5448396167653,0.22175855559092383,85.02995146614548,85.06987965276528,0.039928186619803796,91.73059547308952,91.74278994732927,0.012194474239748843,39,39,0,39,39,0,39,39,0,1925,1925,81,84,3 +minimax-m3,76.22206780769882,76.73558409958234,0.5135162918835192,77.74684769824049,78.260363990124,0.5135162918835192,80.49788261180574,81.01139890368925,0.5135162918835192,81.05286239528125,81.05286239528125,0.0,82.66339739966166,82.69104358129653,0.02764618163486432,91.10339700696609,91.1124525751033,0.009055568137213754,40,40,0,40,40,0,42,42,0,1925,1925,63,66,3 +qwen3.8-max,75.73581565120303,76.05579963730511,0.31998398610208767,76.34204929573335,76.66203328183543,0.31998398610208767,78.13160247010387,78.45158645620596,0.31998398610208767,79.29276323157752,79.29276323157752,0.0,83.27224694624788,83.29130653036158,0.01905958411370534,89.81424379049652,89.82099877515323,0.006754984656708984,41,41,0,42,43,-1,41,41,0,1925,1925,71,73,2 +claude-haiku-4.5,75.17135724441408,75.3348552864152,0.16349804200112317,76.48315943292914,76.86841603052119,0.385256597592047,77.93550468326225,78.28371491917385,0.34821023591159417,79.92798140275424,79.92798140275424,0.0,83.65101496555216,83.66910807597932,0.01809311042715933,89.71726861613516,89.72308358403693,0.005814967901770274,42,42,0,41,41,0,40,40,0,1925,1925,65,66,1 +gpt-5.4-mini,74.13681272299088,74.300310764992,0.16349804200112317,76.10654700081972,76.6853359041932,0.5787889033734785,76.39279665924795,76.97158556262143,0.5787889033734785,78.20676883153043,78.20676883153043,0.0,81.67682653034329,81.69180079966695,0.014974269323658973,88.96879391106151,88.97197993824774,0.0031860271862313994,43,43,0,43,42,1,44,44,0,1925,1925,54,55,1 +claude-sonnet-5,73.15348818863855,73.47347217474064,0.31998398610208767,75.17399412796357,75.49397811406565,0.31998398610208767,76.34870879140091,76.668692777503,0.31998398610208767,77.93477973935505,78.15653829494597,0.22175855559092383,82.63641365513257,82.66961157489125,0.033197919758677585,91.64434375730706,91.6546005546025,0.010256797295440379,44,44,0,44,44,0,43,43,0,1925,1925,76,78,2 +kimi-k2.6,67.82400400344002,67.6605059614389,-0.16349804200112317,67.84555291851878,67.68205487651765,-0.16349804200112317,68.95562830810694,68.79213026610581,-0.16349804200112317,69.41972399331222,69.41972399331222,0.0,69.87087076116264,69.86145560514873,-0.009415156013901083,77.16794591473611,77.16471439163328,-0.0032315231028263725,45,45,0,45,45,0,46,46,0,1925,1925,46,45,-1 +gpt-5.4-nano,65.81448381858281,65.81448381858281,0.0,66.8458443242746,67.03937663005603,0.19353230578143155,69.38846634716995,69.58199865295138,0.19353230578143155,71.29922875658909,71.29922875658909,0.0,76.10932834773762,76.13082043390165,0.021492086164030866,88.35205513270202,88.35849933487545,0.006444202173426561,46,46,0,46,46,0,45,45,0,1925,1925,17,17,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_programs.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_programs.csv new file mode 100644 index 00000000..969f435e --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_plus_salt_programs.csv @@ -0,0 +1,2 @@ +variable,exact_published,exact_proposed,exact_delta,score_published,score_proposed,score_delta,n_published,n_proposed,n_delta +payroll_tax,78.5024154589372,79.53447518664908,1.0320597277118821,94.95457150909301,95.01545619845788,0.060884689364868905,4554,4554,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_programs.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_programs.csv new file mode 100644 index 00000000..969f435e --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_regenerate_programs.csv @@ -0,0 +1,2 @@ +variable,exact_published,exact_proposed,exact_delta,score_published,score_proposed,score_delta,n_published,n_proposed,n_delta +payroll_tax,78.5024154589372,79.53447518664908,1.0320597277118821,94.95457150909301,95.01545619845788,0.060884689364868905,4554,4554,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_models.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_models.csv new file mode 100644 index 00000000..e5b354e1 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_models.csv @@ -0,0 +1,47 @@ +model,exact_published,exact_proposed,exact_delta,within1pct_published,within1pct_proposed,within1pct_delta,within5pct_published,within5pct_proposed,within5pct_delta,within10pct_published,within10pct_proposed,within10pct_delta,score_published,score_proposed,score_delta,outputGroupScore_published,outputGroupScore_proposed,outputGroupScore_delta,exact_rank_published,exact_rank_proposed,exact_rank_change,within1pct_rank_published,within1pct_rank_proposed,within1pct_rank_change,score_rank_published,score_rank_proposed,score_rank_change,n_published,n_proposed,payroll_exact_published,payroll_exact_proposed,payroll_exact_delta +gpt-6-sol,95.00349913571871,95.61458515870082,0.6110860229821071,95.86029750969453,96.25800028047114,0.3977027707766041,96.42213904089867,96.81984181167527,0.3977027707766041,97.06811228110645,97.25243179967755,0.18431951857110107,98.0299198559808,98.06615198900951,0.03623213302871875,98.14179607937497,98.1491468296578,0.007350750282824947,1,1,0,1,1,0,1,1,0,1928,1925,98,98,0 +claude-opus-5.5,93.69749019238289,94.26125665356861,0.5637664611857218,94.88831885276893,95.23870206174915,0.3503832089802188,95.74625086459541,95.91231455500453,0.16606369040911773,96.87372623411001,97.08710948631551,0.21338325220550303,97.6099738003582,97.67971469666311,0.0697408963049071,98.14448136513198,98.15499994008667,0.010518574954687665,2,2,0,2,2,0,2,2,0,1928,1925,92,92,0 +gpt-5.6-sol,93.57174672125771,94.1828327442398,0.6110860229820929,94.26182283296569,94.6595256037423,0.3977027707766041,95.5545604572212,95.9522632279978,0.3977027707766041,95.87353434639427,96.27123711717086,0.3977027707765899,96.85553786104374,96.94152074410239,0.08598288305864799,97.66275018846758,97.68164048821839,0.018890299750808026,3,3,0,3,3,0,6,6,0,1928,1925,94,94,0 +gpt-6-luna,92.09699160500828,92.60798390084705,0.5109922958387614,93.81308317526569,94.11069221889896,0.29760904363327256,95.73666125898856,96.09727003153967,0.36060877255110313,96.6927665361552,97.10069487050269,0.40792833434748843,97.41637983000545,97.46791128693393,0.05153145692847261,97.70199089499401,97.71185743132558,0.009866536331571751,4,5,-1,4,4,0,3,3,0,1928,1925,91,91,0 +claude-sonnet-5.5,92.08077168426426,92.64453814544999,0.5637664611857218,93.10611423025681,93.45649743923701,0.3503832089802046,94.71230799077293,95.07837136687458,0.36606337610164985,96.13462720532719,96.3480104575327,0.21338325220550303,97.01245986422123,97.08748179841196,0.07502193419072967,97.03463446957241,97.04971451456683,0.01508004499441995,5,4,1,5,5,0,4,4,0,1928,1925,92,92,0 +gpt-6-astra,91.67398399934551,92.28507002232762,0.6110860229821071,92.67041936953811,93.06812214031471,0.3977027707766041,93.91637338255535,94.3297563204534,0.41338293789804936,94.53781568309596,94.75119893530146,0.21338325220550303,96.98446045414371,97.06194584714922,0.07748539300550306,98.11217521873846,98.12774864615544,0.015573427416981644,6,6,0,6,6,0,5,5,0,1928,1925,86,86,0 +claude-fable-5.1,90.82809795026888,91.34454484965822,0.5164468993893365,92.24870514513555,92.56744895944082,0.31874381430527876,93.90559636414038,94.02434049275311,0.11874412861273242,95.06434828354557,95.23041197395469,0.16606369040911773,96.61083374877116,96.68102037595823,0.07018662718706992,97.81482986920358,97.82847553728683,0.013645668083256624,7,7,0,7,7,0,8,8,0,1928,1925,88,88,0 +gpt-6.1-sol,90.5950906002756,91.20617662325769,0.6110860229820929,90.97537892746472,91.37308169824132,0.3977027707766041,92.20411569594783,92.61749863384588,0.41338293789804936,93.44346489403966,93.65684814624517,0.21338325220550303,95.58118540969375,95.65866652129478,0.07748111160103122,97.87878895026749,97.89121325216055,0.012424301893062761,8,8,0,9,10,-1,11,11,0,1928,1925,86,86,0 +kimi-k3,90.43471486856777,90.95116176795709,0.5164468993893223,91.48262667645122,91.99907357584055,0.5164468993893365,93.03337615499089,93.644462177973,0.6110860229821071,93.46669844422065,93.65101796279176,0.18431951857110107,94.58318717784636,94.63367736734034,0.05049018949398487,94.53459258075978,94.54317322730626,0.008580646546477055,9,9,0,8,8,0,19,19,0,1928,1925,92,92,0 +gpt-5.6-luna,89.30789108165669,89.91352250108824,0.6056314194315462,90.53600129749496,91.14163271692651,0.6056314194315462,91.70421014385452,92.30984156328607,0.6056314194315462,93.43828154470901,93.61714645972954,0.17886491502052593,94.59529089279916,94.64256693018902,0.047276037389863745,95.84490092723847,95.84658947315877,0.0016885459203024311,10,10,0,11,11,0,18,18,0,1928,1925,89,89,0 +ox-alpha,88.49353035198622,89.00997725137556,0.5164468993893365,89.73663128671126,90.03969493389509,0.3030636471838335,91.41211107760786,91.76249428658808,0.3503832089802188,93.00044543189705,93.39814820267365,0.3977027707766041,94.86340123268947,94.94265921491042,0.07925798222095182,95.98755058341617,95.99218122244062,0.004630639024455263,11,11,0,13,13,0,17,17,0,1928,1925,87,87,0 +grok-4.7,88.3352356001671,88.89900206135282,0.5637664611857218,90.8559540301881,91.41972049137382,0.5637664611857218,93.57550903955742,94.18659506253951,0.6110860229820929,94.84097272845779,95.02529224702889,0.18431951857110107,96.76800515903547,96.83372616430886,0.06572100527338876,97.36508321195915,97.36808696377673,0.0030037518175873856,12,12,0,10,9,1,7,7,0,1928,1925,92,92,0 +inkling,88.22006716270263,88.7783790203378,0.5583118576351609,90.22048592027716,90.5654145257068,0.34492860542964365,91.63940759009171,92.04733592443918,0.4079283343474742,93.53534600182131,93.72989108396328,0.1945450821419712,96.39903404563158,96.46751410286626,0.0684800572346802,96.79351511049066,96.8039513725253,0.010436262034644983,13,13,0,12,12,0,9,9,0,1928,1925,81,81,0 +gpt-5.5,88.15164989242267,88.71541635360839,0.5637664611857218,88.86576150317778,89.4295279643635,0.5637664611857218,91.5182660691513,92.14503225925485,0.6267661901035524,93.06851538766645,93.4818983255645,0.41338293789804936,95.12547300857929,95.25511312953918,0.1296401209598912,96.46404648991201,96.48422201987421,0.020175529962202177,14,14,0,15,15,0,14,13,1,1928,1925,88,88,0 +gpt-5.6-terra,87.69497703300591,88.20596932884467,0.5109922958387614,89.24375535140979,89.54136439504306,0.29760904363327256,91.33116919368196,91.67609779911162,0.34492860542965786,93.25789584167555,93.40282476141265,0.14492891973709732,95.21005308051073,95.23345324999597,0.023400169485242373,96.38139933140464,96.37966628568644,-0.0017330457181969905,15,15,0,14,14,0,13,14,-1,1928,1925,83,83,0 +deepseek-v4.1-flash,87.0552377499924,87.57168464938174,0.5164468993893365,88.63835381257343,88.98873702155365,0.3503832089802188,91.50223230715568,91.89993507793228,0.3977027707766041,92.57265838043338,92.97036115120999,0.3977027707766041,95.05626593089697,95.12537671317271,0.06911078227574308,96.58652674175693,96.58602749925093,-0.0004992425059953121,16,16,0,16,16,0,15,15,0,1928,1925,88,88,0 +grok-4.6,86.40208736627577,86.96585382746149,0.5637664611857218,87.74845427809858,88.32790090640574,0.5794466283071671,90.03659610470429,90.66336229480784,0.6267661901035524,91.51445377859353,91.71445346428608,0.19999968569254634,94.910144673076,95.00688197277564,0.09673729969964029,96.76153897191449,96.75952479388886,-0.002014178025632418,17,17,0,17,17,0,16,16,0,1928,1925,88,88,0 +gemini-3.8-flash,85.94579492214984,86.46224182153918,0.5164468993893365,86.74176419758159,87.25821109697092,0.5164468993893365,91.27675751184425,91.59550132614953,0.31874381430527876,93.17056158315293,93.5366249592546,0.36606337610166406,96.29217728031497,96.40412149584576,0.11194421553078371,97.16992672276967,97.18834204181844,0.018415319048770584,18,18,0,18,18,0,10,10,0,1928,1925,81,81,0 +grok-4.5,85.08110483198445,85.59755173137378,0.5164468993893365,86.00850651724336,86.5249534166327,0.5164468993893365,88.14557170517973,88.77233789528329,0.6267661901035524,90.07512942777474,90.27512911346729,0.19999968569254634,93.72547899612037,93.82543336108115,0.09995436496078014,95.98818070767912,95.98327882340698,-0.004901884272143775,19,19,0,19,19,0,20,20,0,1928,1925,85,85,0 +claude-opus-5,84.02988376350788,84.54087605934664,0.5109922958387614,85.00690856768351,85.51790086352227,0.5109922958387614,86.77312547350387,87.28411776934264,0.5109922958387756,89.09939384862093,89.67338587337753,0.573992024756592,93.10434478976686,93.24580317131411,0.14145838154725254,95.47065392942848,95.49057382466182,0.019919895233343254,20,20,0,20,20,0,24,23,1,1928,1925,85,85,0 +claude-fable-5,83.6253944438549,84.13638673969366,0.5109922958387614,84.0419404286506,84.55293272448937,0.5109922958387614,86.7052616423924,87.26357350002756,0.5583118576351609,88.92460130189772,89.31684946912377,0.39224816722604317,92.87276020256175,92.93410509695849,0.061344894396739846,95.45713913114773,95.45433567003552,-0.0028034611122080833,21,21,0,23,23,0,26,26,0,1928,1925,82,82,0 +deepseek-v4-pro-0813,83.37167580453175,83.88812270392108,0.5164468993893365,84.82718523955461,85.35931230606539,0.5321270665107818,87.42991217590283,87.84329511380088,0.41338293789804936,88.68234431644125,89.0957272543393,0.41338293789804936,93.52207817631968,93.62186976023973,0.09979158392005161,96.2099699376765,96.19919277500318,-0.010777162673321072,22,22,0,21,21,0,21,21,0,1928,1925,79,79,0 +deepseek-v4-flash-0731,83.2758619160859,83.78685421192468,0.5109922958387756,84.18326968295833,84.69426197879709,0.5109922958387614,88.38514671885079,88.68275576248406,0.29760904363327256,89.46053404560782,89.80546265103746,0.34492860542964365,93.13916790103261,93.21799737712904,0.07882947609643054,94.92567743190693,94.92720162977261,0.001524197865677479,23,23,0,22,22,0,23,24,-1,1928,1925,79,79,0 +gemini-3.7-flash,83.11782882466474,83.63427572405408,0.5164468993893365,83.63691536248353,84.15336226187287,0.5164468993893365,86.61828353374163,87.1977301620488,0.5794466283071671,88.37484809941563,88.78823103731368,0.41338293789804936,93.49128429689975,93.58208064897043,0.09079635207068293,96.02906927657445,96.02186358030215,-0.007205696272293949,24,24,0,25,25,0,22,22,0,1928,1925,71,71,0 +gemini-3.6-flash,83.09499152818556,83.6114384275749,0.5164468993893365,83.77568744351451,84.29213434290385,0.5164468993893365,87.90786893770674,88.2739323138084,0.36606337610166406,89.93339257104228,90.34677550894033,0.41338293789804936,95.2495232591938,95.32055269959673,0.07102944040293835,96.81412429532105,96.81374451621733,-0.00037977910372433143,25,25,0,24,24,0,12,12,0,1928,1925,70,70,0 +gemini-3.5-flash-lite,81.9592849674438,82.47573186683313,0.5164468993893223,82.84338141389416,83.40714787507989,0.5637664611857218,84.18154889958008,84.7453153607658,0.5637664611857218,85.9229953561171,86.48676181730282,0.5637664611857218,90.03121813140012,90.10722551090474,0.07600737950461678,92.99058422606034,92.97351203496137,-0.017072191098961298,26,26,0,27,26,1,32,32,0,1928,1925,78,78,0 +gemini-3.1-pro-preview,81.63648613293833,82.15293303232767,0.5164468993893365,82.21503833486231,82.73148523425164,0.5164468993893223,84.53467776508941,84.88506097406963,0.3503832089802188,86.46018765972694,86.66018734541949,0.19999968569254634,92.95579714889485,93.05098119993194,0.09518405103709426,95.53168217701608,95.52190116745766,-0.009781009558423648,27,27,0,29,29,0,25,25,0,1928,1925,66,66,0 +glm-5.3,81.09010878646016,81.6065556858495,0.5164468993893365,82.887622207975,83.19068585515883,0.3030636471838335,85.34735655714336,85.65042020432719,0.3030636471838335,86.74625194707212,87.09663515605234,0.3503832089802188,89.61270934577402,89.79913811903863,0.18642877326460905,91.60488818087755,91.63377955359329,0.02889137271573361,28,28,0,26,27,-1,34,34,0,1928,1925,81,81,0 +claude-opus-4.7,80.9638962030392,81.48034310242853,0.5164468993893365,81.70908369740485,82.22553059679419,0.5164468993893365,84.00944767428736,84.57321413547308,0.5637664611857218,85.62462327242545,86.18838973361117,0.5637664611857218,89.90261973339878,90.07301887909907,0.17039914570028714,92.94155966652754,92.97624792044954,0.034688253921999035,29,29,0,30,30,0,33,33,0,1928,1925,83,83,0 +gemini-3-flash-preview,80.95099070665046,81.46198300248922,0.5109922958387614,81.61251554501574,82.12350784085451,0.5109922958387756,84.75197587267989,85.35760729211142,0.605631419431532,86.62149249317326,87.24280407972626,0.6213115865529915,92.48205252928467,92.57686469893929,0.0948121696546167,95.89663313888457,95.88728363804293,-0.009349500841636882,30,30,0,31,31,0,27,27,0,1928,1925,65,65,0 +grok-4.3,80.62160266220387,81.13259495804265,0.5109922958387756,82.53905982363676,83.05005211947554,0.5109922958387756,83.55781152859795,83.86880413874418,0.31099261014622925,85.89631293707023,85.99392229501095,0.09760935794071202,89.13971012353002,89.17131673925229,0.03160661572226786,93.6220341272469,93.59652582567767,-0.025508301569232117,31,31,0,28,28,0,36,36,0,1928,1925,77,77,0 +grok-build-0.1,80.36084468856437,80.86761629047841,0.5067716019140391,81.44544593251806,81.9522175344321,0.5067716019140391,83.45594821451222,84.01003937822264,0.5540911637104244,85.21073113533913,85.81214186084594,0.6014107255068097,91.1324834497806,91.25170127374535,0.11921782396476033,94.36565089664984,94.35643149634075,-0.009219400309092407,32,32,0,34,34,0,30,30,0,1928,1925,71,71,0 +claude-sonnet-4.6,80.27617496987095,80.78716726570971,0.5109922958387614,81.52873822877525,82.03973052461401,0.5109922958387614,86.84827007162907,87.4539014910606,0.605631419431532,88.42639688767113,88.61864536920461,0.19224848153348262,90.93225103176982,90.98708283257356,0.05483180080373984,94.43492800125435,94.43506195597807,0.0001339547237222405,33,33,0,32,32,0,31,31,0,1928,1925,83,83,0 +gemini-3.5-flash,80.04238564541743,80.55883254480676,0.5164468993893365,80.65165534394116,81.16810224333048,0.5164468993893223,82.65249080337206,83.21625726455777,0.5637664611857076,85.23510538068777,85.86187157079132,0.6267661901035524,91.9574985410053,92.07228767657818,0.11478913557289161,95.30733194933845,95.29947401779633,-0.007857931542119445,34,34,0,36,36,0,28,28,0,1928,1925,61,61,0 +deepseek-v4-pro,79.75124065440951,80.26223295024828,0.5109922958387614,80.91779413249449,81.47610599012964,0.5583118576351467,85.4731858125084,86.09449739906138,0.6213115865529772,87.93991141377664,88.56122300032963,0.6213115865529915,91.625686267379,91.7086754644478,0.08298919706879815,94.88443394488235,94.87130101083795,-0.013132934044392641,35,35,0,35,35,0,29,29,0,1928,1925,68,68,0 +gemini-3.1-flash-lite-preview,79.73940156545854,80.25584846484787,0.5164468993893365,81.48438903762849,82.00083593701783,0.5164468993893365,83.00632269033785,83.30938633752167,0.3030636471838193,84.74813485700159,85.05119850418542,0.3030636471838335,89.14415485236766,89.22491184785987,0.08075699549220872,93.26084731439737,93.231555966066,-0.029291348331369704,36,36,0,33,33,0,35,35,0,1928,1925,76,76,0 +qwen-3.7-max,77.38613945075842,77.89713174659718,0.5109922958387614,78.79437682154989,79.30536911738866,0.5109922958387756,81.27196352984511,81.87759494927666,0.6056314194315462,83.17596242907331,83.56821059629935,0.39224816722604317,87.33527967112641,87.53257387229348,0.1972942011670682,91.48372111324807,91.49129327238187,0.0075721591338009375,37,37,0,37,37,0,37,37,0,1928,1925,73,73,0 +glm-5.2,77.04754617804436,77.4682400104827,0.4206938324383458,78.40294653618636,78.87095993042108,0.4680133942347169,82.5432767994509,83.05860975548201,0.5153329560311164,84.06122691696918,84.36317662079479,0.30194970382561337,86.86613898600048,86.86436654742563,-0.0017724385748465465,88.41384713439248,88.41147135632066,-0.002375778071822765,38,38,0,38,38,0,38,38,0,1928,1925,73,73,0 +claude-opus-4.8,76.17545674851087,76.69190364790019,0.5164468993893223,77.25613559407958,77.77258249346892,0.5164468993893365,78.75534238142073,79.07178959511752,0.3164472136967902,81.00663384747759,81.32308106117438,0.3164472136967902,84.91925054467029,85.02995146614548,0.11070092147518551,91.72770115914611,91.73059547308952,0.0028943139434147724,39,39,0,40,39,1,39,39,0,1928,1925,81,81,0 +minimax-m3,75.74649533851503,76.22206780769882,0.47557246918378837,77.2712752290567,77.74684769824049,0.47557246918378837,79.9814357124164,80.49788261180574,0.5164468993893365,80.53641549589192,81.05286239528125,0.5164468993893365,82.10743147259882,82.66339739966166,0.555965927062843,90.96878161865847,91.10339700696609,0.13461538830762265,40,40,0,39,40,-1,43,42,1,1928,1925,63,63,0 +qwen3.8-max,75.22482335536425,75.73581565120303,0.5109922958387756,75.83105699989459,76.34204929573335,0.5109922958387614,77.57329061246871,78.13160247010387,0.5583118576351609,78.73445137394236,79.29276323157752,0.5583118576351609,82.95047048737395,83.27224694624788,0.32177645887392714,89.7743259143331,89.81424379049652,0.03991787616341469,41,41,0,42,42,0,41,41,0,1928,1925,71,71,0 +claude-haiku-4.5,74.6603649485753,75.17135724441408,0.5109922958387756,75.97216713709038,76.48315943292914,0.5109922958387614,77.83789532532154,77.93550468326225,0.09760935794071202,79.78305248301713,79.92798140275424,0.14492891973711153,83.64631494260415,83.65101496555216,0.004700022948014748,89.75967813092348,89.71726861613516,-0.04240951478831789,42,42,0,41,41,0,40,40,0,1928,1925,65,65,0 +gpt-5.4-mini,73.66669485735765,74.13681272299088,0.47011786563322744,75.63642913518649,76.10654700081972,0.47011786563322744,75.92267879361472,76.39279665924795,0.47011786563322744,77.68933140410081,78.20676883153043,0.5174374274296127,81.29939112383417,81.67682653034329,0.37743540650912166,88.88801014329086,88.96879391106151,0.08078376777065444,43,43,0,43,43,0,44,44,0,1928,1925,54,54,0 +claude-sonnet-5,72.64249589279979,73.15348818863855,0.5109922958387614,74.6630018321248,75.17399412796357,0.5109922958387614,75.83771649556213,76.34870879140091,0.5109922958387756,77.42378744351629,77.93477973935505,0.5109922958387614,82.49049611863575,82.63641365513257,0.1459175364968246,91.63640300577666,91.64434375730706,0.007940751530398416,44,44,0,44,44,0,42,43,-1,1928,1925,76,76,0 +kimi-k2.6,67.40417281119927,67.82400400344002,0.41983119224074983,67.42572172627803,67.84555291851878,0.41983119224074983,68.53579711586619,68.95562830810694,0.41983119224074983,68.99989280107147,69.41972399331222,0.41983119224074983,69.45103956892189,69.87087076116264,0.41983119224074983,77.0569946963265,77.16794591473611,0.11095121840961042,45,45,0,45,45,0,46,46,0,1928,1925,46,46,0 +gpt-5.4-nano,65.33891134939903,65.81448381858281,0.47557246918378837,66.37027185509079,66.8458443242746,0.4755724691838026,68.91289387798616,69.38846634716995,0.47557246918378837,70.82365628740529,71.29922875658909,0.4755724691838026,75.66934446405278,76.10932834773762,0.43998388368483177,88.25864207573021,88.35205513270202,0.09341305697181213,46,46,0,46,46,0,45,45,0,1928,1925,17,17,0 diff --git a/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_programs.csv b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_programs.csv new file mode 100644 index 00000000..02ad94b7 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/leaderboard_impact_salt_programs.csv @@ -0,0 +1,2 @@ +variable,exact_published,exact_proposed,exact_delta,score_published,score_proposed,score_delta,n_published,n_proposed,n_delta +federal_income_tax_before_refundable_credits,61.6118769883351,63.9515685195377,2.339691531202604,82.70114441901764,82.91135864963636,0.21021423061871758,3772,3634,-138 diff --git a/reference_audit/2026-10-05-payroll/verification/model_answers.csv b/reference_audit/2026-10-05-payroll/verification/model_answers.csv new file mode 100644 index 00000000..49857e8d --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/model_answers.csv @@ -0,0 +1,553 @@ +scenario_id,state,model,prediction,scored,exact_published,within1pct_published,exact_no_optional,within1pct_no_optional,exact_federal_only,within1pct_federal_only +scenario_005,CA,claude-fable-5,33296.55,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-fable-5.1,30454.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_005,CA,claude-haiku-4.5,41859.6,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-opus-4.7,35446.6,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-opus-4.8,28586.6,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-opus-5,30437.0,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_005,CA,claude-opus-5.5,36044.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_005,CA,claude-sonnet-4.6,34851.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-sonnet-5,27306.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,claude-sonnet-5.5,36044.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_005,CA,deepseek-v4-flash-0731,19108.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,deepseek-v4-pro,34342.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,deepseek-v4-pro-0813,33962.54,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,deepseek-v4.1-flash,35595.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3-flash-preview,33827.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.1-flash-lite-preview,31398.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.1-pro-preview,34258.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.5-flash,33700.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.5-flash-lite,26588.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.6-flash,34091.05,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.7-flash,33814.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gemini-3.8-flash,29691.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,glm-5.2,34274.42,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,glm-5.3,30454.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_005,CA,gpt-5.4-mini,22295.1,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-5.4-nano,13905.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-5.5,35558.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-5.6-luna,36667.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-5.6-sol,35614.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-5.6-terra,35614.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-6-astra,35128.91,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-6-luna,35614.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,gpt-6-sol,36044.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_005,CA,gpt-6.1-sol,35128.91,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,grok-4.3,34515.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,grok-4.5,34525.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,grok-4.6,34704.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,grok-4.7,36044.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_005,CA,grok-build-0.1,31456.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,inkling,32773.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,kimi-k3,35614.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,minimax-m3,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,ox-alpha,31084.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,qwen-3.7-max,33061.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_005,CA,qwen3.8-max,25474.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,claude-fable-5,2178.85,False,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_008,NJ,claude-fable-5.1,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-haiku-4.5,2052.12,False,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_008,NJ,claude-opus-4.7,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-opus-4.8,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-opus-5,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-opus-5.5,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-sonnet-4.6,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-sonnet-5,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,claude-sonnet-5.5,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,deepseek-v4-flash-0731,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,deepseek-v4-pro,2101.12,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,deepseek-v4-pro-0813,2204.3,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,deepseek-v4.1-flash,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,gemini-3-flash-preview,2188.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gemini-3.1-flash-lite-preview,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,gemini-3.1-pro-preview,2188.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gemini-3.5-flash,2174.0,False,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_008,NJ,gemini-3.5-flash-lite,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,gemini-3.6-flash,2188.22,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gemini-3.7-flash,2189.52,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gemini-3.8-flash,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,glm-5.2,,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,glm-5.3,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,gpt-5.4-mini,4097.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-5.4-nano,1525.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-5.5,2302.79,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-5.6-luna,2303.46,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-5.6-sol,2314.18,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-5.6-terra,2252.54,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-6-astra,2276.66,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-6-luna,2314.18,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-6-sol,2276.66,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,gpt-6.1-sol,2265.27,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,grok-4.3,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,grok-4.5,2098.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,grok-4.6,2228.0,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,grok-4.7,2314.18,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,grok-build-0.1,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,inkling,2177.0,False,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_008,NJ,kimi-k2.6,,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,kimi-k3,2308.15,False,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_008,NJ,minimax-m3,2050.0,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,ox-alpha,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_008,NJ,qwen-3.7-max,2152.71,False,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_008,NJ,qwen3.8-max,2050.2,False,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,claude-fable-5,4557.61,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,claude-fable-5.1,4026.58,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,claude-haiku-4.5,4028.27,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_022,CA,claude-opus-4.7,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,claude-opus-4.8,4026.58,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,claude-opus-5,4026.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,claude-opus-5.5,4711.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,claude-sonnet-4.6,4605.57,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,claude-sonnet-5,3264.37,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,claude-sonnet-5.5,4710.84,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,deepseek-v4-flash-0731,4026.61,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,deepseek-v4-pro,4605.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,deepseek-v4-pro-0813,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,deepseek-v4.1-flash,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3-flash-preview,4605.57,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3.1-flash-lite-preview,4027.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,gemini-3.1-pro-preview,4606.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3.5-flash,4606.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3.5-flash-lite,4027.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,gemini-3.6-flash,4500.3,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3.7-flash,4500.29,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gemini-3.8-flash,4026.58,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,glm-5.2,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,glm-5.3,4026.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,gpt-5.4-mini,4030.0,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_022,CA,gpt-5.4-nano,3308.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gpt-5.5,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gpt-5.6-luna,4710.84,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,gpt-5.6-sol,4711.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,gpt-5.6-terra,4027.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,gpt-6-astra,4710.83,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,gpt-6-luna,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,gpt-6-sol,4711.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,gpt-6.1-sol,4710.83,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_022,CA,grok-4.3,4027.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,grok-4.5,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,grok-4.6,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,grok-4.7,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,grok-build-0.1,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,inkling,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,kimi-k3,4658.2,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,minimax-m3,4026.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_022,CA,ox-alpha,4658.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,qwen-3.7-max,4605.57,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_022,CA,qwen3.8-max,4026.58,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,claude-fable-5,1531.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,claude-fable-5.1,1334.39,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,claude-haiku-4.5,1335.85,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_023,CA,claude-opus-4.7,1543.71,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,claude-opus-4.8,1467.32,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,claude-opus-5,1546.0,True,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_023,CA,claude-opus-5.5,1561.15,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_023,CA,claude-sonnet-4.6,1526.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,claude-sonnet-5,1334.39,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,claude-sonnet-5.5,1334.39,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,deepseek-v4-flash-0731,716.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,deepseek-v4-pro,827.83,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,deepseek-v4-pro-0813,1526.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,deepseek-v4.1-flash,1543.71,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3-flash-preview,1526.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.1-flash-lite-preview,1334.44,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,gemini-3.1-pro-preview,818.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.5-flash,818.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.5-flash-lite,1595.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.6-flash,1526.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.7-flash,809.12,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gemini-3.8-flash,1526.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,glm-5.2,1543.71,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,glm-5.3,1329.0,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_023,CA,gpt-5.4-mini,1375.966,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gpt-5.4-nano,1208.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gpt-5.5,1543.71,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gpt-5.6-luna,1561.15,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_023,CA,gpt-5.6-sol,1561.15,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_023,CA,gpt-5.6-terra,1334.39,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,gpt-6-astra,1561.15,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_023,CA,gpt-6-luna,1544.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,gpt-6-sol,1561.15,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_023,CA,gpt-6.1-sol,837.18,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,grok-4.3,1334.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,grok-4.5,1544.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,grok-4.6,1544.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,grok-4.7,1543.71,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,grok-build-0.1,1283.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,inkling,828.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,kimi-k3,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,minimax-m3,1334.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,ox-alpha,1544.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_023,CA,qwen-3.7-max,1334.39,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_023,CA,qwen3.8-max,261.65,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_028,PA,claude-fable-5,5730.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_028,PA,claude-fable-5.1,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-haiku-4.5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-opus-4.7,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,claude-opus-4.8,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-opus-5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-opus-5.5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-sonnet-4.6,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-sonnet-5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,claude-sonnet-5.5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,deepseek-v4-flash-0731,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,deepseek-v4-pro,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,deepseek-v4-pro-0813,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,deepseek-v4.1-flash,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gemini-3-flash-preview,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gemini-3.1-flash-lite-preview,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gemini-3.1-pro-preview,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gemini-3.5-flash,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gemini-3.5-flash-lite,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gemini-3.6-flash,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gemini-3.7-flash,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gemini-3.8-flash,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,glm-5.2,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_028,PA,glm-5.3,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gpt-5.4-mini,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gpt-5.4-nano,4600.0,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_028,PA,gpt-5.5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gpt-5.6-luna,4597.0,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_028,PA,gpt-5.6-sol,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gpt-5.6-terra,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gpt-6-astra,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gpt-6-luna,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,gpt-6-sol,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,gpt-6.1-sol,4632.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_028,PA,grok-4.3,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,grok-4.5,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,grok-4.6,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,grok-4.7,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,grok-build-0.1,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,inkling,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_028,PA,kimi-k3,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,minimax-m3,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,ox-alpha,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,qwen-3.7-max,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_028,PA,qwen3.8-max,4590.0,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_032,MN,claude-fable-5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-fable-5.1,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-haiku-4.5,2175.9,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,claude-opus-4.7,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-opus-4.8,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-opus-5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-opus-5.5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-sonnet-4.6,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-sonnet-5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,claude-sonnet-5.5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,deepseek-v4-flash-0731,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,deepseek-v4-pro,2320.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,deepseek-v4-pro-0813,2320.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,deepseek-v4.1-flash,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gemini-3-flash-preview,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.1-flash-lite-preview,2219.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.1-pro-preview,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.5-flash,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.5-flash-lite,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.6-flash,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.7-flash,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gemini-3.8-flash,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,glm-5.2,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,glm-5.3,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gpt-5.4-mini,4428.6,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gpt-5.4-nano,1520.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gpt-5.5,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gpt-5.6-luna,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gpt-5.6-sol,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gpt-5.6-terra,2219.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gpt-6-astra,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,gpt-6-luna,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gpt-6-sol,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,gpt-6.1-sol,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,grok-4.3,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,grok-4.5,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,grok-4.6,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,grok-4.7,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,grok-build-0.1,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,inkling,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_032,MN,kimi-k3,2346.1,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_032,MN,minimax-m3,2219.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,ox-alpha,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,qwen-3.7-max,2219.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_032,MN,qwen3.8-max,2218.5,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-fable-5,315.14,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_043,CO,claude-fable-5.1,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-haiku-4.5,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-opus-4.7,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-opus-4.8,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-opus-5,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-opus-5.5,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-sonnet-4.6,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-sonnet-5,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,claude-sonnet-5.5,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,deepseek-v4-flash-0731,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,deepseek-v4-pro,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,deepseek-v4-pro-0813,337.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_043,CO,deepseek-v4.1-flash,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gemini-3-flash-preview,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gemini-3.1-flash-lite-preview,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gemini-3.1-pro-preview,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gemini-3.5-flash,312.74,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gemini-3.5-flash-lite,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gemini-3.6-flash,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gemini-3.7-flash,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gemini-3.8-flash,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,glm-5.2,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,glm-5.3,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gpt-5.4-mini,312.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gpt-5.4-nano,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-5.5,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gpt-5.6-luna,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-5.6-sol,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-5.6-terra,312.732,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,gpt-6-astra,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-6-luna,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-6-sol,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,gpt-6.1-sol,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,grok-4.3,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,grok-4.5,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,grok-4.6,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,grok-4.7,330.72,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,grok-build-0.1,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,inkling,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,kimi-k2.6,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,kimi-k3,331.13,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_043,CO,minimax-m3,313.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,ox-alpha,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,qwen-3.7-max,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_043,CO,qwen3.8-max,312.73,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,claude-fable-5,13268.7,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,claude-fable-5.1,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,claude-haiku-4.5,13383.41,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,claude-opus-4.7,12321.73,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,claude-opus-4.8,12717.62,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,claude-opus-5,13385.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,claude-opus-5.5,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,claude-sonnet-4.6,14444.66,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,claude-sonnet-5,12318.65,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,claude-sonnet-5.5,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,deepseek-v4-flash-0731,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,deepseek-v4-pro,14053.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,deepseek-v4-pro-0813,13593.41,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,deepseek-v4.1-flash,14157.66,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3-flash-preview,13944.16,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3.1-flash-lite-preview,11843.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3.1-pro-preview,13402.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,gemini-3.5-flash,13224.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3.5-flash-lite,14133.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3.6-flash,12822.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gemini-3.7-flash,13444.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,gemini-3.8-flash,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,glm-5.2,14319.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,glm-5.3,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,gpt-5.4-mini,13389.37,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_081,MA,gpt-5.4-nano,5420.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-5.5,14192.66,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-5.6-luna,14193.0,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-5.6-sol,14192.66,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-5.6-terra,14350.16,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-6-astra,14192.66,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-6-luna,13388.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,gpt-6-sol,14192.6622,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,gpt-6.1-sol,14192.66,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,grok-4.3,15926.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,grok-4.5,13944.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,grok-4.6,14193.0,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,grok-4.7,14192.66,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,grok-build-0.1,13388.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,inkling,14193.0,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_081,MA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,kimi-k3,13835.66,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,minimax-m3,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,ox-alpha,13387.65,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_081,MA,qwen-3.7-max,14525.24,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_081,MA,qwen3.8-max,13922.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,claude-fable-5,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,claude-fable-5.1,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,claude-haiku-4.5,8304.92,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,claude-opus-4.7,7745.43,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,claude-opus-4.8,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,claude-opus-5,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,claude-opus-5.5,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,claude-sonnet-4.6,8042.77,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,claude-sonnet-5,7794.31,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,claude-sonnet-5.5,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,deepseek-v4-flash-0731,7651.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,deepseek-v4-pro,7696.12,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,deepseek-v4-pro-0813,8034.6,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,deepseek-v4.1-flash,8128.96,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3-flash-preview,8029.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3.1-flash-lite-preview,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,gemini-3.1-pro-preview,6368.33,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3.5-flash,6436.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3.5-flash-lite,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,gemini-3.6-flash,6313.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3.7-flash,6376.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gemini-3.8-flash,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,glm-5.2,8069.38,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,glm-5.3,7765.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gpt-5.4-mini,7660.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,gpt-5.4-nano,7659.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,gpt-5.5,8050.65,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gpt-5.6-luna,7696.12,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,gpt-5.6-sol,7664.92,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,gpt-5.6-terra,7654.0,True,0.0,0.0,0.0,1.0,0.0,1.0 +scenario_082,NY,gpt-6-astra,8128.96,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gpt-6-luna,8108.03,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gpt-6-sol,8108.03,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,gpt-6.1-sol,8108.03,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,grok-4.3,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,grok-4.5,6042.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,grok-4.6,8032.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,grok-4.7,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,grok-build-0.1,6029.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,inkling,8061.0,True,0.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,kimi-k3,8107.98,True,1.0,1.0,0.0,0.0,0.0,0.0 +scenario_082,NY,minimax-m3,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,ox-alpha,7665.0,True,0.0,0.0,1.0,1.0,1.0,1.0 +scenario_082,NY,qwen-3.7-max,7464.33,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_082,NY,qwen3.8-max,7752.42,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,claude-fable-5,14118.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,claude-fable-5.1,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,claude-haiku-4.5,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,claude-opus-4.7,13197.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,claude-opus-4.8,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,claude-opus-5,12622.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,claude-opus-5.5,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,claude-sonnet-4.6,14437.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,claude-sonnet-5,12903.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,claude-sonnet-5.5,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,deepseek-v4-flash-0731,14438.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,deepseek-v4-pro,12285.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,deepseek-v4-pro-0813,12285.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,deepseek-v4.1-flash,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3-flash-preview,14437.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.1-flash-lite-preview,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,gemini-3.1-pro-preview,12285.26,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.5-flash,14437.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.5-flash-lite,12963.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.6-flash,13280.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.7-flash,12425.67,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gemini-3.8-flash,14103.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,glm-5.2,12632.4,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_099,CA,glm-5.3,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-5.4-mini,12622.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,gpt-5.4-nano,13577.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-5.5,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-5.6-luna,14312.4,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-5.6-sol,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-5.6-terra,14603.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,gpt-6-astra,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,gpt-6-luna,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,gpt-6-sol,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,gpt-6.1-sol,14767.5,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_099,CA,grok-4.3,12623.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,grok-4.5,14438.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,grok-4.6,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,grok-4.7,14602.5,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,grok-build-0.1,10741.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,inkling,14603.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,kimi-k3,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,minimax-m3,25290.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_099,CA,ox-alpha,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,qwen-3.7-max,12622.5,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_099,CA,qwen3.8-max,13094.7,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,claude-fable-5,13736.63,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,claude-fable-5.1,12668.17,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,claude-haiku-4.5,12619.41,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_120,CT,claude-opus-4.7,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,claude-opus-4.8,12668.17,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,claude-opus-5,10680.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,claude-opus-5.5,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,claude-sonnet-4.6,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,claude-sonnet-5,12668.17,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,claude-sonnet-5.5,12668.17,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,deepseek-v4-flash-0731,12054.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,deepseek-v4-pro,12891.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,deepseek-v4-pro-0813,13496.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,deepseek-v4.1-flash,12668.17,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,gemini-3-flash-preview,13496.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,gemini-3.1-flash-lite-preview,14197.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.1-pro-preview,12891.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.5-flash,12890.7,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.5-flash-lite,14352.19,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.6-flash,12890.7,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.7-flash,12891.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gemini-3.8-flash,12891.1,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,glm-5.2,12044.31,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,glm-5.3,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,gpt-5.4-mini,10087.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gpt-5.4-nano,6180.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gpt-5.5,13496.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,gpt-5.6-luna,13590.67,True,0.0,1.0,0.0,1.0,0.0,0.0 +scenario_120,CT,gpt-5.6-sol,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,gpt-5.6-terra,12621.0,True,0.0,0.0,0.0,0.0,0.0,1.0 +scenario_120,CT,gpt-6-astra,12890.69,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,gpt-6-luna,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,gpt-6-sol,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,gpt-6.1-sol,12890.69,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,grok-4.3,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,grok-4.5,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,grok-4.6,13496.0,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,grok-4.7,13496.16,True,1.0,1.0,1.0,1.0,0.0,0.0 +scenario_120,CT,grok-build-0.1,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,inkling,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,kimi-k3,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,minimax-m3,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_120,CT,ox-alpha,12668.0,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,qwen-3.7-max,12668.18,True,0.0,0.0,0.0,0.0,1.0,1.0 +scenario_120,CT,qwen3.8-max,9655.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,claude-fable-5,11663.35,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,claude-fable-5.1,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,claude-haiku-4.5,11741.39,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,claude-opus-4.7,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,claude-opus-4.8,11097.5,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,claude-opus-5,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,claude-opus-5.5,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,claude-sonnet-4.6,11106.5,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,claude-sonnet-5,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,claude-sonnet-5.5,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,deepseek-v4-flash-0731,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,deepseek-v4-pro,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,deepseek-v4-pro-0813,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,deepseek-v4.1-flash,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,gemini-3-flash-preview,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.1-flash-lite-preview,11108.0,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.1-pro-preview,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.5-flash,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.5-flash-lite,11138.0,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.6-flash,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.7-flash,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gemini-3.8-flash,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,glm-5.2,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,glm-5.3,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,gpt-5.4-mini,12366.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,gpt-5.4-nano,15399.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,gpt-5.5,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gpt-5.6-luna,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gpt-5.6-sol,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gpt-5.6-terra,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,gpt-6-astra,11179.5,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,gpt-6-luna,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,gpt-6-sol,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,gpt-6.1-sol,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,grok-4.3,11107.5,True,0.0,1.0,0.0,1.0,0.0,1.0 +scenario_123,PA,grok-4.5,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,grok-4.6,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,grok-4.7,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,grok-build-0.1,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,inkling,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,kimi-k2.6,,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,kimi-k3,11194.0,True,1.0,1.0,1.0,1.0,0.0,1.0 +scenario_123,PA,minimax-m3,0.0,True,0.0,0.0,0.0,0.0,0.0,0.0 +scenario_123,PA,ox-alpha,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,qwen-3.7-max,11092.5,True,0.0,1.0,0.0,1.0,1.0,1.0 +scenario_123,PA,qwen3.8-max,10851.95,True,0.0,0.0,0.0,0.0,0.0,0.0 diff --git a/reference_audit/2026-10-05-payroll/verification/model_answers_summary.json b/reference_audit/2026-10-05-payroll/verification/model_answers_summary.json new file mode 100644 index 00000000..15929429 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/model_answers_summary.json @@ -0,0 +1,280 @@ +{ + "outputs": [ + { + "scenario_id": "scenario_005", + "state": "CA", + "scored": true, + "state_programs": "ca_employee_state_disability_insurance_contribution=5590.00", + "optional_amount": 0, + "reference_published": 36044.0, + "reference_no_optional": 36044.0, + "reference_federal_only": 30454.0, + "models": 46, + "exact_published": 4, + "exact_no_optional": 4, + "exact_federal_only": 2, + "within1pct_published": 4, + "within1pct_no_optional": 4, + "within1pct_federal_only": 3, + "modal_answers": { + "36044.00": 4, + "35614.00": 4, + "30454.00": 2, + "35128.91": 2 + } + }, + { + "scenario_id": "scenario_008", + "state": "NJ", + "scored": false, + "state_programs": "nj_employee_family_leave_insurance_contribution=61.64;nj_employee_temporary_disability_insurance_contribution=50.92", + "optional_amount": 0, + "reference_published": 2162.76, + "reference_no_optional": 2162.76, + "reference_federal_only": 2050.2, + "models": 46, + "exact_published": 0, + "exact_no_optional": 0, + "exact_federal_only": 19, + "within1pct_published": 4, + "within1pct_no_optional": 4, + "within1pct_federal_only": 20, + "modal_answers": { + "2050.20": 13, + "2050.00": 6, + "2314.18": 3, + "2188.00": 2 + } + }, + { + "scenario_id": "scenario_022", + "state": "CA", + "scored": true, + "state_programs": "ca_employee_state_disability_insurance_contribution=684.26", + "optional_amount": 0, + "reference_published": 4710.83, + "reference_no_optional": 4710.83, + "reference_federal_only": 4026.58, + "models": 46, + "exact_published": 7, + "exact_no_optional": 7, + "exact_federal_only": 12, + "within1pct_published": 7, + "within1pct_no_optional": 7, + "within1pct_federal_only": 14, + "modal_answers": { + "4658.20": 7, + "4658.00": 6, + "4026.58": 4, + "4027.00": 4 + } + }, + { + "scenario_id": "scenario_023", + "state": "CA", + "scored": true, + "state_programs": "ca_employee_state_disability_insurance_contribution=226.75", + "optional_amount": 0, + "reference_published": 1561.12, + "reference_no_optional": 1561.12, + "reference_federal_only": 1334.36, + "models": 46, + "exact_published": 5, + "exact_no_optional": 5, + "exact_federal_only": 8, + "within1pct_published": 6, + "within1pct_no_optional": 6, + "within1pct_federal_only": 10, + "modal_answers": { + "1334.39": 5, + "1543.71": 5, + "1561.15": 5, + "1526.26": 5 + } + }, + { + "scenario_id": "scenario_028", + "state": "PA", + "scored": true, + "state_programs": "pa_employee_unemployment_compensation_contribution=42.00", + "optional_amount": 0, + "reference_published": 4632.0, + "reference_no_optional": 4632.0, + "reference_federal_only": 4590.0, + "models": 46, + "exact_published": 10, + "exact_no_optional": 10, + "exact_federal_only": 31, + "within1pct_published": 43, + "within1pct_no_optional": 43, + "within1pct_federal_only": 43, + "modal_answers": { + "4590.00": 31, + "4632.00": 10, + "5730.00": 1, + "4600.00": 1 + } + }, + { + "scenario_id": "scenario_032", + "state": "MN", + "scored": true, + "state_programs": "mn_employee_paid_leave_contribution=127.60", + "optional_amount": 127.6, + "reference_published": 2346.1, + "reference_no_optional": 2218.5, + "reference_federal_only": 2218.5, + "models": 46, + "exact_published": 6, + "exact_no_optional": 33, + "exact_federal_only": 33, + "within1pct_published": 6, + "within1pct_no_optional": 33, + "within1pct_federal_only": 33, + "modal_answers": { + "2218.50": 29, + "2346.10": 6, + "2219.00": 4, + "2320.00": 2 + } + }, + { + "scenario_id": "scenario_043", + "state": "CO", + "scored": true, + "state_programs": "co_employee_famli_contribution=17.99", + "optional_amount": 17.99, + "reference_published": 330.73, + "reference_no_optional": 312.74, + "reference_federal_only": 312.74, + "models": 46, + "exact_published": 16, + "exact_no_optional": 26, + "exact_federal_only": 26, + "within1pct_published": 16, + "within1pct_no_optional": 27, + "within1pct_federal_only": 27, + "modal_answers": { + "312.73": 18, + "331.13": 9, + "330.72": 7, + "313.00": 6 + } + }, + { + "scenario_id": "scenario_081", + "state": "MA", + "scored": true, + "state_programs": "ma_employee_paid_leave_contribution=805.01", + "optional_amount": 805.01, + "reference_published": 14192.66, + "reference_no_optional": 13387.65, + "reference_federal_only": 13387.65, + "models": 46, + "exact_published": 9, + "exact_no_optional": 9, + "exact_federal_only": 9, + "within1pct_published": 13, + "within1pct_no_optional": 15, + "within1pct_federal_only": 15, + "modal_answers": { + "13387.65": 7, + "14192.66": 6, + "14193.00": 3, + "13388.00": 2 + } + }, + { + "scenario_id": "scenario_082", + "state": "NY", + "scored": true, + "state_programs": "ny_employee_disability_benefits_contribution=31.20;ny_employee_paid_family_leave_contribution=411.91", + "optional_amount": 443.11, + "reference_published": 8108.03, + "reference_no_optional": 7664.92, + "reference_federal_only": 7664.92, + "models": 46, + "exact_published": 4, + "exact_no_optional": 14, + "exact_federal_only": 14, + "within1pct_published": 13, + "within1pct_no_optional": 20, + "within1pct_federal_only": 20, + "modal_answers": { + "7664.92": 8, + "7665.00": 6, + "8108.03": 3, + "7696.12": 2 + } + }, + { + "scenario_id": "scenario_099", + "state": "CA", + "scored": true, + "state_programs": "ca_employee_state_disability_insurance_contribution=2145.00", + "optional_amount": 0, + "reference_published": 14767.5, + "reference_no_optional": 14767.5, + "reference_federal_only": 12622.5, + "models": 46, + "exact_published": 6, + "exact_no_optional": 6, + "exact_federal_only": 9, + "within1pct_published": 6, + "within1pct_no_optional": 6, + "within1pct_federal_only": 10, + "modal_answers": { + "12622.50": 6, + "14767.50": 6, + "14602.50": 6, + "14437.50": 3 + } + }, + { + "scenario_id": "scenario_120", + "state": "CT", + "scored": true, + "state_programs": "ct_employee_paid_leave_contribution=827.98", + "optional_amount": 0, + "reference_published": 13496.16, + "reference_no_optional": 13496.16, + "reference_federal_only": 12668.17, + "models": 46, + "exact_published": 10, + "exact_no_optional": 10, + "exact_federal_only": 13, + "within1pct_published": 11, + "within1pct_no_optional": 11, + "within1pct_federal_only": 15, + "modal_answers": { + "12668.00": 7, + "13496.16": 6, + "12668.17": 5, + "13496.00": 4 + } + }, + { + "scenario_id": "scenario_123", + "state": "PA", + "scored": true, + "state_programs": "pa_employee_unemployment_compensation_contribution=101.50", + "optional_amount": 0, + "reference_published": 11194.0, + "reference_no_optional": 11194.0, + "reference_federal_only": 11092.5, + "models": 46, + "exact_published": 11, + "exact_no_optional": 11, + "exact_federal_only": 22, + "within1pct_published": 39, + "within1pct_no_optional": 39, + "within1pct_federal_only": 39, + "modal_answers": { + "11092.50": 22, + "11194.00": 11, + "11663.35": 1, + "11741.39": 1 + } + } + ] +} diff --git a/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.csv b/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.csv new file mode 100644 index 00000000..b43f0482 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.csv @@ -0,0 +1,101 @@ +scenario_id,state,reference,recomputed,matches_reference,employee_social_security_tax,employee_medicare_tax,additional_medicare_tax,federal_total,employee_state_payroll_tax,parts_add_up,state_programs,leaves_add_up,excluded,excluded_reason,household_wages +scenario_000,TX,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_001,VA,2223.393798828125,2223.393798828125,True,1801.9661865234375,421.42755126953125,0.0,2223.3937377929688,0.0,True,,True,False,,29063.970703125 +scenario_002,WA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_003,TX,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_004,NY,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_005,CA,36044.0,36044.0,True,22599.0,6235.0,1620.0,30454.0,5590.0,True,ca_employee_state_disability_insurance_contribution=5590.00,True,False,,430000.0 +scenario_007,ID,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_008,NJ,2162.760009765625,2162.760009765625,True,1661.5999755859375,388.5999755859375,0.0,2050.199951171875,112.56000518798828,True,nj_employee_family_leave_insurance_contribution=61.64;nj_employee_temporary_disability_insurance_contribution=50.92,True,True,reference_engine_defect,26800.0 +scenario_009,NC,4590.0,4590.0,True,3720.0,870.0,0.0,4590.0,0.0,True,,True,False,,60000.0 +scenario_012,MS,1683.0,1683.0,True,1364.0,319.0,0.0,1683.0,0.0,True,,True,False,,22000.0 +scenario_013,AZ,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_014,WV,6802.96484375,6802.96484375,True,5513.51416015625,1289.4508056640625,0.0,6802.9649658203125,0.0,True,,True,False,,88927.6484375 +scenario_015,IN,3442.5,3442.5,True,2790.0,652.5,0.0,3442.5,0.0,True,,True,False,,45000.0 +scenario_016,FL,2649.80224609375,2649.80224609375,True,2147.55224609375,502.2501220703125,0.0,2649.8023681640625,0.0,True,,True,False,,34637.94140625 +scenario_018,AZ,5202.38232421875,5202.38232421875,True,4216.31005859375,986.0724487304688,0.0,5202.382507324219,0.0,True,,True,False,,68005.0 +scenario_020,TX,16689.0,16689.0,True,11439.0,4350.0,900.0,16689.0,0.0,True,,True,False,,300000.0 +scenario_021,MO,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_022,CA,4710.83251953125,4710.83251953125,True,3263.369873046875,763.2074584960938,0.0,4026.5773315429688,684.2550048828125,True,ca_employee_state_disability_insurance_contribution=684.26,True,False,,52635.0 +scenario_023,CA,1561.1168212890625,1561.1168212890625,True,1081.444091796875,252.91836547851562,0.0,1334.3624572753906,226.75440979003906,True,ca_employee_state_disability_insurance_contribution=226.75,True,False,,17442.646484375 +scenario_025,OH,4798.48486328125,4798.48486328125,True,3888.968017578125,909.5167236328125,0.0,4798.4847412109375,0.0,True,,True,False,,62725.29296875 +scenario_026,NC,6518.474609375,6518.474609375,True,5282.94677734375,1235.52783203125,0.0,6518.474609375,0.0,True,,True,False,,85208.8203125 +scenario_027,CT,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_028,PA,4632.0,4632.0,True,3720.0,870.0,0.0,4590.0,42.0,True,pa_employee_unemployment_compensation_contribution=42.00,True,False,,60000.0 +scenario_029,OK,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_030,TX,994.5,994.5,True,806.0,188.5,0.0,994.5,0.0,True,,True,False,,13000.0 +scenario_031,CA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_032,MN,2346.10009765625,2346.10009765625,True,1798.0,420.5,0.0,2218.5,127.5999984741211,True,mn_employee_paid_leave_contribution=127.60,True,False,,29000.0 +scenario_033,SD,6043.5,6043.5,True,4898.0,1145.5,0.0,6043.5,0.0,True,,True,False,,79000.0 +scenario_036,NJ,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_037,NC,3208.668701171875,3208.668701171875,True,2600.48974609375,608.1790161132812,0.0,3208.6687622070312,0.0,True,,True,False,,41943.3828125 +scenario_038,LA,571.2637329101562,571.2637329101562,True,462.9849853515625,108.27874755859375,0.0,571.2637329101562,0.0,True,,True,False,,7467.5 +scenario_039,VA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_040,AZ,2142.0,2142.0,True,1736.0,406.0,0.0,2142.0,0.0,True,,True,False,,28000.0 +scenario_042,WI,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_043,CO,330.7310791015625,330.7310791015625,True,253.46510314941406,59.27812957763672,0.0,312.7432327270508,17.98784637451172,True,co_employee_famli_contribution=17.99,True,False,,4088.14697265625 +scenario_044,KS,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_045,MI,2775.08251953125,2775.08251953125,True,2249.08642578125,525.9960327148438,0.0,2775.0824584960938,0.0,True,,True,False,,36275.58984375 +scenario_046,OK,7038.0,7038.0,True,5704.0,1334.0,0.0,7038.0,0.0,True,,True,False,,92000.0 +scenario_048,OH,585.6839599609375,585.6839599609375,True,474.6719970703125,111.01199340820312,0.0,585.6839904785156,0.0,True,,True,False,,7656.0 +scenario_049,NH,15134.5,15134.5,True,11439.0,3668.5,27.0,15134.5,0.0,True,,True,False,,253000.0 +scenario_051,LA,3060.0,3060.0,True,2480.0,580.0,0.0,3060.0,0.0,True,,True,False,,40000.0 +scenario_052,TX,20939.0,20939.0,True,11439.0,7250.0,2250.0,20939.0,0.0,True,,True,False,,500000.0 +scenario_053,ID,5123.10400390625,5123.10400390625,True,4152.05810546875,971.0458374023438,0.0,5123.103942871094,0.0,True,,True,False,,66968.6796875 +scenario_054,NC,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_055,FL,6398.08837890625,6398.08837890625,True,5185.37890625,1212.7095947265625,0.0,6398.0885009765625,0.0,True,,True,False,,83635.1484375 +scenario_056,NJ,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_057,LA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_059,FL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_060,TX,229.5,229.5,True,186.0,43.5,0.0,229.5,0.0,True,,True,False,,3000.0 +scenario_062,FL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_064,WI,7443.0673828125,7443.0673828125,True,6032.2900390625,1410.7774658203125,0.0,7443.0675048828125,0.0,True,,True,False,,97295.0 +scenario_066,VA,39.779998779296875,39.779998779296875,True,32.23999786376953,7.539999961853027,0.0,39.77999782562256,0.0,True,,True,False,,520.0 +scenario_067,IN,6300.5400390625,6300.5400390625,True,5106.31982421875,1194.219970703125,0.0,6300.539794921875,0.0,True,,True,False,,82360.0 +scenario_068,MD,2607.39013671875,2607.39013671875,True,2113.178955078125,494.211181640625,0.0,2607.39013671875,0.0,True,,True,False,,34083.53125 +scenario_070,IL,2907.0,2907.0,True,2356.0,551.0,0.0,2907.0,0.0,True,,True,False,,38000.0 +scenario_071,NY,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_072,MI,1530.0,1530.0,True,1240.0,290.0,0.0,1530.0,0.0,True,,True,False,,20000.0 +scenario_073,MI,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_074,LA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_075,FL,968.2120971679688,968.2120971679688,True,784.6947631835938,183.51731872558594,0.0,968.2120819091797,0.0,True,,True,False,,12656.3671875 +scenario_076,ID,12240.0,12240.0,True,9920.0,2320.0,0.0,12240.0,0.0,True,,True,False,,160000.0 +scenario_077,LA,1989.472412109375,1989.472412109375,True,1612.3828125,377.08953857421875,0.0,1989.4723510742188,0.0,True,,True,False,,26006.17578125 +scenario_078,MD,14317.322265625,14317.322265625,True,11439.0,2878.322509765625,0.0,14317.322509765625,0.0,True,,True,False,,198505.0 +scenario_079,AZ,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_080,PA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_081,MA,14192.662109375,14192.662109375,True,10850.1240234375,2537.52880859375,0.0,13387.65283203125,805.0092163085938,True,ma_employee_paid_leave_contribution=805.01,True,False,,175002.0 +scenario_082,NY,8108.02734375,8108.02734375,True,6212.08984375,1452.8275146484375,0.0,7664.9173583984375,443.1100158691406,True,ny_employee_disability_benefits_contribution=31.20;ny_employee_paid_family_leave_contribution=411.91,True,False,,100195.0 +scenario_083,TX,539.3250122070312,539.3250122070312,True,437.1000061035156,102.2249984741211,0.0,539.3250045776367,0.0,True,,True,False,,7050.0 +scenario_084,NC,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_085,PA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_086,GA,4743.0,4743.0,True,3844.0,899.0,0.0,4743.0,0.0,True,,True,False,,62000.0 +scenario_088,TX,1421.1448974609375,1421.1448974609375,True,1151.777587890625,269.3673400878906,0.0,1421.1449279785156,0.0,True,,True,False,,18577.05859375 +scenario_089,NC,5039.91015625,5039.91015625,True,4084.633056640625,955.277099609375,0.0,5039.91015625,0.0,True,,True,False,,65881.1796875 +scenario_090,KS,535.5,535.5,True,434.0,101.5,0.0,535.5,0.0,True,,True,False,,7000.0 +scenario_091,WI,2065.5,2065.5,True,1674.0,391.5,0.0,2065.5,0.0,True,,True,False,,27000.0 +scenario_092,AL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_093,MO,12469.5,12469.5,True,10106.0,2363.5,0.0,12469.5,0.0,True,,True,False,,163000.0 +scenario_095,NJ,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_098,IL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_099,CA,14767.5,14767.5,True,10230.0,2392.5,0.0,12622.5,2145.0,True,ca_employee_state_disability_insurance_contribution=2145.00,True,False,,165000.0 +scenario_100,MT,452.47613525390625,452.47613525390625,True,366.7126770019531,85.7634506225586,0.0,452.4761276245117,0.0,True,,True,False,,5914.720703125 +scenario_101,TX,7879.5,7879.5,True,6386.0,1493.5,0.0,7879.5,0.0,True,,True,False,,103000.0 +scenario_102,NC,6670.7998046875,6670.7998046875,True,5406.39990234375,1264.39990234375,0.0,6670.7998046875,0.0,True,,True,False,,87200.0 +scenario_104,NY,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_107,OH,486.9607238769531,486.9607238769531,True,394.6609802246094,92.29974365234375,0.0,486.9607238769531,0.0,True,,True,False,,6365.5 +scenario_108,WI,585.6839599609375,585.6839599609375,True,474.6719970703125,111.01199340820312,0.0,585.6839904785156,0.0,True,,True,False,,7656.0 +scenario_109,FL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_110,OH,7650.0,7650.0,True,6200.0,1450.0,0.0,7650.0,0.0,True,,True,False,,100000.0 +scenario_111,WA,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_112,TX,453.0960083007813,453.09600830078125,True,367.2150573730469,85.88094329833984,0.0,453.0960006713867,0.0,True,,True,False,,5922.82373046875 +scenario_114,VA,1736.550048828125,1736.550048828125,True,1407.4000244140625,329.1499938964844,0.0,1736.5500183105469,0.0,True,,True,False,,22700.0 +scenario_115,AL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_116,FL,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_117,AR,15044.787109375,15044.787109375,True,11439.0,3605.787353515625,0.0,15044.787353515625,0.0,True,,True,False,,248675.0 +scenario_118,NY,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_119,VA,4207.5,4207.5,True,3410.0,797.5,0.0,4207.5,0.0,True,,True,False,,55000.0 +scenario_120,CT,13496.1552734375,13496.1552734375,True,10267.013671875,2401.156494140625,0.0,12668.170166015625,827.9849853515625,True,ct_employee_paid_leave_contribution=827.98,True,False,,165597.0 +scenario_121,SC,1963.3724365234375,1963.3724365234375,True,1591.22998046875,372.1424865722656,0.0,1963.3724670410156,0.0,True,,True,False,,25665.0 +scenario_122,MN,0.0,0.0,True,0.0,0.0,0.0,0.0,0.0,True,,True,False,,0.0 +scenario_123,PA,11194.0,11194.0,True,8990.0,2102.5,0.0,11092.5,101.5,True,pa_employee_unemployment_compensation_contribution=101.50,True,False,,145000.0 diff --git a/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.json b/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.json new file mode 100644 index 00000000..d8d1c2b8 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/payroll_decomposition.json @@ -0,0 +1,1985 @@ +{ + "policyengine_us": "2.15.17", + "households": [ + { + "scenario_id": "scenario_000", + "state": "TX", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_001", + "state": "VA", + "payroll_tax": 2223.393798828125, + "parts": { + "employee_social_security_tax": 1801.9661865234375, + "employee_medicare_tax": 421.42755126953125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 29063.970703125, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_002", + "state": "WA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_003", + "state": "TX", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_004", + "state": "NY", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_005", + "state": "CA", + "payroll_tax": 36044.0, + "parts": { + "employee_social_security_tax": 22599.0, + "employee_medicare_tax": 6235.0, + "additional_medicare_tax": 1620.0, + "employee_state_payroll_tax": 5590.0 + }, + "state_leaves": { + "ca_employee_state_disability_insurance_contribution": 5590.0 + }, + "employment_income": [ + 180000.0, + 250000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_007", + "state": "ID", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_008", + "state": "NJ", + "payroll_tax": 2162.760009765625, + "parts": { + "employee_social_security_tax": 1661.5999755859375, + "employee_medicare_tax": 388.5999755859375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 112.56000518798828 + }, + "state_leaves": { + "nj_employee_temporary_disability_insurance_contribution": 50.92000198364258, + "nj_employee_family_leave_insurance_contribution": 61.6400032043457 + }, + "employment_income": [ + 0.0, + 26800.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2", + "child3", + "child4", + "child5", + "child6" + ] + }, + { + "scenario_id": "scenario_009", + "state": "NC", + "payroll_tax": 4590.0, + "parts": { + "employee_social_security_tax": 3720.0, + "employee_medicare_tax": 870.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 60000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_012", + "state": "MS", + "payroll_tax": 1683.0, + "parts": { + "employee_social_security_tax": 1364.0, + "employee_medicare_tax": 319.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 22000.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1" + ] + }, + { + "scenario_id": "scenario_013", + "state": "AZ", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_014", + "state": "WV", + "payroll_tax": 6802.96484375, + "parts": { + "employee_social_security_tax": 5513.51416015625, + "employee_medicare_tax": 1289.4508056640625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 88927.6484375 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_015", + "state": "IN", + "payroll_tax": 3442.5, + "parts": { + "employee_social_security_tax": 2790.0, + "employee_medicare_tax": 652.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 45000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_016", + "state": "FL", + "payroll_tax": 2649.80224609375, + "parts": { + "employee_social_security_tax": 2147.55224609375, + "employee_medicare_tax": 502.2501220703125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 34637.94140625 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_018", + "state": "AZ", + "payroll_tax": 5202.38232421875, + "parts": { + "employee_social_security_tax": 4216.31005859375, + "employee_medicare_tax": 986.0724487304688, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 68005.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_020", + "state": "TX", + "payroll_tax": 16689.0, + "parts": { + "employee_social_security_tax": 11439.0, + "employee_medicare_tax": 4350.0, + "additional_medicare_tax": 900.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 300000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_021", + "state": "MO", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_022", + "state": "CA", + "payroll_tax": 4710.83251953125, + "parts": { + "employee_social_security_tax": 3263.369873046875, + "employee_medicare_tax": 763.2074584960938, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 684.2550048828125 + }, + "state_leaves": { + "ca_employee_state_disability_insurance_contribution": 684.2550048828125 + }, + "employment_income": [ + 52635.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_023", + "state": "CA", + "payroll_tax": 1561.1168212890625, + "parts": { + "employee_social_security_tax": 1081.444091796875, + "employee_medicare_tax": 252.91836547851562, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 226.75440979003906 + }, + "state_leaves": { + "ca_employee_state_disability_insurance_contribution": 226.75440979003906 + }, + "employment_income": [ + 17442.646484375 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_025", + "state": "OH", + "payroll_tax": 4798.48486328125, + "parts": { + "employee_social_security_tax": 3888.968017578125, + "employee_medicare_tax": 909.5167236328125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 62725.29296875 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_026", + "state": "NC", + "payroll_tax": 6518.474609375, + "parts": { + "employee_social_security_tax": 5282.94677734375, + "employee_medicare_tax": 1235.52783203125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 85208.8203125, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2", + "child3" + ] + }, + { + "scenario_id": "scenario_027", + "state": "CT", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_028", + "state": "PA", + "payroll_tax": 4632.0, + "parts": { + "employee_social_security_tax": 3720.0, + "employee_medicare_tax": 870.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 42.0 + }, + "state_leaves": { + "pa_employee_unemployment_compensation_contribution": 42.0 + }, + "employment_income": [ + 60000.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "child1", + "child2", + "child3" + ] + }, + { + "scenario_id": "scenario_029", + "state": "OK", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_030", + "state": "TX", + "payroll_tax": 994.5, + "parts": { + "employee_social_security_tax": 806.0, + "employee_medicare_tax": 188.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 13000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_031", + "state": "CA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_032", + "state": "MN", + "payroll_tax": 2346.10009765625, + "parts": { + "employee_social_security_tax": 1798.0, + "employee_medicare_tax": 420.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 127.5999984741211 + }, + "state_leaves": { + "mn_employee_paid_leave_contribution": 127.5999984741211 + }, + "employment_income": [ + 29000.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1" + ] + }, + { + "scenario_id": "scenario_033", + "state": "SD", + "payroll_tax": 6043.5, + "parts": { + "employee_social_security_tax": 4898.0, + "employee_medicare_tax": 1145.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 26000.0, + 53000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_036", + "state": "NJ", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_037", + "state": "NC", + "payroll_tax": 3208.668701171875, + "parts": { + "employee_social_security_tax": 2600.48974609375, + "employee_medicare_tax": 608.1790161132812, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 41943.3828125 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_038", + "state": "LA", + "payroll_tax": 571.2637329101562, + "parts": { + "employee_social_security_tax": 462.9849853515625, + "employee_medicare_tax": 108.27874755859375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 7467.5, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_039", + "state": "VA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_040", + "state": "AZ", + "payroll_tax": 2142.0, + "parts": { + "employee_social_security_tax": 1736.0, + "employee_medicare_tax": 406.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 20000.0, + 8000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_042", + "state": "WI", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_043", + "state": "CO", + "payroll_tax": 330.7310791015625, + "parts": { + "employee_social_security_tax": 253.46510314941406, + "employee_medicare_tax": 59.27812957763672, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 17.98784637451172 + }, + "state_leaves": { + "co_employee_famli_contribution": 17.98784637451172 + }, + "employment_income": [ + 4088.14697265625 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_044", + "state": "KS", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_045", + "state": "MI", + "payroll_tax": 2775.08251953125, + "parts": { + "employee_social_security_tax": 2249.08642578125, + "employee_medicare_tax": 525.9960327148438, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 36275.58984375 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_046", + "state": "OK", + "payroll_tax": 7038.0, + "parts": { + "employee_social_security_tax": 5704.0, + "employee_medicare_tax": 1334.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 92000.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_048", + "state": "OH", + "payroll_tax": 585.6839599609375, + "parts": { + "employee_social_security_tax": 474.6719970703125, + "employee_medicare_tax": 111.01199340820312, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 7656.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_049", + "state": "NH", + "payroll_tax": 15134.5, + "parts": { + "employee_social_security_tax": 11439.0, + "employee_medicare_tax": 3668.5, + "additional_medicare_tax": 27.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 253000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_051", + "state": "LA", + "payroll_tax": 3060.0, + "parts": { + "employee_social_security_tax": 2480.0, + "employee_medicare_tax": 580.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 40000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_052", + "state": "TX", + "payroll_tax": 20939.0, + "parts": { + "employee_social_security_tax": 11439.0, + "employee_medicare_tax": 7250.0, + "additional_medicare_tax": 2250.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 500000.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_053", + "state": "ID", + "payroll_tax": 5123.10400390625, + "parts": { + "employee_social_security_tax": 4152.05810546875, + "employee_medicare_tax": 971.0458374023438, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 66968.6796875 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_054", + "state": "NC", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_055", + "state": "FL", + "payroll_tax": 6398.08837890625, + "parts": { + "employee_social_security_tax": 5185.37890625, + "employee_medicare_tax": 1212.7095947265625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 83635.1484375 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_056", + "state": "NJ", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_057", + "state": "LA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_059", + "state": "FL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_060", + "state": "TX", + "payroll_tax": 229.5, + "parts": { + "employee_social_security_tax": 186.0, + "employee_medicare_tax": 43.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 3000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_062", + "state": "FL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_064", + "state": "WI", + "payroll_tax": 7443.0673828125, + "parts": { + "employee_social_security_tax": 6032.2900390625, + "employee_medicare_tax": 1410.7774658203125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 97295.0, + 0.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "dependent1", + "dependent2", + "child1" + ] + }, + { + "scenario_id": "scenario_066", + "state": "VA", + "payroll_tax": 39.779998779296875, + "parts": { + "employee_social_security_tax": 32.23999786376953, + "employee_medicare_tax": 7.539999961853027, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 520.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_067", + "state": "IN", + "payroll_tax": 6300.5400390625, + "parts": { + "employee_social_security_tax": 5106.31982421875, + "employee_medicare_tax": 1194.219970703125, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 82360.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "dependent1" + ] + }, + { + "scenario_id": "scenario_068", + "state": "MD", + "payroll_tax": 2607.39013671875, + "parts": { + "employee_social_security_tax": 2113.178955078125, + "employee_medicare_tax": 494.211181640625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 34083.53125 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_070", + "state": "IL", + "payroll_tax": 2907.0, + "parts": { + "employee_social_security_tax": 2356.0, + "employee_medicare_tax": 551.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 38000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_071", + "state": "NY", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_072", + "state": "MI", + "payroll_tax": 1530.0, + "parts": { + "employee_social_security_tax": 1240.0, + "employee_medicare_tax": 290.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 20000.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_073", + "state": "MI", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_074", + "state": "LA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_075", + "state": "FL", + "payroll_tax": 968.2120971679688, + "parts": { + "employee_social_security_tax": 784.6947631835938, + "employee_medicare_tax": 183.51731872558594, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 12656.3671875 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_076", + "state": "ID", + "payroll_tax": 12240.0, + "parts": { + "employee_social_security_tax": 9920.0, + "employee_medicare_tax": 2320.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 160000.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_077", + "state": "LA", + "payroll_tax": 1989.472412109375, + "parts": { + "employee_social_security_tax": 1612.3828125, + "employee_medicare_tax": 377.08953857421875, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 26006.17578125 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_078", + "state": "MD", + "payroll_tax": 14317.322265625, + "parts": { + "employee_social_security_tax": 11439.0, + "employee_medicare_tax": 2878.322509765625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 198505.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_079", + "state": "AZ", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_080", + "state": "PA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_081", + "state": "MA", + "payroll_tax": 14192.662109375, + "parts": { + "employee_social_security_tax": 10850.1240234375, + "employee_medicare_tax": 2537.52880859375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 805.0092163085938 + }, + "state_leaves": { + "ma_employee_paid_leave_contribution": 805.0092163085938 + }, + "employment_income": [ + 175002.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_082", + "state": "NY", + "payroll_tax": 8108.02734375, + "parts": { + "employee_social_security_tax": 6212.08984375, + "employee_medicare_tax": 1452.8275146484375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 443.1100158691406 + }, + "state_leaves": { + "ny_employee_paid_family_leave_contribution": 411.9100036621094, + "ny_employee_disability_benefits_contribution": 31.200000762939453 + }, + "employment_income": [ + 100195.0, + 0.0 + ], + "people": [ + "head", + "child1" + ] + }, + { + "scenario_id": "scenario_083", + "state": "TX", + "payroll_tax": 539.3250122070312, + "parts": { + "employee_social_security_tax": 437.1000061035156, + "employee_medicare_tax": 102.2249984741211, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 7050.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_084", + "state": "NC", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_085", + "state": "PA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_086", + "state": "GA", + "payroll_tax": 4743.0, + "parts": { + "employee_social_security_tax": 3844.0, + "employee_medicare_tax": 899.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 62000.0, + 0.0 + ], + "people": [ + "head", + "child1" + ] + }, + { + "scenario_id": "scenario_088", + "state": "TX", + "payroll_tax": 1421.1448974609375, + "parts": { + "employee_social_security_tax": 1151.777587890625, + "employee_medicare_tax": 269.3673400878906, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 18577.05859375 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_089", + "state": "NC", + "payroll_tax": 5039.91015625, + "parts": { + "employee_social_security_tax": 4084.633056640625, + "employee_medicare_tax": 955.277099609375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 65881.1796875 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_090", + "state": "KS", + "payroll_tax": 535.5, + "parts": { + "employee_social_security_tax": 434.0, + "employee_medicare_tax": 101.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 7000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_091", + "state": "WI", + "payroll_tax": 2065.5, + "parts": { + "employee_social_security_tax": 1674.0, + "employee_medicare_tax": 391.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 27000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_092", + "state": "AL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_093", + "state": "MO", + "payroll_tax": 12469.5, + "parts": { + "employee_social_security_tax": 10106.0, + "employee_medicare_tax": 2363.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 53000.0, + 65000.0, + 45000.0 + ], + "people": [ + "head", + "spouse", + "dependent1" + ] + }, + { + "scenario_id": "scenario_095", + "state": "NJ", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_098", + "state": "IL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_099", + "state": "CA", + "payroll_tax": 14767.5, + "parts": { + "employee_social_security_tax": 10230.0, + "employee_medicare_tax": 2392.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 2145.0 + }, + "state_leaves": { + "ca_employee_state_disability_insurance_contribution": 2145.0 + }, + "employment_income": [ + 85000.0, + 80000.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_100", + "state": "MT", + "payroll_tax": 452.47613525390625, + "parts": { + "employee_social_security_tax": 366.7126770019531, + "employee_medicare_tax": 85.7634506225586, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 5914.720703125, + 0.0, + 0.0 + ], + "people": [ + "head", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_101", + "state": "TX", + "payroll_tax": 7879.5, + "parts": { + "employee_social_security_tax": 6386.0, + "employee_medicare_tax": 1493.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 103000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_102", + "state": "NC", + "payroll_tax": 6670.7998046875, + "parts": { + "employee_social_security_tax": 5406.39990234375, + "employee_medicare_tax": 1264.39990234375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 41200.0, + 46000.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_104", + "state": "NY", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_107", + "state": "OH", + "payroll_tax": 486.9607238769531, + "parts": { + "employee_social_security_tax": 394.6609802246094, + "employee_medicare_tax": 92.29974365234375, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 6365.5 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_108", + "state": "WI", + "payroll_tax": 585.6839599609375, + "parts": { + "employee_social_security_tax": 474.6719970703125, + "employee_medicare_tax": 111.01199340820312, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 7656.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_109", + "state": "FL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2", + "child3" + ] + }, + { + "scenario_id": "scenario_110", + "state": "OH", + "payroll_tax": 7650.0, + "parts": { + "employee_social_security_tax": 6200.0, + "employee_medicare_tax": 1450.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 100000.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_111", + "state": "WA", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_112", + "state": "TX", + "payroll_tax": 453.09600830078125, + "parts": { + "employee_social_security_tax": 367.2150573730469, + "employee_medicare_tax": 85.88094329833984, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 5922.82373046875 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_114", + "state": "VA", + "payroll_tax": 1736.550048828125, + "parts": { + "employee_social_security_tax": 1407.4000244140625, + "employee_medicare_tax": 329.1499938964844, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 22700.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_115", + "state": "AL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_116", + "state": "FL", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse" + ] + }, + { + "scenario_id": "scenario_117", + "state": "AR", + "payroll_tax": 15044.787109375, + "parts": { + "employee_social_security_tax": 11439.0, + "employee_medicare_tax": 3605.787353515625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0, + 248675.0, + 0.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "spouse", + "child1", + "child2", + "child3" + ] + }, + { + "scenario_id": "scenario_118", + "state": "NY", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_119", + "state": "VA", + "payroll_tax": 4207.5, + "parts": { + "employee_social_security_tax": 3410.0, + "employee_medicare_tax": 797.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 55000.0, + 0.0, + 0.0 + ], + "people": [ + "head", + "child1", + "child2" + ] + }, + { + "scenario_id": "scenario_120", + "state": "CT", + "payroll_tax": 13496.1552734375, + "parts": { + "employee_social_security_tax": 10267.013671875, + "employee_medicare_tax": 2401.156494140625, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 827.9849853515625 + }, + "state_leaves": { + "ct_employee_paid_leave_contribution": 827.9849853515625 + }, + "employment_income": [ + 165597.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_121", + "state": "SC", + "payroll_tax": 1963.3724365234375, + "parts": { + "employee_social_security_tax": 1591.22998046875, + "employee_medicare_tax": 372.1424865722656, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 25665.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_122", + "state": "MN", + "payroll_tax": 0.0, + "parts": { + "employee_social_security_tax": 0.0, + "employee_medicare_tax": 0.0, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 0.0 + }, + "state_leaves": {}, + "employment_income": [ + 0.0 + ], + "people": [ + "head" + ] + }, + { + "scenario_id": "scenario_123", + "state": "PA", + "payroll_tax": 11194.0, + "parts": { + "employee_social_security_tax": 8990.0, + "employee_medicare_tax": 2102.5, + "additional_medicare_tax": 0.0, + "employee_state_payroll_tax": 101.5 + }, + "state_leaves": { + "pa_employee_unemployment_compensation_contribution": 101.5 + }, + "employment_income": [ + 100000.0, + 0.0, + 45000.0 + ], + "people": [ + "head", + "spouse", + "child1" + ] + } + ] +} diff --git a/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope.csv b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope.csv new file mode 100644 index 00000000..770c5eb8 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope.csv @@ -0,0 +1,1985 @@ +scenario_id,state,variable,reference,final,final_matches_reference,scoped,delta,moved_any,moved_over_tolerance,excluded,excluded_reason +scenario_000,TX,federal_income_tax_before_refundable_credits,2906.45361328125,2906.45361328125,True,2906.45361328125,0.0,False,False,False, +scenario_000,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_000,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_000,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,payroll_tax,2223.393798828125,2223.393798828125,True,2223.393798828125,0.0,False,False,False, +scenario_001,VA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_001,VA,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_001,VA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_001,VA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_002,WA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_002,WA,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_002,WA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_002,WA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,federal_income_tax_before_refundable_credits,22154.69921875,22154.69921875,True,22154.69921875,0.0,False,False,True,reference_engine_defect +scenario_003,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_003,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_004,NY,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,federal_income_tax_before_refundable_credits,106505.8984375,107833.15625,False,107833.15625,0.0,False,False,True,reference_engine_defect +scenario_005,CA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,payroll_tax,36044.0,36044.0,True,36044.0,0.0,False,False,False, +scenario_005,CA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,state_income_tax_before_refundable_credits,41051.51171875,41267.01171875,False,41267.01171875,0.0,False,False,True,reference_engine_defect +scenario_005,CA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_005,CA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,federal_income_tax_before_refundable_credits,3707.695068359375,3707.695068359375,True,3707.695068359375,0.0,False,False,False, +scenario_007,ID,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_007,ID,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_007,ID,state_income_tax_before_refundable_credits,755.7783203125,761.5570068359375,False,761.5570068359375,0.0,False,False,True,reference_engine_defect +scenario_007,ID,state_refundable_credits,155.0,155.0,True,155.0,0.0,False,False,False, +scenario_007,ID,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,federal_refundable_credits,12433.611328125,12433.611328125,True,12433.611328125,0.0,False,False,False, +scenario_008,NJ,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,payroll_tax,2162.760009765625,2162.760009765625,True,2162.760009765625,0.0,False,False,True,reference_engine_defect +scenario_008,NJ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child3_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child4_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child5_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child6_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child3_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child4_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child5_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child6_early_head_start_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child3_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child4_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child5_head_start_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child6_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child3_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child4_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child5_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,child6_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child3_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child4_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child5_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child6_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child3_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child4_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child5_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,child6_wic_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_008,NJ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,self_employment_tax,565.1820068359375,565.1820068359375,True,565.1820068359375,0.0,False,False,False, +scenario_008,NJ,snap,15108.0,15108.0,True,15108.0,0.0,False,False,False, +scenario_008,NJ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_008,NJ,state_income_tax_before_refundable_credits,264.80999755859375,264.80999755859375,True,264.80999755859375,0.0,False,False,False, +scenario_008,NJ,state_refundable_credits,5842.39990234375,5842.39990234375,True,5842.39990234375,0.0,False,False,False, +scenario_008,NJ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,federal_income_tax_before_refundable_credits,8348.0,8348.0,True,8348.0,0.0,False,False,False, +scenario_009,NC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,payroll_tax,4590.0,4590.0,True,4590.0,0.0,False,False,False, +scenario_009,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,state_income_tax_before_refundable_credits,2190.510009765625,2190.510009765625,True,2190.510009765625,0.0,False,False,False, +scenario_009,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_009,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,federal_refundable_credits,6127.0,6127.0,True,6127.0,0.0,False,False,False, +scenario_012,MS,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_012,MS,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,payroll_tax,1683.0,1683.0,True,1683.0,0.0,False,False,False, +scenario_012,MS,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_012,MS,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_012,MS,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,spouse_wic_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_012,MS,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,snap,4884.0,4884.0,True,4884.0,0.0,False,False,False, +scenario_012,MS,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_012,MS,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_013,AZ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,snap,240.0,240.0,True,240.0,0.0,False,False,False, +scenario_013,AZ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_013,AZ,state_refundable_credits,25.0,25.0,True,25.0,0.0,False,False,False, +scenario_013,AZ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,federal_income_tax_before_refundable_credits,6311.31787109375,6311.31787109375,True,6311.31787109375,0.0,False,False,False, +scenario_014,WV,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,payroll_tax,6802.96484375,6802.96484375,True,6802.96484375,0.0,False,False,False, +scenario_014,WV,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,state_income_tax_before_refundable_credits,3092.186279296875,3092.186279296875,True,3092.186279296875,0.0,False,False,False, +scenario_014,WV,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_014,WV,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,federal_income_tax_before_refundable_credits,2969.1259765625,2969.1259765625,True,2969.1259765625,0.0,False,False,False, +scenario_015,IN,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,payroll_tax,3442.5,3442.5,True,3442.5,0.0,False,False,False, +scenario_015,IN,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,state_income_tax_before_refundable_credits,1236.3267822265625,1236.3267822265625,True,1236.3267822265625,0.0,False,False,False, +scenario_015,IN,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_015,IN,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,payroll_tax,2649.80224609375,2649.80224609375,True,2649.80224609375,0.0,False,False,False, +scenario_016,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_016,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,federal_income_tax_before_refundable_credits,5211.05078125,5211.05078125,True,5211.05078125,0.0,False,False,False, +scenario_018,AZ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,payroll_tax,5202.38232421875,5202.38232421875,True,5202.38232421875,0.0,False,False,False, +scenario_018,AZ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,state_income_tax_before_refundable_credits,1146.05224609375,1146.05224609375,True,1146.05224609375,0.0,False,False,False, +scenario_018,AZ,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_018,AZ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,federal_income_tax_before_refundable_credits,68056.7109375,68334.734375,False,68334.734375,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_020,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,payroll_tax,16689.0,16689.0,True,16689.0,0.0,False,False,False, +scenario_020,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_020,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_021,MO,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_021,MO,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,federal_income_tax_before_refundable_credits,11113.5703125,11113.5703125,True,11113.5703125,0.0,False,False,False, +scenario_022,CA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,payroll_tax,4710.83251953125,4710.83251953125,True,4710.83251953125,0.0,False,False,False, +scenario_022,CA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_022,CA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,state_income_tax_before_refundable_credits,2439.650146484375,2505.8701171875,False,2505.8701171875,0.0,False,False,True,reference_engine_defect +scenario_022,CA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_022,CA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,federal_income_tax_before_refundable_credits,643.43359375,643.43359375,True,643.43359375,0.0,False,False,False, +scenario_023,CA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,payroll_tax,1561.1168212890625,1561.1168212890625,True,1561.1168212890625,0.0,False,False,False, +scenario_023,CA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_023,CA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,snap,461.3397216796875,408.0,False,408.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_023,CA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_023,CA,state_income_tax_before_refundable_credits,12.776718139648438,12.776718139648438,True,12.776718139648438,0.0,False,False,False, +scenario_023,CA,state_refundable_credits,95.1540069580078,95.15400695800781,True,95.15400695800781,0.0,False,False,False, +scenario_023,CA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,federal_income_tax_before_refundable_credits,7031.03564453125,7031.03564453125,True,7031.03564453125,0.0,False,False,False, +scenario_025,OH,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,payroll_tax,4798.48486328125,4798.48486328125,True,4798.48486328125,0.0,False,False,False, +scenario_025,OH,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,state_income_tax_before_refundable_credits,1921.57275390625,1921.57275390625,True,1921.57275390625,0.0,False,False,False, +scenario_025,OH,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_025,OH,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,federal_refundable_credits,2013.4892578125,2013.4892578125,True,2013.4892578125,0.0,False,False,False, +scenario_026,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,payroll_tax,6518.474609375,6518.474609375,True,6518.474609375,0.0,False,False,False, +scenario_026,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child3_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child3_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child3_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_026,NC,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_026,NC,child3_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_026,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child3_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,child3_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,state_income_tax_before_refundable_credits,2086.79052734375,2086.79052734375,True,2086.79052734375,0.0,False,False,False, +scenario_026,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_026,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_027,CT,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,snap,288.0,288.0,True,288.0,0.0,False,False,False, +scenario_027,CT,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_027,CT,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,federal_refundable_credits,3274.927978515625,3274.927978515625,True,3274.927978515625,0.0,False,False,False, +scenario_028,PA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,payroll_tax,4632.0,4632.0,True,4632.0,0.0,False,False,False, +scenario_028,PA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,child3_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_028,PA,state_income_tax_before_refundable_credits,1842.3070068359375,1842.3070068359375,True,1842.3070068359375,0.0,False,False,False, +scenario_028,PA,state_refundable_credits,62.41279220581055,62.41279220581055,True,62.41279220581055,0.0,False,False,False, +scenario_028,PA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_029,OK,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_029,OK,state_refundable_credits,40.0,40.0,True,40.0,0.0,False,False,False, +scenario_029,OK,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,payroll_tax,994.5,994.5,True,994.5,0.0,False,False,False, +scenario_030,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_030,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,head_wic_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_030,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,snap,288.0,288.0,True,288.0,0.0,False,False,False, +scenario_030,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_030,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_031,CA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_031,CA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_031,CA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,federal_refundable_credits,6127.0,6127.0,True,6127.0,0.0,False,False,False, +scenario_032,MN,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_032,MN,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,payroll_tax,2346.10009765625,2346.10009765625,True,2218.5,-127.60009765625,True,True,False, +scenario_032,MN,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_032,MN,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_032,MN,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_032,MN,state_refundable_credits,2187.60009765625,2187.60009765625,True,2187.60009765625,0.0,False,False,False, +scenario_032,MN,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,federal_income_tax_before_refundable_credits,3818.148193359375,3818.148193359375,True,3818.148193359375,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_033,SD,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,payroll_tax,6043.5,6043.5,True,6043.5,0.0,False,False,False, +scenario_033,SD,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_033,SD,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_033,SD,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_033,SD,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,federal_income_tax_before_refundable_credits,4926.39990234375,4926.39990234375,True,4926.39990234375,0.0,False,False,False, +scenario_036,NJ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,state_income_tax_before_refundable_credits,1724.155029296875,1724.155029296875,True,1724.155029296875,0.0,False,False,False, +scenario_036,NJ,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_036,NJ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,federal_income_tax_before_refundable_credits,1649.2513427734375,1649.2513427734375,True,1649.2513427734375,0.0,False,False,False, +scenario_037,NC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,payroll_tax,3208.668701171875,3208.668701171875,True,3208.668701171875,0.0,False,False,False, +scenario_037,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,state_income_tax_before_refundable_credits,1148.69921875,1148.69921875,True,1148.69921875,0.0,False,False,False, +scenario_037,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_037,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,federal_refundable_credits,10225.353515625,10225.353515625,True,10225.353515625,0.0,False,False,False, +scenario_038,LA,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_038,LA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,payroll_tax,571.2637329101562,571.2637329101562,True,571.2637329101562,0.0,False,False,False, +scenario_038,LA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_038,LA,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_038,LA,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_038,LA,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_038,LA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,self_employment_tax,2193.612548828125,2193.612548828125,True,2193.612548828125,0.0,False,False,False, +scenario_038,LA,snap,7212.0,7212.0,True,7212.0,0.0,False,False,False, +scenario_038,LA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_038,LA,state_refundable_credits,365.8000183105469,365.8000183105469,True,365.8000183105469,0.0,False,False,False, +scenario_038,LA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,federal_income_tax_before_refundable_credits,1345.510498046875,1345.510498046875,True,1345.510498046875,0.0,False,False,True,reference_engine_defect +scenario_039,VA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,state_income_tax_before_refundable_credits,514.4984130859375,514.4984130859375,True,514.4984130859375,0.0,False,False,True,reference_engine_defect +scenario_039,VA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_039,VA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,payroll_tax,2142.0,2142.0,True,2142.0,0.0,False,False,False, +scenario_040,AZ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_040,AZ,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_040,AZ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_040,AZ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,federal_income_tax_before_refundable_credits,2361.96484375,2361.96484375,True,2361.96484375,0.0,False,False,False, +scenario_042,WI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_042,WI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,state_income_tax_before_refundable_credits,284.74090576171875,464.18280029296875,False,464.18280029296875,0.0,False,False,True,reference_engine_defect +scenario_042,WI,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_042,WI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,payroll_tax,330.7310791015625,330.7310791015625,True,312.74322509765625,-17.98785400390625,True,True,False, +scenario_043,CO,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_043,CO,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,snap,3576.0,3576.0,True,3576.0,0.0,False,False,False, +scenario_043,CO,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_043,CO,state_refundable_credits,19.0,19.0,True,19.0,0.0,False,False,False, +scenario_043,CO,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_044,KS,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_044,KS,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,federal_income_tax_before_refundable_credits,2173.07080078125,2173.07080078125,True,2173.07080078125,0.0,False,False,False, +scenario_045,MI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,payroll_tax,2775.08251953125,2775.08251953125,True,2775.08251953125,0.0,False,False,False, +scenario_045,MI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_045,MI,state_income_tax_before_refundable_credits,1290.9625244140625,1290.9625244140625,True,1290.9625244140625,0.0,False,False,False, +scenario_045,MI,state_refundable_credits,760.7887573242188,760.7887573242188,True,760.7887573242188,0.0,False,False,False, +scenario_045,MI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,federal_income_tax_before_refundable_credits,499.71484375,499.71484375,True,499.71484375,0.0,False,False,False, +scenario_046,OK,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,payroll_tax,7038.0,7038.0,True,7038.0,0.0,False,False,False, +scenario_046,OK,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,state_income_tax_before_refundable_credits,2739.135009765625,2739.135009765625,True,2739.135009765625,0.0,False,False,False, +scenario_046,OK,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_046,OK,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,payroll_tax,585.6839599609375,585.6839599609375,True,585.6839599609375,0.0,False,False,False, +scenario_048,OH,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_048,OH,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_048,OH,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,federal_income_tax_before_refundable_credits,30543.908203125,30543.908203125,True,30543.908203125,0.0,False,False,True,reference_engine_defect +scenario_049,NH,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,payroll_tax,15134.5,15134.5,True,15134.5,0.0,False,False,False, +scenario_049,NH,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_049,NH,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,federal_income_tax_before_refundable_credits,2620.0,2620.0,True,2620.0,0.0,False,False,False, +scenario_051,LA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,payroll_tax,3060.0,3060.0,True,3060.0,0.0,False,False,False, +scenario_051,LA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,state_income_tax_before_refundable_credits,820.3499755859375,820.3499755859375,True,820.3499755859375,0.0,False,False,False, +scenario_051,LA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_051,LA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,federal_income_tax_before_refundable_credits,104211.40625,104211.40625,True,104211.40625,0.0,False,False,True,reference_engine_defect +scenario_052,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,payroll_tax,20939.0,20939.0,True,20939.0,0.0,False,False,False, +scenario_052,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_052,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,federal_income_tax_before_refundable_credits,5903.109375,5903.109375,True,5903.109375,0.0,False,False,False, +scenario_053,ID,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,payroll_tax,5123.10400390625,5123.10400390625,True,5123.10400390625,0.0,False,False,False, +scenario_053,ID,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_053,ID,state_income_tax_before_refundable_credits,2435.2783203125,2441.05712890625,False,2441.05712890625,0.0,False,False,True,reference_engine_defect +scenario_053,ID,state_refundable_credits,155.0,155.0,True,155.0,0.0,False,False,False, +scenario_053,ID,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,federal_refundable_credits,8365.55859375,8365.55859375,True,8365.55859375,0.0,False,False,False, +scenario_054,NC,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child2_early_head_start_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,child2_wic_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_054,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,self_employment_tax,2416.15283203125,2416.15283203125,True,2416.15283203125,0.0,False,False,False, +scenario_054,NC,snap,6060.0,6060.0,True,6060.0,0.0,False,False,False, +scenario_054,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_054,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,federal_income_tax_before_refundable_credits,25238.578125,25238.578125,True,25238.578125,0.0,False,False,False, +scenario_055,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,payroll_tax,6398.08837890625,6398.08837890625,True,6398.08837890625,0.0,False,False,False, +scenario_055,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_055,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_055,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_056,NJ,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,self_employment_tax,613.2224731445312,613.2224731445312,True,613.2224731445312,0.0,False,False,False, +scenario_056,NJ,snap,1140.0,95.0,False,95.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_056,NJ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_056,NJ,state_refundable_credits,265.6000061035156,265.6000061035156,True,265.6000061035156,0.0,False,False,False, +scenario_056,NJ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,federal_refundable_credits,664.0,664.0,True,664.0,0.0,False,False,False, +scenario_057,LA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_057,LA,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_057,LA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,self_employment_tax,1469.4732666015625,1469.4732666015625,True,1469.4732666015625,0.0,False,False,False, +scenario_057,LA,snap,2669.217041015625,2628.0,False,2628.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_057,LA,ssi,7231.00146484375,7231.00146484375,True,7231.00146484375,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_057,LA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_057,LA,state_refundable_credits,33.20000076293945,33.20000076293945,True,33.20000076293945,0.0,False,False,False, +scenario_057,LA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_059,FL,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_059,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_059,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,payroll_tax,229.5,229.5,True,229.5,0.0,False,False,False, +scenario_060,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_060,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_060,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_062,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_062,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_062,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,federal_income_tax_before_refundable_credits,4439.28955078125,4439.28955078125,True,4439.28955078125,0.0,False,False,True,reference_engine_defect +scenario_064,WI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,payroll_tax,7443.0673828125,7443.0673828125,True,7443.0673828125,0.0,False,False,False, +scenario_064,WI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_064,WI,dependent2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_064,WI,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,dependent2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,ssi,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_064,WI,state_income_tax_before_refundable_credits,4605.9970703125,4598.47607421875,False,4598.47607421875,0.0,False,False,True,reference_engine_defect +scenario_064,WI,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_064,WI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,federal_refundable_credits,39.779998779296875,39.779998779296875,True,39.779998779296875,0.0,False,False,False, +scenario_066,VA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,payroll_tax,39.779998779296875,39.779998779296875,True,39.779998779296875,0.0,False,False,False, +scenario_066,VA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_066,VA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,snap,3576.0,3576.0,True,3576.0,0.0,False,False,False, +scenario_066,VA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_066,VA,state_refundable_credits,7.955999851226807,7.955999851226807,True,7.955999851226807,0.0,False,False,False, +scenario_066,VA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,federal_income_tax_before_refundable_credits,5616.240234375,5616.240234375,True,5616.240234375,0.0,False,False,False, +scenario_067,IN,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,payroll_tax,6300.5400390625,6300.5400390625,True,6300.5400390625,0.0,False,False,False, +scenario_067,IN,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,dependent1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,dependent1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_067,IN,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,dependent1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,dependent1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,ssi,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_067,IN,state_income_tax_before_refundable_credits,2486.908935546875,2486.908935546875,True,2486.908935546875,0.0,False,False,False, +scenario_067,IN,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_067,IN,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,federal_income_tax_before_refundable_credits,1910.0238037109375,1910.0238037109375,True,1910.0238037109375,0.0,False,False,False, +scenario_068,MD,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,payroll_tax,2607.39013671875,2607.39013671875,True,2607.39013671875,0.0,False,False,False, +scenario_068,MD,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,state_income_tax_before_refundable_credits,1255.3427734375,1255.3427734375,True,1255.3427734375,0.0,False,False,False, +scenario_068,MD,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_068,MD,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,federal_income_tax_before_refundable_credits,1979.9605712890625,1979.9605712890625,True,1979.9605712890625,0.0,False,False,False, +scenario_070,IL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,payroll_tax,2907.0,2907.0,True,2907.0,0.0,False,False,False, +scenario_070,IL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,state_income_tax_before_refundable_credits,1650.561279296875,1650.561279296875,True,1650.561279296875,0.0,False,False,False, +scenario_070,IL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_070,IL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_071,NY,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_071,NY,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,payroll_tax,1530.0,1530.0,True,1530.0,0.0,False,False,False, +scenario_072,MI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_072,MI,spouse_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_072,MI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_072,MI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_073,MI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,snap,288.0,288.0,True,288.0,0.0,False,False,False, +scenario_073,MI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_073,MI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_074,LA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_074,LA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,federal_income_tax_before_refundable_credits,14777.1455078125,14777.1455078125,True,14777.1455078125,0.0,False,False,False, +scenario_075,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,payroll_tax,968.2120971679688,968.2120971679688,True,968.2120971679688,0.0,False,False,False, +scenario_075,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_075,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,federal_income_tax_before_refundable_credits,19611.80078125,19611.80078125,True,19611.80078125,0.0,False,False,False, +scenario_076,ID,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,payroll_tax,12240.0,12240.0,True,12240.0,0.0,False,False,False, +scenario_076,ID,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_076,ID,state_income_tax_before_refundable_credits,6818.34423828125,6818.34423828125,True,6818.34423828125,0.0,False,False,False, +scenario_076,ID,state_refundable_credits,465.0,465.0,True,465.0,0.0,False,False,False, +scenario_076,ID,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,federal_income_tax_before_refundable_credits,690.6175537109375,690.6175537109375,True,690.6175537109375,0.0,False,False,False, +scenario_077,LA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,payroll_tax,1989.472412109375,1989.472412109375,True,1989.472412109375,0.0,False,False,False, +scenario_077,LA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,state_income_tax_before_refundable_credits,305.13525390625,305.13525390625,True,305.13525390625,0.0,False,False,False, +scenario_077,LA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_077,LA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,federal_income_tax_before_refundable_credits,24164.45703125,24164.45703125,True,24164.45703125,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_078,MD,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,payroll_tax,14317.322265625,14317.322265625,True,14317.322265625,0.0,False,False,False, +scenario_078,MD,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,state_income_tax_before_refundable_credits,6936.87646484375,6936.87646484375,True,6936.87646484375,0.0,False,False,False, +scenario_078,MD,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_078,MD,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_079,AZ,spouse_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_079,AZ,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_079,AZ,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,snap,2376.0,2376.0,True,2376.0,0.0,False,False,False, +scenario_079,AZ,ssi,10572.0,10572.0,True,10572.0,0.0,False,False,False, +scenario_079,AZ,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_079,AZ,state_refundable_credits,50.0,50.0,True,50.0,0.0,False,False,False, +scenario_079,AZ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_080,PA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,snap,3596.039794921875,3576.0,False,3576.0,0.0,False,False,True,reference_engine_defect +scenario_080,PA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_080,PA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,federal_income_tax_before_refundable_credits,26932.2734375,26932.2734375,True,26932.2734375,0.0,False,False,False, +scenario_081,MA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,payroll_tax,14192.662109375,14192.662109375,True,13387.65234375,-805.009765625,True,True,False, +scenario_081,MA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,state_income_tax_before_refundable_credits,8238.40625,8238.40625,True,8238.40625,0.0,False,False,True,reference_engine_defect +scenario_081,MA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_081,MA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,federal_income_tax_before_refundable_credits,9563.052734375,9577.90234375,False,9577.90234375,0.0,False,False,True,reference_engine_defect +scenario_082,NY,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,payroll_tax,8108.02734375,8108.02734375,True,7664.91748046875,-443.10986328125,True,True,False, +scenario_082,NY,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_082,NY,state_income_tax_before_refundable_credits,5598.55224609375,5598.55224609375,True,5598.55224609375,0.0,False,False,False, +scenario_082,NY,state_refundable_credits,667.0,667.0,True,667.0,0.0,False,False,False, +scenario_082,NY,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,payroll_tax,539.3250122070312,539.3250122070312,True,539.3250122070312,0.0,False,False,False, +scenario_083,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_083,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_084,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_084,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,federal_income_tax_before_refundable_credits,2322.760009765625,2322.760009765625,True,2322.760009765625,0.0,False,False,False, +scenario_085,PA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_085,PA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_085,PA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,federal_income_tax_before_refundable_credits,3255.5478515625,3255.5478515625,True,3255.5478515625,0.0,False,False,False, +scenario_086,GA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,payroll_tax,4743.0,4743.0,True,4743.0,0.0,False,False,False, +scenario_086,GA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,state_income_tax_before_refundable_credits,2078.958740234375,2078.958740234375,True,2078.958740234375,0.0,False,False,False, +scenario_086,GA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_086,GA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,payroll_tax,1421.1448974609375,1421.1448974609375,True,1421.1448974609375,0.0,False,False,False, +scenario_088,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_088,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,federal_income_tax_before_refundable_credits,6819.7119140625,6819.7119140625,True,6819.7119140625,0.0,False,False,False, +scenario_089,NC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,payroll_tax,5039.91015625,5039.91015625,True,5039.91015625,0.0,False,False,False, +scenario_089,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,self_employment_tax,4286.24072265625,4286.24072265625,True,4286.24072265625,0.0,False,False,False, +scenario_089,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,state_income_tax_before_refundable_credits,3219.935302734375,3219.935302734375,True,3219.935302734375,0.0,False,False,False, +scenario_089,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_089,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,payroll_tax,535.5,535.5,True,535.5,0.0,False,False,False, +scenario_090,KS,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_090,KS,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,federal_income_tax_before_refundable_credits,1349.998779296875,1349.998779296875,True,1349.998779296875,0.0,False,False,False, +scenario_091,WI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,payroll_tax,2065.5,2065.5,True,2065.5,0.0,False,False,False, +scenario_091,WI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,state_income_tax_before_refundable_credits,843.6614990234375,895.8131103515625,False,895.8131103515625,0.0,False,False,True,reference_engine_defect +scenario_091,WI,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_091,WI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_092,AL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_092,AL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,federal_income_tax_before_refundable_credits,8628.990234375,8628.990234375,True,8628.990234375,0.0,False,False,False, +scenario_093,MO,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,payroll_tax,12469.5,12469.5,True,12469.5,0.0,False,False,False, +scenario_093,MO,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,dependent1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,dependent1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,dependent1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,dependent1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,state_income_tax_before_refundable_credits,3388.24560546875,3388.24560546875,True,3388.24560546875,0.0,False,False,False, +scenario_093,MO,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_093,MO,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_095,NJ,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,self_employment_tax,593.2291259765625,593.2291259765625,True,593.2291259765625,0.0,False,False,False, +scenario_095,NJ,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_095,NJ,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_098,IL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_098,IL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,federal_income_tax_before_refundable_credits,10679.75,10679.75,True,10679.75,0.0,False,False,True,reference_engine_defect +scenario_099,CA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_099,CA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,payroll_tax,14767.5,14767.5,True,14767.5,0.0,False,False,False, +scenario_099,CA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,state_income_tax_before_refundable_credits,4493.7431640625,4640.77734375,False,4640.77734375,0.0,False,False,True,reference_engine_defect +scenario_099,CA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_099,CA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,federal_refundable_credits,2878.0966796875,2878.0966796875,True,2878.0966796875,0.0,False,False,True,reference_engine_defect +scenario_100,MT,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_100,MT,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,payroll_tax,452.47613525390625,452.47613525390625,True,452.47613525390625,0.0,False,False,False, +scenario_100,MT,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child2_head_start_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_100,MT,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_100,MT,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_100,MT,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_100,MT,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,snap,8625.8896484375,8556.0,False,8556.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_100,MT,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_100,MT,state_refundable_credits,473.1777038574219,473.1777038574219,True,473.1777038574219,0.0,False,False,True,reference_engine_defect +scenario_100,MT,tanf,6064.958984375,6064.958984375,True,6064.958984375,0.0,False,False,False, +scenario_101,TX,federal_income_tax_before_refundable_credits,13418.599609375,13418.599609375,True,13418.599609375,0.0,False,False,False, +scenario_101,TX,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,payroll_tax,7879.5,7879.5,True,7879.5,0.0,False,False,False, +scenario_101,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_101,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,federal_income_tax_before_refundable_credits,6104.09619140625,6104.09619140625,True,6104.09619140625,0.0,False,False,False, +scenario_102,NC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,payroll_tax,6670.7998046875,6670.7998046875,True,6670.7998046875,0.0,False,False,False, +scenario_102,NC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,state_income_tax_before_refundable_credits,2461.869873046875,2461.869873046875,True,2461.869873046875,0.0,False,False,False, +scenario_102,NC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_102,NC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_104,NY,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_104,NY,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_107,OH,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,payroll_tax,486.9607238769531,486.9607238769531,True,486.9607238769531,0.0,False,False,False, +scenario_107,OH,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_107,OH,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_107,OH,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,payroll_tax,585.6839599609375,585.6839599609375,True,585.6839599609375,0.0,False,False,False, +scenario_108,WI,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_108,WI,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,snap,288.0,288.0,True,288.0,0.0,False,False,False, +scenario_108,WI,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_108,WI,state_refundable_credits,271.3916015625,271.3916015625,True,271.3916015625,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_108,WI,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,federal_refundable_credits,12038.0859375,12038.0859375,True,12038.0859375,0.0,False,False,False, +scenario_109,FL,free_school_meals_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child3_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child3_early_head_start_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child3_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child1_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,child2_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,child3_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child3_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,child3_wic_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_109,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,self_employment_tax,4238.86474609375,4238.86474609375,True,4238.86474609375,0.0,False,False,False, +scenario_109,FL,snap,7932.0,7932.0,True,7932.0,0.0,False,False,False, +scenario_109,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_109,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,federal_income_tax_before_refundable_credits,23897.443359375,23897.443359375,True,23897.443359375,0.0,False,False,True,reference_engine_defect +scenario_110,OH,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,payroll_tax,7650.0,7650.0,True,7650.0,0.0,False,False,False, +scenario_110,OH,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,state_income_tax_before_refundable_credits,4057.47216796875,4057.47216796875,True,4057.47216796875,0.0,False,False,True,reference_engine_defect +scenario_110,OH,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_110,OH,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_111,WA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_111,WA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,federal_refundable_credits,453.0960083007813,453.09600830078125,True,453.09600830078125,0.0,False,False,False, +scenario_112,TX,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,payroll_tax,453.0960083007813,453.09600830078125,True,453.09600830078125,0.0,False,False,False, +scenario_112,TX,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,snap,287.68316650390625,0.0,False,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_112,TX,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_112,TX,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,federal_income_tax_before_refundable_credits,10729.6123046875,10729.6123046875,True,10729.6123046875,0.0,False,False,False, +scenario_114,VA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,payroll_tax,1736.550048828125,1736.550048828125,True,1736.550048828125,0.0,False,False,False, +scenario_114,VA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_114,VA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,state_income_tax_before_refundable_credits,3514.14501953125,3514.14501953125,True,3514.14501953125,0.0,False,False,False, +scenario_114,VA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_114,VA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_115,AL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,state_income_tax_before_refundable_credits,4.679999828338623,4.679999828338623,True,4.679999828338623,0.0,False,False,False, +scenario_115,AL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_115,AL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_116,FL,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,federal_income_tax_before_refundable_credits,24961.33984375,24961.33984375,True,24961.33984375,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_117,AR,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,payroll_tax,15044.787109375,15044.787109375,True,15044.787109375,0.0,False,False,False, +scenario_117,AR,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,child3_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,state_income_tax_before_refundable_credits,7984.53369140625,7984.53369140625,True,7984.53369140625,0.0,False,False,False, +scenario_117,AR,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_117,AR,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,head_medicaid_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_118,NY,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_118,NY,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,snap,2903.9404296875,2868.0,False,2868.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_118,NY,ssi,9368.0,9368.0,True,9368.0,0.0,False,False,False, +scenario_118,NY,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_118,NY,state_refundable_credits,375.0,375.0,True,375.0,0.0,False,False,False, +scenario_118,NY,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,federal_refundable_credits,2854.0087890625,2854.0087890625,True,2854.0087890625,0.0,False,False,True,reference_engine_defect +scenario_119,VA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,payroll_tax,4207.5,4207.5,True,4207.5,0.0,False,False,False, +scenario_119,VA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,child2_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,state_income_tax_before_refundable_credits,1859.0943603515625,1859.0943603515625,True,1859.0943603515625,0.0,False,False,True,reference_engine_defect +scenario_119,VA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_119,VA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,federal_income_tax_before_refundable_credits,40021.81640625,40416.66796875,False,40416.66796875,0.0,False,False,True,reference_engine_defect +scenario_120,CT,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,payroll_tax,13496.1552734375,13496.1552734375,True,13496.1552734375,0.0,False,False,False, +scenario_120,CT,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_120,CT,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,state_income_tax_before_refundable_credits,11917.18359375,11917.18359375,True,11917.18359375,0.0,False,False,True,reference_engine_defect +scenario_120,CT,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_120,CT,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,federal_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_121,SC,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,payroll_tax,1963.3724365234375,1963.3724365234375,True,1963.3724365234375,0.0,False,False,False, +scenario_121,SC,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,state_income_tax_before_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,True,reference_depends_on_unlisted_input +scenario_121,SC,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_121,SC,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,federal_income_tax_before_refundable_credits,4746.66162109375,4746.66162109375,True,4746.66162109375,0.0,False,False,False, +scenario_122,MN,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,payroll_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,head_medicare_eligible,1.0,1.0,True,1.0,0.0,False,False,False, +scenario_122,MN,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,state_income_tax_before_refundable_credits,1025.54150390625,1025.54150390625,True,1025.54150390625,0.0,False,False,False, +scenario_122,MN,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_122,MN,tanf,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,federal_income_tax_before_refundable_credits,5200.240234375,5200.240234375,True,5200.240234375,0.0,False,False,False, +scenario_123,PA,federal_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,free_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,local_income_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,payroll_tax,11194.0,11194.0,True,11194.0,0.0,False,False,False, +scenario_123,PA,head_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,spouse_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_chip_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_early_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_head_start_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,head_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,spouse_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_medicaid_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,head_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,spouse_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_medicare_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,head_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,spouse_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,child1_wic_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,reduced_price_school_meals_eligible,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,self_employment_tax,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,snap,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,ssi,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,state_income_tax_before_refundable_credits,3070.061279296875,3070.061279296875,True,3070.061279296875,0.0,False,False,False, +scenario_123,PA,state_refundable_credits,0.0,0.0,True,0.0,0.0,False,False,False, +scenario_123,PA,tanf,0.0,0.0,True,0.0,0.0,False,False,False, diff --git a/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json new file mode 100644 index 00000000..e1ed4865 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json @@ -0,0 +1,506 @@ +{ + "adapter_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "households": { + "scenario_000": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_001": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "VA" + }, + "scenario_002": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WA" + }, + "scenario_003": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_004": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NY" + }, + "scenario_005": { + "employee_state_payroll_tax_final": 5590.0, + "employee_state_payroll_tax_scoped": 5590.0, + "state": "CA" + }, + "scenario_007": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "ID" + }, + "scenario_008": { + "employee_state_payroll_tax_final": 112.56000518798828, + "employee_state_payroll_tax_scoped": 112.56000518798828, + "state": "NJ" + }, + "scenario_009": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_012": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MS" + }, + "scenario_013": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AZ" + }, + "scenario_014": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WV" + }, + "scenario_015": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "IN" + }, + "scenario_016": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_018": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AZ" + }, + "scenario_020": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_021": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MO" + }, + "scenario_022": { + "employee_state_payroll_tax_final": 684.2550048828125, + "employee_state_payroll_tax_scoped": 684.2550048828125, + "state": "CA" + }, + "scenario_023": { + "employee_state_payroll_tax_final": 226.75440979003906, + "employee_state_payroll_tax_scoped": 226.75440979003906, + "state": "CA" + }, + "scenario_025": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OH" + }, + "scenario_026": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_027": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "CT" + }, + "scenario_028": { + "employee_state_payroll_tax_final": 42.0, + "employee_state_payroll_tax_scoped": 42.0, + "state": "PA" + }, + "scenario_029": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OK" + }, + "scenario_030": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_031": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "CA" + }, + "scenario_032": { + "employee_state_payroll_tax_final": 127.5999984741211, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MN" + }, + "scenario_033": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "SD" + }, + "scenario_036": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NJ" + }, + "scenario_037": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_038": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "LA" + }, + "scenario_039": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "VA" + }, + "scenario_040": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AZ" + }, + "scenario_042": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WI" + }, + "scenario_043": { + "employee_state_payroll_tax_final": 17.98784637451172, + "employee_state_payroll_tax_scoped": 0.0, + "state": "CO" + }, + "scenario_044": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "KS" + }, + "scenario_045": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MI" + }, + "scenario_046": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OK" + }, + "scenario_048": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OH" + }, + "scenario_049": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NH" + }, + "scenario_051": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "LA" + }, + "scenario_052": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_053": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "ID" + }, + "scenario_054": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_055": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_056": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NJ" + }, + "scenario_057": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "LA" + }, + "scenario_059": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_060": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_062": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_064": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WI" + }, + "scenario_066": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "VA" + }, + "scenario_067": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "IN" + }, + "scenario_068": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MD" + }, + "scenario_070": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "IL" + }, + "scenario_071": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NY" + }, + "scenario_072": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MI" + }, + "scenario_073": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MI" + }, + "scenario_074": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "LA" + }, + "scenario_075": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_076": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "ID" + }, + "scenario_077": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "LA" + }, + "scenario_078": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MD" + }, + "scenario_079": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AZ" + }, + "scenario_080": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "PA" + }, + "scenario_081": { + "employee_state_payroll_tax_final": 805.0092163085938, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MA" + }, + "scenario_082": { + "employee_state_payroll_tax_final": 443.1100158691406, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NY" + }, + "scenario_083": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_084": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_085": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "PA" + }, + "scenario_086": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "GA" + }, + "scenario_088": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_089": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_090": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "KS" + }, + "scenario_091": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WI" + }, + "scenario_092": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AL" + }, + "scenario_093": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MO" + }, + "scenario_095": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NJ" + }, + "scenario_098": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "IL" + }, + "scenario_099": { + "employee_state_payroll_tax_final": 2145.0, + "employee_state_payroll_tax_scoped": 2145.0, + "state": "CA" + }, + "scenario_100": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MT" + }, + "scenario_101": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_102": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NC" + }, + "scenario_104": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NY" + }, + "scenario_107": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OH" + }, + "scenario_108": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WI" + }, + "scenario_109": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_110": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "OH" + }, + "scenario_111": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "WA" + }, + "scenario_112": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "TX" + }, + "scenario_114": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "VA" + }, + "scenario_115": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AL" + }, + "scenario_116": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "FL" + }, + "scenario_117": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "AR" + }, + "scenario_118": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "NY" + }, + "scenario_119": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "VA" + }, + "scenario_120": { + "employee_state_payroll_tax_final": 827.9849853515625, + "employee_state_payroll_tax_scoped": 827.9849853515625, + "state": "CT" + }, + "scenario_121": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "SC" + }, + "scenario_122": { + "employee_state_payroll_tax_final": 0.0, + "employee_state_payroll_tax_scoped": 0.0, + "state": "MN" + }, + "scenario_123": { + "employee_state_payroll_tax_final": 101.5, + "employee_state_payroll_tax_scoped": 101.5, + "state": "PA" + } + }, + "policyengine_us": "2.15.17" +} From 088929ce227a7071533ca7178cf44a288397036f Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Mon, 5 Oct 2026 12:52:10 -0400 Subject: [PATCH 2/3] Rest the payroll recommendation on the reference rules; record the independent review An eight-agent review of 8714770 (differential impact and sweep, mechanism, advocates for keep and regenerate with a judge, release checklist, claim audit) agreed on every number: 2,438 impact values, all 1,984 swept outputs bit for bit, and the 36 hand-computed payroll parts. The judge upheld exclusion but struck the draft's board-effect and model-behaviour arguments, which no rule uses. - README: the recommendation now rests on 2026-09-22/09-28 rule 4 and the r24, r25 and r14 precedents; regeneration is the strict reading of "mandatory". Adds both options' board effects side by side, the counter-evidence the reviewers found, the Acts 2026 c. 101 line mapping, the review summary and the release checklist. Fixes the claim audit's five errors and thirteen imprecisions. - proposed_exclusions.json: one shared unlisted input; the basis cites the rules and precedents (no engine input exists for the employer's choice). - payroll_mandatory_scope.py: docstring states its scope (DE, ME, VT and WA left for upstream). Sweep, proposals and impact rerun on the new hash; values unchanged. - verification/independent_reviews.json: the eight reports. Changes no reference or published number. Co-Authored-By: Claude Opus 5.5 --- reference_audit/2026-10-05-payroll/README.md | 171 ++-- .../fixes/payroll_mandatory_scope.py | 16 +- .../program_classification.json | 10 +- .../proposed_exclusions.json | 26 +- .../proposed_regenerations.json | 14 +- .../scripts/propose_changes.py | 57 +- .../verification/independent_reviews.json | 850 ++++++++++++++++++ .../sweep_payroll_scope_households.json | 2 +- 8 files changed, 1036 insertions(+), 110 deletions(-) create mode 100644 reference_audit/2026-10-05-payroll/verification/independent_reviews.json diff --git a/reference_audit/2026-10-05-payroll/README.md b/reference_audit/2026-10-05-payroll/README.md index f0318a1b..ed0dd568 100644 --- a/reference_audit/2026-10-05-payroll/README.md +++ b/reference_audit/2026-10-05-payroll/README.md @@ -1,25 +1,26 @@ # Optional employer pass-through in the payroll tax references, October 5, 2026 -This directory audits the state part of PolicyBench's US `payroll_tax` references in release `dashboard-data-20260930`. Four scored references count an employee share of a state paid-leave premium that the law lets the employer deduct but does not require. The directory proposes two alternative records for those four outputs, excluding them or regenerating them, and changes no reference. Either one changes published scores, so they wait for Max's ruling (cos decision DECISION_ID). +This directory audits the state part of PolicyBench's US `payroll_tax` references in release `dashboard-data-20260930`. Four scored references count an employee share of a state paid-leave or disability premium that the law lets the employer deduct but does not require. The directory proposes two alternative records for those four outputs, excluding them or regenerating them, and changes no reference. Either one changes published scores, so they wait for Max's ruling (cos decision DECISION_ID). ## Recommendation -Exclude the four outputs (scenario_032 MN, scenario_043 CO, scenario_081 MA, scenario_082 NY) from scoring in the next release, with `proposed_exclusions.json`. In the next prompt version, state how optional employee shares count, so these households can be scored again on a fresh run. +Exclude the four outputs (scenario_032 MN, scenario_043 CO, scenario_081 MA, scenario_082 NY) from scoring in the next release, with `proposed_exclusions.json`. Then state in the payroll output's definition how these shares count, so a fresh run can score such households again; until then the board has no scored test of the line between mandatory and optional employee shares. -- **Keep fails.** Under each state's law the premium is the employer's, and the employee pays only what the employer chooses to deduct, from zero up to a cap (16 of 16 independent reviews agree). The frozen reference assumes the cap. policyengine-us documents that assumption for Minnesota and Massachusetts ("assuming the employer withholds the maximum permitted employee share"), and the prompt never states the employer's choice. The reference therefore rests on a fact the prompt does not give. -- **Regenerate picks a side of a real split.** The prompt asks for "mandatory employee state payroll taxes" and says to exclude employer payroll taxes. On that wording the frozen value is too high. But the same laws speak of the employee share in mandatory terms: Minnesota's statute says employees "must pay the remaining portion, if any", New York's says every employee "shall contribute", and Massachusetts tells employers they "must withhold PFML contributions". 22 of 46 models included the Massachusetts and New York contributions. That is the case the exclusion rule covers: a careful reader could take the stated facts the other way. -- **Regenerating would mostly reward answers that never engaged with the program.** Across the four outputs it would newly credit 82 answers. One gives the optional-deduction reason; 36 say the state has no employee payroll tax and 39 never mention a state program, the same pattern as on Pennsylvania's mandatory contribution, where 17 and 16 models say the state has none. It would take credit from 35 answers that include the program at exactly the engine's amount. -- **Exclusion is the symmetric remedy and moves the board least.** GPT-6 Sol stays first (95.00% to 95.09%); five adjacent pairs swap. Regeneration moves GPT-6 Sol to 94.63% and puts Claude Opus 5.5 first on the within-1% metric. +- **Keep fails the repo's test.** A scored reference must follow from the stated facts under the law (2026-09-22 rule 1). All eight reviews of these four programs find that the premium is the employer's and that the law requires no amount of the employee: the employee pays only what the employer chooses to deduct, from $0 up to a cap. The frozen references count the cap. policyengine-us states that assumption for Minnesota and Massachusetts ("assuming the employer withholds the maximum permitted employee share"), and the prompt never states the employer's choice. The judges who annotated the 149 wrong answers on these outputs flagged no reference, but they were not asked to research the deduction law. +- **The reference turns on a definition the prompt does not settle.** "Mandatory employee state payroll taxes" has two defensible readings here. One is the amount the law requires of the employee whatever the employer does: $0, the regenerated value. The other is the employee share the statute sets, withheld unless the employer elects to pay it: the frozen value. The statutes frame the employer's choice both ways. Colorado treats an employer that deducts less as having "elected to pay" the rest; Minnesota has employees pay "the remaining portion, if any" above the employer's floor; New York says every employee "shall contribute ... to the extent and in the manner herein provided" and then only authorizes collection. The prompt's default for unlisted facts (treat them as false) cannot choose between the readings, because each statute frames a different act as the choice. A reference that turns on "an input or definition the prompt never states" is excluded (2026-09-22 rule 4; 2026-09-28 rule 4). The closest precedents were excluded under the same reason code: r24, who paid for the coverage behind listed disability benefits, an employer-arrangement fact; r25, whether the federal output includes the net investment income tax, a definition question; and r14, an unlisted 40 weekly hours, where the audit excluded rather than apply the zero default. +- **Regenerate is the alternative if Max reads "mandatory" strictly.** On that reading the frozen values are too high, and the regenerated value is the only one that no caveat in the law record moves: Acts 2026 c. 101, private plans, collective agreements and employer size each change the cap, never the $0 floor. `proposed_regenerations.json` is ready for that ruling. The Maryland output-scope adapter (2026-09-28 rule 3) is the form it would take, but not a precedent that decides the case: there the definition was explicit ("excluding local income ... taxes") and the adapter undid an engine change made after the freeze, while here the definition is the contested part and regeneration would change values that have stood since the first run. + +Board and model effects are not criteria in the rules, so they do not decide this; both options' effects are reported below. ## How the reference is built -These files hold for policyengine-us 2.15.17, as installed in the reference venv, plus `latest_final`, the system that built the published references. +These facts hold for policyengine-us 2.15.17, as installed in the reference venv, plus `latest_final`, the system that built the published references. - `payroll_tax` is `spm_unit_payroll_tax` (`policybench/benchmark_specs.json`), which sums `employee_payroll_tax` over the SPM unit's tax units (`variables/household/expense/tax/spm_unit_payroll_tax.py`). -- `employee_payroll_tax` adds employee Social Security tax, employee Medicare tax, Additional Medicare Tax and `employee_state_payroll_tax` (`variables/gov/irs/tax/payroll/employee_payroll_tax.py`). +- `employee_payroll_tax` adds employee Social Security tax, employee Medicare tax, Additional Medicare Tax and `employee_state_payroll_tax` (`variables/gov/irs/tax/payroll/employee_payroll_tax.py`), unless the contributed `abolish_payroll_tax` parameter is set (it is not). - `employee_state_payroll_tax` adds one `_employee_state_payroll_tax` per state for 14 states. Its documentation reads "Employee-side mandatory state payroll taxes and payroll-funded contributions" (`variables/gov/states/tax/payroll/employee_state_payroll_tax.py`). -- Each state aggregate adds its programs' employee contributions. For Minnesota, Massachusetts, Delaware, Maine and Vermont, the contribution's documentation says it assumes "the employer withholds the maximum permitted employee share". Colorado's, New York's and Washington's say no such thing, though each computes the largest permitted share. -- Nothing else in the engine reads these contributions except through `employee_payroll_tax`, which also feeds `spm_unit_paycheck_withholdings` and `household_tax_before_refundable_credits`. No PolicyBench output other than `payroll_tax` moves when they change (the sweep below). +- Each state aggregate adds its programs' employee contributions. For Minnesota, Massachusetts, Delaware, Maine and Vermont, the contribution's documentation says it assumes "the employer withholds the maximum permitted employee share". Colorado's, New York's and Washington's say no such thing, though each computes the largest permitted share. The engine has an `employer_headcount` input, which sets employer-side contributions in several states and Delaware's contribution rate, but no input for the employer's deduction choice. +- The five optional contributions are read only by their state aggregates. `employee_payroll_tax` also feeds `spm_unit_paycheck_withholdings`, `household_tax_before_refundable_credits`, Alabama's itemized deductions, `fica_marginal_tax_rate` and `taxsim_tfica`, and California SDI also feeds Santa Clara County general assistance. No PolicyBench output other than `payroll_tax` moves when the five change (the sweep below). ## Every payroll reference with a state component @@ -40,31 +41,37 @@ These files hold for policyengine-us 2.15.17, as installed in the reference venv | scenario_081 | MA PFML | 805.01 | 13,387.65 | 14,192.66 | **optional** | scored; proposed | | scenario_082 | NY PFL 411.91 + DBL 31.20 | 443.11 | 7,664.92 | 8,108.03 | **optional** | scored; proposed | -Every household with wages in a state with a modeled program carries that program; no benchmark household lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington. The four proposed references have had these values since the first run (2026-06-15). +Amounts are the engine's (float32): Connecticut's 0.5% of $165,597 is $827.985, which the engine stores as $827.98. Every household with wages in a state with a modeled program carries that program. No household with wages lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington; two Washington households (scenario_002 and scenario_111) have none. The four proposed references have had these values since the first run (references generated 2026-06-12, committed 2026-06-15). ## The law, program by program -The test is who owes the contribution. A **mandatory** employee contribution is one the statute puts on the worker and requires the employer to withhold. An **optional employer pass-through** is a premium the statute puts on the employer, which may deduct up to a share from wages and may instead pay it, so the employee amount runs from zero to a cap at the employer's choice. `program_classification.json` has each program's citation, 2026 rate and engine amounts; `verification/law_research.json` has every record, quote and URL. +The test is who owes the contribution. A **mandatory** employee contribution is one the statute puts on the worker and requires the employer to withhold. An **optional employer pass-through** is a premium the statute puts on the employer, which may deduct up to a share from wages and may instead pay it, so the employee amount runs from $0 to a cap at the employer's choice. `program_classification.json` has each program's citation, 2026 rate and engine amounts; `verification/law_research.json` has every record, quote, URL and verdict. | Program | Who owes it | Employee share | Class | Primary source | |---|---|---|---|---| -| MN Paid Leave | Premiums "accrue and become payable by each employer"; employer pays at least 50% | Employees pay "the remaining portion, if any, of the premium not paid by the employer": 0 to 0.44% | optional | [Minn. Stat. 268B.14](https://www.revisor.mn.gov/statutes/cite/268B.14) subds. 1, 3, 5a | -| CO FAMLI | Employer "shall remit" premiums | Employer "may deduct up to 50 percent" (0.44%); one that deducts less "is considered to have elected to pay" it | optional | [C.R.S. 8-13.3-507](https://olls.info/crs/crs2025-title-08.pdf)(2), (5); 7 CCR 1107-1 s. 1.4(6)(A) | -| MA PFML | Employer "shall remit" contributions | Employer may deduct up to 100% of family and 40% of medical (0.46%) and may deduct less | optional | [M.G.L. c. 175M, s. 6](https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6)(a), (c); 458 CMR 2.05(5)(c) | -| NY PFL and DBL | Covered employer provides the coverage and pays its cost beyond what it collects | Employer "is authorized to collect" up to the caps (PFL 0.432%, at most $411.91; DBL $0.60 a week) | optional | [N.Y. WCL s. 209](https://www.nysenate.gov/legislation/laws/WKC/209)(3)-(4), s. 210(1) | -| CT Paid Leave | Each employee "shall contribute" | Employer "shall deduct and withhold" 0.5%; no employer share | mandatory | [Conn. Gen. Stat. 31-49g](https://www.cga.ct.gov/2026/sup/chap_557.htm)(b)(1), (3) | -| CA SDI | Each worker "shall pay worker contributions" | Employer must withhold 1.3% in trust; one that pays it instead still owes the worker's contribution, and the payment counts as wages | mandatory | [Cal. UIC s. 984](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.)(a)(1), s. 986(a) | -| PA UC | Each employee "shall pay contributions" | Employer "shall withhold, in trust" 0.07% of all wages | mandatory | [UC Law s. 301.4](https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3)(a)-(b) | -| NJ TDI and FLI | Each worker "shall contribute" | Employer "shall ... withhold in trust" (TDI 0.19%, FLI 0.23%) | mandatory | [N.J.S.A. 43:21-7](https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf)(d)(1)(E), (G) | +| MN Paid Leave | Premiums "accrue and become payable by each employer"; the employer pays at least 50% (small employers: at least 0.22% of a 0.66% premium) | Employees pay "the remaining portion, if any, of the premium not paid by the employer": $0 to 0.44% | optional | [Minn. Stat. 268B.14](https://www.revisor.mn.gov/statutes/cite/268B.14) subds. 1, 3, 5a | +| CO FAMLI | The employer "shall remit" premiums | The employer "may deduct up to 50 percent" (0.44%); one that deducts less "is considered to have elected to pay" it | optional | [C.R.S. 8-13.3-507](https://olls.info/crs/crs2025-title-08.pdf)(2), (5); 7 CCR 1107-1 s. 1.4(6)(A) | +| MA PFML | The employer "shall remit" contributions | The employer may deduct up to 100% of family and 40% of medical (0.46%) and may deduct less | optional | [M.G.L. c. 175M, s. 6](https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6)(a), (c); 458 CMR 2.05(5)(c) | +| NY PFL and DBL | The covered employer provides the coverage; it pays the DBL cost beyond what it collects, and no employer is required to fund PFL | The employer "is authorized to collect" up to the caps (PFL 0.432%, at most $411.91; DBL $0.60 a week) | optional | [N.Y. WCL s. 209](https://www.nysenate.gov/legislation/laws/WKC/209)(3)-(4), s. 210(1) | +| CT Paid Leave | Each employee "shall contribute" | The employer "shall deduct and withhold" 0.5%; there is no employer share | mandatory | [Conn. Gen. Stat. 31-49g](https://www.cga.ct.gov/2026/sup/chap_557.htm)(b)(1), (3) | +| CA SDI | Each worker "shall pay worker contributions" | The employer must withhold 1.3% in trust; one that pays it instead still owes the worker's contribution, and the payment counts as wages | mandatory | [Cal. UIC s. 984](https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=UIC§ionNum=984.)(a)(1), s. 986(a) | +| PA UC | Each employee "shall pay contributions" | The employer "shall withhold, in trust" 0.07% of all wages | mandatory | [UC Law s. 301.4](https://www.palegis.us/statutes/unconsolidated/law-information/view-statute?iFrame=true&txtType=HTM&yr=1936&sessInd=2&smthLwInd=2&act=001&chpt=3)(a)-(b) | +| NJ TDI and FLI | Each worker "shall contribute" | The employer "shall ... withhold in trust" (TDI 0.19%, FLI 0.23%) | mandatory | [N.J.S.A. 43:21-7](https://www.nj.gov/labor/myunemployment/assets/pdfs/UI_statute.pdf)(d)(1)(E), (G) | + +One researcher per program read the statute, rules, agency pages and engine source; then two reviewers per program, one on the statute and one on the rules and agency guidance, tried to refute each classification, re-fetching the sources and re-running the engine amounts. None of the 16 reviews refuted a classification; eight of them cover the four optional programs. All ran as separate agents in one Claude Code workflow on 2026-10-05, with no reviewer outside it. -For the record of an upstream fix, the same test classifies the modeled programs no benchmark household reaches; these were researched once and not adversarially reviewed. Delaware Paid Leave, Maine PFML and the Vermont Child Care Contribution are optional pass-throughs; Washington Cares, Paid Leave Oregon, the Oregon statewide transit tax and Rhode Island TDI are mandatory; Washington PFML is mixed (the employer "may deduct" the employee share and "may elect to pay" it). +For the record of an upstream fix, the same test classifies the modeled programs that reach no wage-earning household; these were researched once and not reviewed. Delaware Paid Leave, Maine PFML and the Vermont Child Care Contribution are optional pass-throughs; Washington Cares, Paid Leave Oregon, the Oregon statewide transit tax and Rhode Island TDI are mandatory; Washington PFML is mixed (the employer "may deduct" the employee share and "may elect to pay" it). -Every 2026 rate, share and cap above was published before the 2026-07-03 reference freeze, and each engine amount equals the law's largest employee share to the cent. Points that cut the other way, all recorded in `verification/law_research.json`: +Every 2026 rate, share and cap above was published before the 2026-07-03 reference freeze. Each engine amount equals the law's largest employee share within a cent, with two qualifications: New York's DBL cap is $0.60 a week, which the engine annualizes over 52 weeks ($31.20; a weekly payroll with 53 paydays in 2026 could reach $31.80), and the Massachusetts cap is subject to Acts 2026 c. 101 (below). -- **Minnesota.** Subd. 3 says employees "must pay the remaining portion, if any", and subd. 5a(b)(2) that the employer "must make wage deductions as necessary". The remainder is whatever the employer leaves above its 50% floor, and the Department of Employment and Economic Development tells employers they "can deduct up to 0.44%" and "may choose to pay more than their required portion". -- **New York.** WCL s. 209(1) says every employee "shall contribute ... to the extent and in the manner herein provided"; s. 209(4) then only authorizes collection, and the Workers' Compensation Board says an employer "is allowed, but not required" to collect. -- **Massachusetts.** The Department of Family and Medical Leave's overview says employers "must withhold PFML contributions from employees' paychecks"; the same page says they "may choose to cover a larger share", and 458 CMR 2.05(5)(c) covers an employer that "opts to deduct a lower percentage". Acts 2026 c. 101 (approved 2026-06-12, before the freeze) amends the deduction caps in s. 6. The two reviewers read its scope differently, and its s. 45 applies it to taxable years beginning on or after 2026-01-01, while the department, after the freeze, dates the change to 2027. If it reached 2026, the most a large employer could deduct for scenario_081 would be $1,351.02 (0.772%) rather than $805.01. The share stays optional either way, so this matters only if the reference is kept. -- **California.** An employer may pay a worker's SDI instead of deducting it (s. 986(a)(2)), but the contribution remains the worker's and the payment counts as wages, the same structure as an employer paying the employee's FICA. +What points the other way, all recorded in `verification/law_research.json`: + +- **Minnesota.** Subd. 3 says employees "must pay the remaining portion, if any", and subd. 5a(b)(2) that the employer "must make wage deductions as necessary". The Paid Leave office says that if an employer pays more than its required minimum, "this is additional compensation to the employee and is included in the employee's federal gross income as wages". The same office tells employers they "can deduct up to 0.44%" and "may choose to pay more than their required portion". +- **Colorado.** The FAMLI Division's employer page says "Most Colorado businesses should be deducting FAMLI premiums from all employees on their payroll", and its FAQ tells employers to "collect and remit the 0.44% of the employee's share". The same page says some employers "may choose not to deduct any premiums", and the statute says "may deduct". +- **Massachusetts.** The Department of Family and Medical Leave says employers "must withhold PFML contributions from employees' paychecks" and "Most Massachusetts employers must make payroll withholdings"; c. 175M s. 1 counts payments by an employee as contributions, and s. 4(a)(ii) requires notice of the employee's contribution amount. The same page says employers "may choose to cover a larger share", and 458 CMR 2.05(5)(c) covers an employer that "opts to deduct a lower percentage". +- **New York.** WCL s. 209(1) says every employee "shall contribute to the cost of providing disability and ... family leave benefits", the Department of Financial Services' 2026 rate decision says "Employers shall collect employee contributions consistent with this Decision", and the Workers' Compensation Board calls Paid Family Leave "fully paid for by employees". The same Board says an employer "is allowed, but not required" to collect, and s. 209(4) only authorizes collection. +- **California, for contrast.** An employer may pay a worker's SDI instead of deducting it (s. 986(a)(2)), but the worker still owes the contribution and the payment is grossed up as wages. In Minnesota an employer that pays more than its floor leaves the employee nothing owed: the employee pays "the remaining portion, if any". +- **Massachusetts, Acts 2026 c. 101** (approved 2026-06-12, before the freeze). Its s. 25 strikes "40" in lines 22, 33 and 39 of c. 175M s. 6 and inserts "100"; s. 26 strikes "100" in lines 25, 33 and 43 and inserts "40". Section 6 contains each figure exactly three times, in subsections (c), (d) and (e), so the swap reaches the deduction caps: up to 40% of family and 100% of medical. (Applied in sequence, line 33, subsection (d), would end with both figures at 40, which reads as a drafting slip.) Its s. 45 applies ss. 25-26 "for taxable years beginning on or after January 1, 2026", while its s. 43(a) asks for guidance on "the impact for employers and employees in calendar year 2027". The department's rates page, captured 2026-06-17 after the act's approval, still showed the 0.46% maximum, and the department later dated the change 2027-01-01. Under the literal s. 45 reading, the most a large employer could deduct for scenario_081 in 2026 would be $1,351.02 (0.772%) rather than $805.01. The share stays optional either way, so this bears only on keeping the reference. ## What the models answered @@ -77,7 +84,7 @@ Every 2026 rate, share and cap above was published before the 2026-07-03 referen | scenario_081 MA | 14,192.66 | 13,387.65 | 9 | 9 | 13,387.65 (7), 14,192.66 (6), 14,193 (3) | | scenario_082 NY | 8,108.03 | 7,664.92 | 4 | 14 | 7,664.92 (8), 7,665 (6), 8,108.03 (3) | -Of 46 models, the explanations on each output (`verification/explanations.json`, one reviewer per group of outputs, every model classified): +Every model's explanation on each output, classified by one reviewer per group of outputs (`verification/explanations.json`; 46 models per output): | Output | Include the state program | Leave it out as optional or employer-paid | Say the state has no employee payroll tax | Never mention it | Other or unparsed | |---|---:|---:|---:|---:|---:| @@ -85,11 +92,6 @@ Of 46 models, the explanations on each output (`verification/explanations.json`, | scenario_043 CO | 18 | 0 | 14 | 11 | 3 | | scenario_081 MA | 22 | 1 | 7 | 9 | 7 | | scenario_082 NY | 22 | 0 | 6 | 12 | 6 | - -Where the law requires the employee contribution, the explanations look the same (mandatory programs, all scored): - -| Output | Include the state program | Leave it out as optional or employer-paid | Say the state has no employee payroll tax | Never mention it | Other or unparsed | -|---|---:|---:|---:|---:|---:| | scenario_005 CA SDI | 31 | 0 | 2 | 8 | 5 | | scenario_022 CA SDI | 29 | 2 | 4 | 9 | 2 | | scenario_023 CA SDI | 30 | 0 | 5 | 7 | 4 | @@ -98,9 +100,18 @@ Where the law requires the employee contribution, the explanations look the same | scenario_123 PA UC | 15 | 0 | 16 | 11 | 4 | | scenario_120 CT Paid Leave | 20 | 0 | 11 | 9 | 6 | -So a federal-only answer usually means the model did not know or did not apply the program: on Pennsylvania's mandatory contribution, 17 and 16 models say the state has no employee payroll tax. Across the four optional outputs, a regenerated reference would newly credit 82 answers. One of them gives the optional-deduction reason (Grok 4.7 on scenario_032: "paid-leave withholding is optional"); 36 say the state has no employee payroll tax, 39 never mention a state program and 6 are other or unparsed. It would take credit from 35 answers that include the program at exactly the engine's amount. +The explanations do not separate the optional programs from the mandatory ones. Pennsylvania's and Connecticut's mandatory contributions draw the same mix of "no employee payroll tax" and silence as Minnesota's and Colorado's optional ones; California SDI is included far more often. -Model by model, the readings split by family. Claude Opus 5.5, Claude Fable 5.1, Ox Alpha and Gemini 3.8 Flash answer federal tax alone on all four. GPT-6.1 Sol answers the frozen reference on all four. GPT-6 Sol answers federal tax alone on Minnesota and the frozen reference on Colorado, Massachusetts and New York; it also matches all seven mandatory references. Claude Fable 5.1 and Ox Alpha leave out every state program, mandatory ones included. +If the four references were regenerated, 82 answers would newly be exact. Of these: + +- 1 gives the optional-deduction reason (Grok 4.7 on scenario_032: "paid-leave withholding is optional"); +- 36 say the state has no employee payroll tax, 25 of them "no mandatory" tax, which is true on the strict reading; +- 39 never mention a state program; +- 6 are other: four name the program and leave it out, among them Claude Opus 5.5 on scenario_081 ("excluded on the assumption the model does not include it"). + +66 of the 82 come from models that include a mandatory state program elsewhere. Regeneration would also take credit from 35 answers within $1 of the frozen references; 15 equal them to the cent, and 9 are Colorado answers of 331.13 at the 2025 rate of 0.45%. Exclusion removes all 184 rows, so it stops scoring both groups. + +Model by model, Claude models almost never answer the frozen value on the four outputs (1 of 40 answers), and the newer GPT models mostly do: GPT-6.1 Sol on all four, and GPT-6 Sol, GPT-6 Astra and GPT-5.6 Sol on three. GPT-6 Sol answers federal tax alone on Minnesota and the frozen value on Colorado, Massachusetts and New York, and matches all seven mandatory references. Claude Opus 5.5 answers federal tax alone on all four; it matches all four California SDI references but leaves out Connecticut's mandatory contribution "on the assumption that PolicyEngine does not model it" and misses both Pennsylvania outputs. Claude Fable 5.1 answers federal tax alone on all eleven scored outputs, mandatory ones included. Ox Alpha and Gemini 3.8 Flash answer federal tax alone on the four; Ox Alpha includes California SDI at the 2025 rate on scenario_022 and scenario_023. ## Board impact @@ -124,29 +135,32 @@ Model by model, the readings split by family. Claude Opus 5.5, Claude Fable 5.1, Payroll exact counts are out of 99 scored payroll outputs (95 under exclusion). -**Exclude** (1,928 to 1,924 scored outputs per model): - -- Every model's headline exact rate moves by -0.07 to +0.62 points. GPT-6 Sol: 95.00% to 95.09%, still first. -- Five adjacent pairs swap: Claude Sonnet 5.5 and GPT-6 Luna (#4/#5), Kimi K3 and GPT-6.1 Sol (#8/#9), Inkling and Grok 4.7 (#12/#13), Gemini 3 Flash Preview and Claude Opus 4.7 (#29/#30), and Gemini 3.1 Flash Lite Preview and DeepSeek V4 Pro (#35/#36). -- The always-zero baseline moves from 69.98% to 70.53%, so GPT-6 Sol's lead over it goes from 25.02 to 24.57 points. -- Payroll exact match across models rises from 78.5% to 81.0%. +| | Keep | Exclude | Regenerate | +|---|---|---|---| +| Scored outputs per model | 1,928 | 1,924 | 1,928 | +| Headline change per model | none | -0.07 to +0.62 points (mean absolute 0.44) | -0.69 to +0.69 points (mean absolute 0.33) | +| GPT-6 Sol | 95.00%, first | 95.09%, first | 94.63%, first | +| GPT-6 Sol's lead over Claude Opus 5.5 | 1.31 points | 0.78 | 0.24 | +| Within-1% leader (lead over second) | GPT-6 Sol (0.97) | GPT-6 Sol (0.46) | Claude Opus 5.5 (0.09) | +| Models whose headline falls | none | GPT-6.1 Sol (-0.07, 8th to 9th) | 15, GPT-6.1 Sol most (-0.69) | +| Adjacent headline swaps | none | 5 | 10 | +| Bounded-score rank changes | none | 4 | 2 | +| Always-zero baseline | 69.98% | 70.53% | 69.98% | +| GPT-6 Sol's lead over always-zero | 25.02 points | 24.57 | 24.65 | +| Payroll exact across models | 78.5% | 81.0% | 79.5% | -**Regenerate** (1,928 scored outputs per model): +The exclusion swaps are Claude Sonnet 5.5 and GPT-6 Luna (#4/#5), Kimi K3 and GPT-6.1 Sol (#8/#9), Inkling and Grok 4.7 (#12/#13), Gemini 3 Flash Preview and Claude Opus 4.7 (#29/#30), and Gemini 3.1 Flash Lite Preview and DeepSeek V4 Pro (#35/#36). Payroll exact rises under exclusion because most of the 184 removed rows were misses. -- Headline exact rates move by -0.69 to +0.69 points. GPT-6 Sol: 95.00% to 94.63%, still first; Claude Opus 5.5: 93.70% to 94.39%, so the gap closes from 1.31 to 0.24 points. -- Ten adjacent pairs swap on the headline, among them Claude Sonnet 5.5 and GPT-6 Luna (#4/#5) and Claude Fable 5.1 and GPT-6 Astra (#6/#7). On the within-1% metric Claude Opus 5.5 passes GPT-6 Sol for first. -- The always-zero baseline is unchanged at 69.98%; GPT-6 Sol's lead over it goes from 25.02 to 24.65 points. - -**With the state income tax withholding proposal.** PolicyEngine/policybench#191 (cos d963) proposes three federal exclusions and may land in the same release. Its records alone reproduce #191's figures here (GPT-6 Sol 95.62%; Claude Sonnet 5.5 passes GPT-6 Luna). On top of them, exclusion takes GPT-6 Sol to 95.75% and Claude Opus 5.5 to 94.97% with three adjacent swaps; regeneration takes them to 95.19% and 95.00%. `verification/leaderboard_impact.json` has every figure. +**With the state income tax withholding proposal.** PolicyEngine/policybench#191 (cos d963) proposes three federal exclusions and may land in the same release. Its records alone reproduce its own figures here: GPT-6 Sol 95.61% (95.6146; #191's README rounds it to 95.62%), and Claude Sonnet 5.5 passes GPT-6 Luna. On top of them, exclusion takes GPT-6 Sol to 95.75% and Claude Opus 5.5 to 94.97%, with three adjacent swaps; regeneration takes them to 95.19% and 95.00%. `verification/leaderboard_impact.json` has every figure. ## Method Run from this checkout. `` is `results/local/adds202609/triage` in the main checkout, which holds the policyengine-us 2.15.17 venv; `` is the repo's `.venv`. -1. **Decompose.** `scripts/decompose_payroll.py` recomputes every payroll reference on policyengine-us 2.15.17 with `latest_final` (`../2026-09-28/fixes/`) and splits it into federal and per-program state parts (`verification/payroll_decomposition.csv` and `.json`). All 100 reproduce the published reference, and the parts add to the total. -2. **Classify.** One researcher per program read the statute, rules, agency pages and engine source; two reviewers per program, one on the statute and one on the rules and agency guidance, then tried to refute each classification, re-fetching the sources and re-running the engine amounts. None refuted. `program_classification.json` is the result; `verification/law_research.json` holds every record and verdict. -3. **Sweep.** `scripts/sweep_payroll_scope.py` recomputes all 1,984 outputs with and without `fixes/payroll_mandatory_scope.py`, an output-scope adapter in the form of the Maryland one (`../2026-09-28/fixes/latest_md_local_output_scope.py`). It drops the five optional contributions from the list behind `employee_state_payroll_tax` and changes no formula or rate. The baseline reproduces all 1,928 scored references; the 19 excluded outputs that differ keep the values they were decided on, each listed in the sidecar's `excluded_outputs_rechecked`. The adapter moves four outputs, the four payroll references above, and no other output by even a cent (`verification/sweep_payroll_scope.csv` and `.log`). `CountryTaxBenefitSystem` applies a reform twice to the same system, so the adapter is idempotent. -4. **Propose.** `scripts/propose_changes.py` writes both alternatives in the format a release installs: `proposed_exclusions.json` (four `reference_depends_on_unlisted_input` records) and `proposed_regenerations.json` (four regenerated values with the adapter's sha256). It refuses a sweep that moves any output the proposal does not cover. +1. **Decompose.** `scripts/decompose_payroll.py` recomputes every payroll reference on policyengine-us 2.15.17 with `latest_final` (`../2026-09-28/fixes/`) and splits it into federal and per-program state parts (`verification/payroll_decomposition.csv`, `.json` and `.log`). All 100 reproduce the published reference, and the parts add to the total. +2. **Classify.** As above; `program_classification.json` is the result, and `verification/law_research.json` holds the records and verdicts. +3. **Sweep.** `scripts/sweep_payroll_scope.py` recomputes all 1,984 outputs with and without `fixes/payroll_mandatory_scope.py`, an output-scope adapter that drops the five reviewed optional contributions from the list behind `employee_state_payroll_tax` and changes no formula or rate. The baseline reproduces all 1,928 scored references; the 19 excluded outputs that differ keep the values they were decided on, each listed in the sidecar's `excluded_outputs_rechecked`. The adapter moves four outputs, the four payroll references above, and no other output by even a cent (`verification/sweep_payroll_scope.csv` and `.log`). `CountryTaxBenefitSystem` applies a reform twice to the same system (`policyengine_us/system.py`), so the adapter is written to be idempotent: its second pass leaves the already-flattened list unchanged. +4. **Propose.** `scripts/propose_changes.py` writes both alternatives in the format a release installs: `proposed_exclusions.json` (four `reference_depends_on_unlisted_input` records sharing one unlisted input) and `proposed_regenerations.json` (four regenerated values with the adapter's sha256). It refuses a sweep that moves any output the proposal does not cover, and an adapter whose program list differs from the reviewed optional programs. 5. **Answers.** `scripts/model_answers.py` writes `verification/model_answers.csv` and `model_answers_summary.json`. 6. **Impact.** `scripts/leaderboard_impact.py --with-salt` writes `verification/leaderboard_impact*.csv`, `leaderboard_impact.json` and `.log`. It reads #191's records from its commit `8af912a0`. @@ -158,14 +172,61 @@ PYTHONPATH=$PWD /bin/python reference_audit/2026-10-05-payroll PYTHONPATH=$PWD /bin/python reference_audit/2026-10-05-payroll/scripts/leaderboard_impact.py --scratch --with-salt ``` -INDEPENDENT_REVIEW +## Independent review + +Eight reviewers checked the work in a second Claude Code workflow on 2026-10-05, each in its own scratch directory and without changing the checkout. Their structured reports are in `verification/independent_reviews.json`. + +- **Differential impact.** One re-implemented both options without the CLI, once through `policybench.analysis`'s functions and once in its own numpy code. All 2,438 compared values match (largest rate difference 2.8e-14), as do 966 more for the #191 combinations. It found that #191's 95.62% is a rounding of 95.6146. +- **Differential sweep.** One recomputed all 1,984 outputs by a different mechanism: the five contributions set to 0 as household inputs, with no adapter. Every value is bit-identical to the audit's, and exactly the four payroll outputs move. +- **Mechanism.** One traced the engine path, found the double reform application at `system.py` lines 136 and 153, and hand-computed all 36 federal, state and total values and the four regenerated values from the household facts (largest gap $0.00066, float32 storage). It corrected three statements, fixed above: the engine readers of these variables, the Washington households, and the `employer_headcount` input. +- **Recommendation.** Two advocates built the strongest cases for keep and for regenerate, and a judge weighed them against the rules. The judge found exclusion best supported and the recommendation sound, but rejected several of the first draft's arguments. Board effects and which answers gain credit are not rule criteria; inclusion counts cannot tell ambiguous outputs from clear ones; and exclusion moves the board more than regeneration on most measures. This README now rests the recommendation on the rules and precedents above. +- **Release checklist.** One listed what a release adopting either option must change, by installing each option's records on scratch clones and running the tests (below). +- **Claims.** One checked 131 claims in the first draft and found five errors and thirteen imprecisions, all corrected here. ## What a release adopting either option must also do -RELEASE_CHECKLIST +The release-checklist reviewer found both options installable, and neither a records-only change. Either needs a release driver (`freeze_gpt61sol.py` refuses any change to the exclusions), a payload refreeze, and changes to tests that pin release 20260929 or 20260930 bytes. + +**Exclude:** + +1. Record the ruling and set `decided_on` on the four records to its date. +2. Insert the four records into `reference_exclusions.json` before the trailing scenario_023 record (`tests/test_reference_upgrade.py:221-232` requires the tail to equal `final_actions.json`'s audit exclusions), and put any new `derivation` text before the 2026-09-29 marker. +3. Add four developer adjudications (`adjudicated_failure_source: prompt_ambiguity`, `excluded_from_scoring: true`, `reference_verdict: unlisted_input`). Take `judged_on_utc` from the published audit tree (`results/local/gpt61sol-stage/gpt61sol-v1/audit/cases`: 2026-09-29 for 032, 081 and 082; 2026-09-22 for 043), not the older `unified_audit` verdicts. Add the 2026-10-05 wave to `date_conventions` and the judge evidence, and fill the 2026-09-29 wave's null commit (`tests/test_adjudications.py:639, 694-784`). +4. Apply the adjudications with `scripts/apply_adjudications.py`: 145 `llm_error` rows become `prompt_ambiguity` (38 on 032, 30 on 043, 36 on 081, 41 on 082); the four `parse_contract_failure` rows stay. +5. Reword the four case notes and the row annotations that call these shares mandatory (at least 47 of the 149). scenario_032's note is wrong under any option: it says "The one model that did include the paid-leave premium used 0.35%", but seven included it and six match the reference. scenario_043's gives FICA as $312.75 and then $312.73 (the engine's is $312.74). +6. Let `tests/test_reference_audit.py:288-300` and `tests/test_reference_upgrade.py:94-108` accept records decided after their waves; re-scope `tests/test_finish_gpt61sol.py:400-418`, which pins release 20260929's exclusion record. +7. Update the pinned counts: + +| Count | Now | Exclude | Exclude with #191 | +|---|---:|---:|---:| +| Excluded outputs | 56 | 60 | 63 | +| Unlisted-input exclusions | 28 | 32 | 35 | +| Households with an exclusion | 39 | 41 | 42 | +| Scored outputs per model | 1,928 | 1,924 | 1,921 | +| Exclusions decided on 2.15.17 | 4 | 8 | 11 | +| Developer adjudications | 69 | 73 | 76 | +| Audited rows | 7,860 | 7,711 | 7,573 | +| Exact-match misses | 7,856 | 7,707 | 7,569 | +| `prompt_ambiguity` rows | 820 | 965 | 1,100 | +| Annotated rows on excluded outputs | 2,111 | 2,260 | 2,398 | +| Parse-contract failures (scored) | 652 | 648 | 645 | +| Leader's weighted exact (`app/tests/expandPage.test.ts`) | 95.003 | 95.093 | 95.749 | + +8. Refreeze the payload, analysis CSVs and manifest; update `app/src/data.artifact.json`, `data.versions.json` (60 exclusions: 52 on 1.755.4, 8 on 2.15.17) and `paperSnapshot.json`; keep the snapshot label. Uploading the release asset is Max's step. +9. Rescore the sensitivity evidence (`scripts/sensitivity_by_variable.py`, then `scripts/rescore_sensitivity_summaries.py`; Claude Fable 5.1 auto goes from 91.728 to 92.349). +10. Revise `paper/index.qmd` (the unlisted-input list near line 1087, the sweep sentence, and a review-table row beside "US payroll and overtime" at line 888), `docs/benchmark_card.md` and `docs/paper.md`; re-render and run `freeze_snapshot.py --rendered-only`. +11. If #191 lands in the same release, reword its scenario_081 record, which counts the $805.01 Massachusetts share as paid. + +**Regenerate**, beyond the refreeze, pointers, sensitivity and paper steps above: + +1. Write the four values into both reference CSVs (the run copy and `paper/snapshot/20260501/us_reference_outputs.csv`) and update `reference_csv_sha256`. +2. Decide how the sidecar records the adapter. `scripts/freeze_snapshot.py` requires a single `engine_upgrade` revision as the last one, and the Maryland adapter is a `fix_modules` entry inside it. Add `payroll_mandatory_scope.py` to `OUTPUT_SCOPE_ADAPTERS` and rewrite the reproducibility note that names only the Maryland adapter (`tests/test_snapshot_artifacts.py:897-935`). +3. Re-run the publication check on the newest policyengine-us with the adapter, as the engine rule requires; `latest_final_2170.csv` has no adapter. +4. Rewrite the four reference narratives, remove the 82 annotations that become exact, annotate the 35 new misses, and re-judge the four cases with Claude Opus 5.5 through `scripts/run_audit_claude.sh`. +5. The pinned exclusion and adjudication counts do not change; audited rows go to 7,813 and misses to 7,809; the leader pin becomes 94.629. ## Related findings -- **Published annotations.** The case notes for these outputs treat the frozen reference as settled. scenario_081's calls the Massachusetts contribution "the mandatory Massachusetts Paid Family and Medical Leave (PFML) employee contribution", and scenario_032's and scenario_043's count the optional-deduction reasoning as a model error. Both options need them reworded. -- **Engine.** policyengine-us has no input for the employer's deduction choice, employer size (except Massachusetts' employer side) or a private plan. An upstream change could add one and default it to the largest share, so users can model either reading. The Connecticut rate parameter cites 31-49e, where 31-49g(b)(1) holds the rate; California's SDI rate parameter carries 1.3% back to 0000-01-01, wrong for years before 2026 (1.2% in 2025). Neither moves a benchmark number. -- **SALT.** policyengine-us 2.15.17 leaves state payroll contributions out of the federal SALT deduction, the defect PolicyEngine/policybench#191 records. +- **Engine.** policyengine-us has no input for the employer's deduction choice. An upstream change could add one, defaulting to the largest share, so users can model either reading; it would also need to settle Delaware, Maine, Vermont and Washington. The Connecticut rate parameter cites 31-49e, where 31-49g(b)(1) holds the rate. California's SDI rate parameter carries 1.3% back to 0000-01-01, which is wrong for earlier years (1.2% in 2025). The Massachusetts parameters do not encode Acts 2026 c. 101. None of these moves a benchmark number. +- **#191.** Its README rounds GPT-6 Sol's 95.6146% to 95.62%. Its scenario_081 record treats the $805.01 Massachusetts share as a contribution the household paid, which this audit classifies as optional. +- **SALT.** policyengine-us 2.15.17 leaves state payroll contributions out of the federal SALT deduction, the defect #191 records. diff --git a/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py b/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py index f79881f5..9b2d3e9d 100644 --- a/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py +++ b/reference_audit/2026-10-05-payroll/fixes/payroll_mandatory_scope.py @@ -15,12 +15,18 @@ "assuming the employer withholds the maximum permitted employee share" (variables/gov/states/mn/tax/payroll/paid_leave/mn_employee_paid_leave_contribution.py). -This module removes those programs (``OPTIONAL``, classified from primary law in -``../program_classification.json``) from the list behind ``employee_state_payroll_tax``, -so the scored output again matches its definition. Only the list changes; no formula, -rate or amount changes, and every mandatory employee contribution stays in. It mirrors -the 2026-09-28 Maryland output-scope adapter +This module removes the reviewed programs of that kind that reach a benchmark household +(``OPTIONAL``, classified from primary law in ``../program_classification.json``) from +the list behind ``employee_state_payroll_tax``, so the scored output reads "mandatory" +as the amount the law requires of the employee. Only the list changes; no formula, rate +or amount changes, and every mandatory employee contribution stays in. It has the form +of the 2026-09-28 Maryland output-scope adapter (reference_audit/2026-09-28/fixes/latest_md_local_output_scope.py). + +It leaves in Delaware's, Maine's and Vermont's employee shares, which the same research +classifies as optional pass-throughs, and Washington's paid leave share (mixed). No +benchmark household earns wages in those states, and they were not adversarially +reviewed, so an upstream change would need to settle them first. """ from policyengine_core.reforms import Reform diff --git a/reference_audit/2026-10-05-payroll/program_classification.json b/reference_audit/2026-10-05-payroll/program_classification.json index 85b494bd..94c8cc6b 100644 --- a/reference_audit/2026-10-05-payroll/program_classification.json +++ b/reference_audit/2026-10-05-payroll/program_classification.json @@ -15,7 +15,7 @@ "classification": "optional_employer_pass_through", "law_citation": "Minn. Stat. 268B.14, subds. 1, 3 and 5a", "law_url": "https://www.revisor.mn.gov/statutes/cite/268B.14", - "employee_share_rule_text": "Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice.", + "employee_share_rule_text": "Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent (small employers: at least a quarter of the full rate, subd. 5a), and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice.", "rate_2026": "0.88% total (0.66% for small employers); employee share at most 0.44% of wages up to the Social Security base", "benchmark_outputs": [ "scenario_032" @@ -72,7 +72,7 @@ "law_citation": "M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5)", "law_url": "https://malegislature.gov/Laws/GeneralLaws/PartI/TitleXXII/Chapter175M/Section6", "employee_share_rule_text": "Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)).", - "rate_2026": "0.88% total (0.18% family, 0.70% medical); employee share at most 0.46% of wages up to the Social Security base. Acts 2026 c. 101 ss. 25-26 (approved 2026-06-12) swap the caps to 40% family and 100% medical; s. 45 applies them to taxable years beginning on or after 2026-01-01, while DFML (after the freeze) dates the change 2027-01-01. Under the literal s. 45 reading the 2026 cap would be 0.772%.", + "rate_2026": "0.88% total (0.18% family, 0.70% medical); employee share at most 0.46% of wages up to the Social Security base. Acts 2026 c. 101 (approved 2026-06-12) s. 25 strikes \"40\" in lines 22, 33 and 39 of s. 6 and s. 26 strikes \"100\" in lines 25, 33 and 43; s. 6 holds each figure exactly three times, in 6(c), 6(d) and 6(e), so the swap reaches the deduction caps (40% family, 100% medical). Its s. 45 applies ss. 25-26 \"for taxable years beginning on or after January 1, 2026\"; its s. 43(a) asks for guidance on \"the impact for employers and employees in calendar year 2027\"; the DFML rates page captured 2026-06-17, after approval, still showed 0.46%, and DFML later (after the freeze) dated the change 2027-01-01. Under the literal s. 45 reading the 2026 cap would be 0.772% ($1,351.02 for scenario_081).", "benchmark_outputs": [ "scenario_081" ], @@ -100,8 +100,8 @@ "classification": "optional_employer_pass_through", "law_citation": "N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1)", "law_url": "https://www.nysenate.gov/legislation/laws/WKC/209", - "employee_share_rule_text": "New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them.", - "rate_2026": "PFL 0.432% of wages, at most $411.91; DBL 0.5% of wages, at most $0.60 a week", + "employee_share_rule_text": "New York's covered employer must provide disability and Paid Family Leave coverage; it pays the disability cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)), no employer is required to fund Paid Family Leave (s. 209(3)(b)), and the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)). The Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them.", + "rate_2026": "PFL 0.432% of wages, at most $411.91; DBL 0.5% of wages, at most $0.60 a week ($31.20 over 52 weeks; a weekly payroll with 53 paydays in 2026 could reach $31.80)", "benchmark_outputs": [ "scenario_082" ], @@ -243,7 +243,7 @@ } ], "programs_without_a_benchmark_household": { - "note": "No benchmark household lives in these states, so no reference depends on them. Each was researched once (verification/law_research.json, key OTHER) and not adversarially reviewed; payroll_mandatory_scope.py does not touch them.", + "note": "No benchmark household with wages lives in these states (two Washington households, scenario_002 and scenario_111, have no wages), so no reference depends on them. Each was researched once (verification/law_research.json, key OTHER) and not adversarially reviewed; payroll_mandatory_scope.py does not touch them.", "programs": [ { "program": "Washington Paid Family and Medical Leave (PFML) premium, employee share", diff --git a/reference_audit/2026-10-05-payroll/proposed_exclusions.json b/reference_audit/2026-10-05-payroll/proposed_exclusions.json index aaa8d1e9..e7d98f14 100644 --- a/reference_audit/2026-10-05-payroll/proposed_exclusions.json +++ b/reference_audit/2026-10-05-payroll/proposed_exclusions.json @@ -3,59 +3,59 @@ "status": "proposed 2026-10-05; changes published scores, so it waits for Max's ruling (cos decision). decided_on is the draft date; a release sets it to the ruling's date. proposed_regenerations.json and proposed_exclusions.json are alternatives: a release adopts one of them, not both.", "classification": "program_classification.json", "sweep": "verification/sweep_payroll_scope.csv", - "basis": "policybench/reference_exclusions.py: an output is excluded for every model when its reference depends on an engine input the prompt never listed and a careful reader could take the stated facts the other way.", + "basis": "A reference that turns on an input or definition the prompt never states is excluded (reference_audit/2026-09-22 rule 4; reference_audit/2026-09-28 rule 4). The input here is an employer-arrangement fact with no engine input, as in r24 (who paid for the coverage behind listed disability benefits) and r25 (whether the federal output includes the net investment income tax), both recorded as reference_depends_on_unlisted_input.", "exclusions": [ { "scenario_id": "scenario_032", "variable": "payroll_tax", "reason_code": "reference_depends_on_unlisted_input", - "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The reference counts the largest employee share the employer may deduct (mn_employee_paid_leave_contribution $127.60), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes, and says to exclude employer payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent (small employers: at least a quarter of the full rate, subd. 5a), and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The reference reads 'mandatory' as the employee share the statute sets, withheld unless the employer elects to pay it, and counts the largest share the employer may deduct (mn_employee_paid_leave_contribution $127.60), as policyengine-us assumes. Read as the amount the law requires of the employee whatever the employer does, which is $0, the output is employee federal payroll tax alone: the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", "frozen_value": 2346.10009765625, "alternative_value": 2218.5, "engine_version": "policyengine-us 2.15.17", "decided_on": "2026-10-05", "decided_by": "developer", - "unlisted_input": "whether the employer deducts the employee share of Minnesota Paid Leave from wages, which the law permits but does not require (Minn. Stat. 268B.14, subds. 1, 3 and 5a)", - "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + "unlisted_input": "whether the employer deducts the employee share of a state paid-leave or disability premium that the law lets it deduct but does not require, which decides whether that share is a mandatory employee state payroll tax", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The law: Minn. Stat. 268B.14, subds. 1, 3 and 5a. The frozen value has been the reference since the first run (references generated 2026-06-12)." }, { "scenario_id": "scenario_043", "variable": "payroll_tax", "reason_code": "reference_depends_on_unlisted_input", - "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Colorado FAMLI premiums are remitted by the employer (C.R.S. 8-13.3-507(2)), which \"may deduct up to 50 percent of the premium\" from wages (507(5)); an employer that deducts less is considered to have elected to pay the rest (7 CCR 1107-1 s. 1.4(6)(A)). The reference counts the largest employee share the employer may deduct (co_employee_famli_contribution $17.99), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes, and says to exclude employer payroll taxes. Colorado FAMLI premiums are remitted by the employer (C.R.S. 8-13.3-507(2)), which \"may deduct up to 50 percent of the premium\" from wages (507(5)); an employer that deducts less is considered to have elected to pay the rest (7 CCR 1107-1 s. 1.4(6)(A)). The reference reads 'mandatory' as the employee share the statute sets, withheld unless the employer elects to pay it, and counts the largest share the employer may deduct (co_employee_famli_contribution $17.99), as policyengine-us assumes. Read as the amount the law requires of the employee whatever the employer does, which is $0, the output is employee federal payroll tax alone: the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", "frozen_value": 330.7310791015625, "alternative_value": 312.74322509765625, "engine_version": "policyengine-us 2.15.17", "decided_on": "2026-10-05", "decided_by": "developer", - "unlisted_input": "whether the employer deducts the employee share of Colorado FAMLI from wages, which the law permits but does not require (C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A))", - "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + "unlisted_input": "whether the employer deducts the employee share of a state paid-leave or disability premium that the law lets it deduct but does not require, which decides whether that share is a mandatory employee state payroll tax", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The law: C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A). The frozen value has been the reference since the first run (references generated 2026-06-12)." }, { "scenario_id": "scenario_081", "variable": "payroll_tax", "reason_code": "reference_depends_on_unlisted_input", - "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)). The reference counts the largest employee share the employer may deduct (ma_employee_paid_leave_contribution $805.01), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes, and says to exclude employer payroll taxes. Massachusetts PFML contributions are remitted by the employer (M.G.L. c. 175M, s. 6(a)), which may deduct up to 100 percent of the family and 40 percent of the medical contribution from wages (s. 6(c)) and may choose to deduct less (458 CMR 2.05(5)(c)). The reference reads 'mandatory' as the employee share the statute sets, withheld unless the employer elects to pay it, and counts the largest share the employer may deduct (ma_employee_paid_leave_contribution $805.01), as policyengine-us assumes. Read as the amount the law requires of the employee whatever the employer does, which is $0, the output is employee federal payroll tax alone: the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", "frozen_value": 14192.662109375, "alternative_value": 13387.65234375, "engine_version": "policyengine-us 2.15.17", "decided_on": "2026-10-05", "decided_by": "developer", - "unlisted_input": "whether the employer deducts the employee share of Massachusetts Paid Family and Medical Leave from wages, which the law permits but does not require (M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5))", - "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + "unlisted_input": "whether the employer deducts the employee share of a state paid-leave or disability premium that the law lets it deduct but does not require, which decides whether that share is a mandatory employee state payroll tax", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The law: M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5). The frozen value has been the reference since the first run (references generated 2026-06-12)." }, { "scenario_id": "scenario_082", "variable": "payroll_tax", "reason_code": "reference_depends_on_unlisted_input", - "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The reference counts the largest employee share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91), as policyengine-us assumes. Read as an employer premium that is not a mandatory employee tax, or with an employer that does not deduct it, the output is employee federal payroll tax alone, the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", + "alternative_reading": "The prompt defines payroll_tax as employee Social Security, Medicare and Additional Medicare Tax plus mandatory employee state payroll taxes, and says to exclude employer payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage; it pays the disability cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)), no employer is required to fund Paid Family Leave (s. 209(3)(b)), and the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)). The Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The reference reads 'mandatory' as the employee share the statute sets, withheld unless the employer elects to pay it, and counts the largest share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91), as policyengine-us assumes. Read as the amount the law requires of the employee whatever the employer does, which is $0, the output is employee federal payroll tax alone: the alternative value (policyengine-us 2.15.17 with latest_final and payroll_mandatory_scope).", "frozen_value": 8108.02734375, "alternative_value": 7664.91748046875, "engine_version": "policyengine-us 2.15.17", "decided_on": "2026-10-05", "decided_by": "developer", - "unlisted_input": "whether the employer deducts the employee share of New York Paid Family Leave and Disability Benefits from wages, which the law permits but does not require (N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1))", - "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The frozen value has been the reference since the first run (2026-06-15)." + "unlisted_input": "whether the employer deducts the employee share of a state paid-leave or disability premium that the law lets it deduct but does not require, which decides whether that share is a mandatory employee state payroll tax", + "note": "Found 2026-10-05 by classifying every state program in the payroll references from primary law (reference_audit/2026-10-05-payroll/program_classification.json) and sweeping all 1,984 outputs with the programs classified as optional employer pass-through left out of employee state payroll tax; that moves this output alone in its household and no other output. The law: N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1). The frozen value has been the reference since the first run (references generated 2026-06-12)." } ] } diff --git a/reference_audit/2026-10-05-payroll/proposed_regenerations.json b/reference_audit/2026-10-05-payroll/proposed_regenerations.json index abd2b0c0..864aa1e9 100644 --- a/reference_audit/2026-10-05-payroll/proposed_regenerations.json +++ b/reference_audit/2026-10-05-payroll/proposed_regenerations.json @@ -3,7 +3,7 @@ "status": "proposed 2026-10-05; changes published scores, so it waits for Max's ruling (cos decision). decided_on is the draft date; a release sets it to the ruling's date. proposed_regenerations.json and proposed_exclusions.json are alternatives: a release adopts one of them, not both.", "classification": "program_classification.json", "sweep": "verification/sweep_payroll_scope.csv", - "basis": "The benchmark's output definitions hold (reference_audit/2026-09-28 rule 3): an output-scope adapter restores the payroll_tax definition, as latest_md_local_output_scope.py restored the state income tax output's.", + "basis": "The benchmark's output definitions hold (reference_audit/2026-09-28 rule 3), reading 'mandatory employee state payroll taxes' as the amount the law requires of the employee. An output-scope adapter applies that reading, in the form of latest_md_local_output_scope.py.", "regenerated": [ { "scenario_id": "scenario_032", @@ -15,11 +15,11 @@ "mn_employee_paid_leave_contribution": 127.6 }, "cause": "payroll_mandatory_scope", - "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent, and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The frozen reference counts the largest share the employer may deduct (mn_employee_paid_leave_contribution $127.60); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. Minnesota Paid Leave premiums \"accrue and become payable by each employer\" (Minn. Stat. 268B.14, subd. 1); the employer must pay at least 50 percent (small employers: at least a quarter of the full rate, subd. 5a), and employees pay through wage deductions only \"the remaining portion, if any, of the premium not paid by the employer\" (subd. 3), so the employee share runs from 0 to 0.44% of wages at the employer's choice. The frozen reference counts the largest share the employer may deduct (mn_employee_paid_leave_contribution $127.60); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", "law": "Minn. Stat. 268B.14, subds. 1, 3 and 5a", "engine_version": "policyengine-us 2.15.17", "fix_module": "payroll_mandatory_scope.py", - "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "fix_module_sha256": "e139fe6f4dded9723721def83b03facc466bff19930532024d6262166846dc54", "applied_together_with": [ "latest_final" ] @@ -38,7 +38,7 @@ "law": "C.R.S. 8-13.3-507(2), (3)(b) and (5); 7 CCR 1107-1 s. 1.4(6)(A)", "engine_version": "policyengine-us 2.15.17", "fix_module": "payroll_mandatory_scope.py", - "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "fix_module_sha256": "e139fe6f4dded9723721def83b03facc466bff19930532024d6262166846dc54", "applied_together_with": [ "latest_final" ] @@ -57,7 +57,7 @@ "law": "M.G.L. c. 175M, s. 6(a), (c) and (d); 458 CMR 2.05(5)", "engine_version": "policyengine-us 2.15.17", "fix_module": "payroll_mandatory_scope.py", - "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "fix_module_sha256": "e139fe6f4dded9723721def83b03facc466bff19930532024d6262166846dc54", "applied_together_with": [ "latest_final" ] @@ -73,11 +73,11 @@ "ny_employee_paid_family_leave_contribution": 411.91 }, "cause": "payroll_mandatory_scope", - "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage and pays its cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)); the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)), and the Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The frozen reference counts the largest share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", + "basis": "The output's definition counts mandatory employee state payroll taxes and excludes employer payroll taxes. New York's covered employer must provide disability and Paid Family Leave coverage; it pays the disability cost beyond what it collects (N.Y. Workers' Compensation Law s. 210(1)), no employer is required to fund Paid Family Leave (s. 209(3)(b)), and the employer \"is authorized to collect\" employee contributions up to the statutory caps by payroll deduction (s. 209(4)). The Workers' Compensation Board says an employer \"is allowed, but not required\" to collect them. The frozen reference counts the largest share the employer may deduct (ny_employee_disability_benefits_contribution $31.20, ny_employee_paid_family_leave_contribution $411.91); the regenerated value leaves it out and keeps employee Social Security and Medicare tax.", "law": "N.Y. Workers' Compensation Law ss. 209(3)-(4) and 210(1)", "engine_version": "policyengine-us 2.15.17", "fix_module": "payroll_mandatory_scope.py", - "fix_module_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "fix_module_sha256": "e139fe6f4dded9723721def83b03facc466bff19930532024d6262166846dc54", "applied_together_with": [ "latest_final" ] diff --git a/reference_audit/2026-10-05-payroll/scripts/propose_changes.py b/reference_audit/2026-10-05-payroll/scripts/propose_changes.py index 48f62027..f9d6e896 100644 --- a/reference_audit/2026-10-05-payroll/scripts/propose_changes.py +++ b/reference_audit/2026-10-05-payroll/scripts/propose_changes.py @@ -8,12 +8,15 @@ One entry per scored output the output-scope adapter moves: the frozen reference, the regenerated value (policyengine-us 2.15.17 + latest_final + fixes/payroll_mandatory_scope.py), and the program law. A release that adopts it - writes the values into the reference CSV and adds one ``output_scope`` revision to - the reference sidecar, as the Maryland adapter was recorded. + writes the values into the reference CSV and records the adapter in the reference + sidecar. The Maryland adapter is a ``fix_modules`` entry inside the single + ``engine_upgrade`` revision, and the freeze requires that revision to be last, so a + release must choose how to record this one (README, release checklist). ``proposed_exclusions.json`` One ``reference_depends_on_unlisted_input`` record per such output, in the format of - the frozen run's ``reference_exclusions.json``. The unlisted input is whether the - employer deducts the employee share, which the law leaves to the employer. + the frozen run's ``reference_exclusions.json``. All four share one unlisted input: + whether the employer deducts the employee share, which the law leaves to the employer, + and so which reading of "mandatory employee state payroll taxes" applies. Both change published scores, so they wait for Max's ruling; ``decided_on`` is the date the proposal was drafted and a release sets it to the ruling's date. @@ -42,6 +45,11 @@ DRAFTED_ON = "2026-10-05" ENGINE = "policyengine-us 2.15.17" OUTPUT = "payroll_tax" +UNLISTED_INPUT = ( + "whether the employer deducts the employee share of a state paid-leave or disability " + "premium that the law lets it deduct but does not require, which decides whether " + "that share is a mandatory employee state payroll tax" +) STATUS = ( f"proposed {DRAFTED_ON}; changes published scores, so it waits for Max's ruling " "(cos decision). decided_on is the draft date; a release sets it to the ruling's " @@ -122,7 +130,6 @@ def main() -> None: program = program_for(leaf, programs) if program not in cited: cited.append(program) - names = " and ".join(p["program"] for p in cited) law = "; ".join(p["law_citation"] for p in cited) removed_text = ", ".join( f"{leaf} {money(amount)}" for leaf, amount in sorted(removed.items()) @@ -163,12 +170,14 @@ def main() -> None: "alternative_reading": ( f"The prompt defines payroll_tax as employee Social Security, " f"Medicare and Additional Medicare Tax plus mandatory employee state " - f"payroll taxes. {why} The reference counts the largest employee " - f"share the employer may deduct ({removed_text}), as policyengine-us " - f"assumes. Read as an employer premium that is not a mandatory " - f"employee tax, or with an employer that does not deduct it, the " - f"output is employee federal payroll tax alone, the alternative " - f"value (policyengine-us 2.15.17 with latest_final and " + f"payroll taxes, and says to exclude employer payroll taxes. {why} " + f"The reference reads 'mandatory' as the employee share the statute " + f"sets, withheld unless the employer elects to pay it, and counts " + f"the largest share the employer may deduct ({removed_text}), as " + f"policyengine-us assumes. Read as the amount the law requires of " + f"the employee whatever the employer does, which is $0, the output " + f"is employee federal payroll tax alone: the alternative value " + f"(policyengine-us 2.15.17 with latest_final and " f"payroll_mandatory_scope)." ), "frozen_value": float(row.final), @@ -176,10 +185,7 @@ def main() -> None: "engine_version": ENGINE, "decided_on": DRAFTED_ON, "decided_by": "developer", - "unlisted_input": ( - f"whether the employer deducts the employee share of {names} from " - f"wages, which the law permits but does not require ({law})" - ), + "unlisted_input": UNLISTED_INPUT, "note": ( f"Found {DRAFTED_ON} by classifying every state program in the " f"payroll references from primary law " @@ -187,8 +193,8 @@ def main() -> None: f"and sweeping all 1,984 outputs with the programs classified as " f"optional employer pass-through left out of employee state payroll " f"tax; that moves this output alone in its household and no other " - f"output. The frozen value has been the reference since the first " - f"run (2026-06-15)." + f"output. The law: {law}. The frozen value has been the reference " + f"since the first run (references generated 2026-06-12)." ), } ) @@ -205,9 +211,9 @@ def main() -> None: **common, "basis": ( "The benchmark's output definitions hold (reference_audit/2026-09-28 " - "rule 3): an output-scope adapter restores the payroll_tax " - "definition, as latest_md_local_output_scope.py restored the state " - "income tax output's." + "rule 3), reading 'mandatory employee state payroll taxes' as the " + "amount the law requires of the employee. An output-scope adapter " + "applies that reading, in the form of latest_md_local_output_scope.py." ), "regenerated": regenerated, }, @@ -220,10 +226,13 @@ def main() -> None: { **common, "basis": ( - "policybench/reference_exclusions.py: an output is excluded for every " - "model when its reference depends on an engine input the prompt " - "never listed and a careful reader could take the stated facts the " - "other way." + "A reference that turns on an input or definition the prompt never " + "states is excluded (reference_audit/2026-09-22 rule 4; " + "reference_audit/2026-09-28 rule 4). The input here is an " + "employer-arrangement fact with no engine input, as in r24 (who paid " + "for the coverage behind listed disability benefits) and r25 (whether " + "the federal output includes the net investment income tax), both " + "recorded as reference_depends_on_unlisted_input." ), "exclusions": excluded, }, diff --git a/reference_audit/2026-10-05-payroll/verification/independent_reviews.json b/reference_audit/2026-10-05-payroll/verification/independent_reviews.json new file mode 100644 index 00000000..da789cb5 --- /dev/null +++ b/reference_audit/2026-10-05-payroll/verification/independent_reviews.json @@ -0,0 +1,850 @@ +{ + "date": "2026-10-05", + "reviewed_commit": "8714770979394463b193caddb366de8d18f54dc8", + "method": "Claude Code workflow wf_b0ed54d5-a21 (run after an account switch interrupted the first launch): eight agents, each in its own scratch directory, none changing the checkout. Scratch paths are not kept.", + "differential_impact": { + "agrees": true, + "compared": "2,438 values across the two required variants, all within tolerance: 0 mismatches. For each of the 46 models, under exclude and under regenerate: exact, within1pct and score, both `*_proposed` and `*_published`, computed two independent ways (path A and path B, described under method). Also n_proposed, n_published, payroll_exact_proposed, payroll_exact_published and payroll_exact_delta. Separately, the audit's published exact, within1pct, score and n columns against the payload `data.json.gz` modelStats, and my own published values against the payload. The largest absolute difference on any rate was 2.84e-14 (tolerance 1e-9). Every integer matched exactly.\n\nPayroll exact counts came from my own hit computation over scored payroll_tax rows. That is 99 rows per model for published and regenerate and 95 for exclude, with missing predictions counted as misses (66 missing under published). They match the audit CSVs, and the published counts also match the published payload's scenarioPredictions (`scored` and `exact==100`) for all 46 models.\n\nBeyond scope, also checked with 0 mismatches:\n- **SALT variants:** salt, exclude+salt and regenerate+salt, 966 more comparisons against the `*_plus_salt` and `salt` CSVs, with #191's records read via `git show 8af912a0`.\n- **Always-zero baseline:** 69.97959457368849 published, 70.52527771951148 exclude, 69.97959457368849 regenerate, equal to `leaderboard_impact.json`.\n- **Rank changes:** the rank-change sets for exact, within1pct and score equal the json in both variants.\n- **README figures:** every cell of the 13-row README table (published, exclude and regenerate exact to two decimals, both ranks, and the three payroll exact counts) is correct. The delta ranges (exclude -0.0667 to +0.6197, regenerate -0.6880 to +0.6880) and the five exclude and ten regenerate adjacent swaps on exact (the same pairs the README names) are correct. On regenerate, Claude Opus 5.5 leads within1pct (95.5763). Payroll exact across models goes 78.502% to 81.007% (exclude) and 79.534% (regenerate). The GPT-6 Sol vs Claude Opus 5.5 gap goes 1.3060 to 0.2431, and the lead over always-zero goes 25.02 to 24.57 (exclude) and 24.65 (regenerate).", + "mismatches": [], + "method": "I did not use the CLI, the staged scratch copies or the dashboard payload. All inputs were loaded in memory from the frozen run: reference_outputs.csv, reference_exclusions.json (56 records), predictions.csv.gz (91,264 rows, no duplicate model/scenario/variable keys) and scenarios.csv.\n\n- **Exclude:** I appended the 4 records from `proposed_exclusions.json` to the base list in memory and called `policybench.reference_exclusions.split_reference`. That gives 1,924 scored rows.\n- **Regenerate:** I set the 4 rows' float values to `regenerated_value` in the DataFrame. I asserted each frozen value first, and did not write the CSV and re-parse a `repr` string as the audit does. That gives 1,928 scored rows.\n- **Path A (library functions called directly, not `analyze_no_tools`):** exact and within1pct are `weighted_hit_rate_scores_by_model` × 100. Score is the mean of `bounded_household_scores` × 100. n is `summary_by_model(compute_metrics(...)).total_n`. I read `cli.py`'s analyze branch and `analysis.build_dashboard_payload` this session and confirmed that modelStats exact, within1pct, score and n map to these.\n- **Path B (hand-rolled):** the only library pieces reused are the output-group population weights (`bounded_global_variable_weights`, which I confirmed comes from the committed `population_weights.json` covering all 18 groups, so it does not depend on reference values) and `metric_type_for_output`/`BINARY_PROGRAMS`. Everything else is my own vectorized numpy code:\n - Rows split their group's weight equally within the household and are renormalized to sum to 1 per household; households with zero total weight are dropped.\n - Exact is |p−t|≤1. Within1pct is |p−t|/|t|≤0.01, or |p|≤1 when t=0. Binary outputs compare 0/1 flags.\n - Score is max(0, 1−|p−t|/|t|), with t=0 scoring 1 only when p==0.\n - Each model's rate is the equal-household mean; n is the number of scored reference rows.\n- **Payroll exact counts:** my own count of path-B exact hits on scored payroll_tax rows.", + "commands_run": [ + "git log -1; git status --porcelain (checkout clean before and after)", + "cat CLAUDE.md, reference_audit/2026-10-05-payroll/README.md, scripts/leaderboard_impact.py, proposed_exclusions.json, proposed_regenerations.json", + "sed -n policybench/cli.py (analyze parser 750-800, analyze branch 1401-1520)", + "sed -n policybench/analysis.py (60-245 row scoring, 291-630 weights and household scoring, 1051-1312 detail rows/compute_metrics/summary_by_model, 1468-1590 analyze_no_tools, 2176-2345 build_dashboard_payload modelStats)", + "cat policybench/reference_exclusions.py; sed policybench/population_weights.py", + "PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH= /Users/maxghenis/PolicyEngine/policybench/.venv/bin/python /review/differential-impact/recompute.py /review/differential-impact -> 2438 comparisons, 0 mismatches", + "same venv: derived.py (weight source, combined-exclusion validation, always-zero baseline, rank changes, adjacent swaps, README table, gaps)", + "same venv: salt_and_payload.py (payload scenarioPredictions payroll counts; salt, exclude+salt, regenerate+salt vs audit CSVs, 966 comparisons, 0 mismatches)", + "git show 8af912a0:reference_audit/2026-10-05/{proposed_exclusions.json,README.md,verification/leaderboard_impact_models.csv} (read-only)" + ], + "other_findings": [ + "Rounding error in the README's SALT figure. Line 140 says the #191 records alone give 'GPT-6 Sol 95.62%', and #191's own README at 8af912a0 says the same (line 95: 'GPT-6 Sol moves from 95.00% to 95.62%'). But the computed value is 95.61458515870082 in my recomputation, in this audit's leaderboard_impact_salt_models.csv and in #191's own leaderboard_impact_models.csv. That rounds to 95.61%, not 95.62%. The delta is +0.6110860229821071. The figure is wrong in both READMEs.", + "Combined exclusion record is valid. The 56 base plus 4 proposed records pass `load_reference_exclusions` (required fields, no duplicates, frozen != alternative) and the `split_reference` frozen-value check. None of the 4 proposed keys overlaps a base exclusion, and none of the 4 regenerated keys is already excluded.", + "The regenerated values equal the exclusion records' alternative_value exactly for all 4 outputs. frozen minus regenerated is 127.6001 (MN), 17.9879 (CO), 805.0098 (MA) and 443.1099 (NY), against the `removed` amounts recorded as 127.6, 17.99, 805.01 and 443.11. The gaps of up to 0.0098 are float32 rounding in the reference CSV. They are far below the $1 exact tolerance, and no score I computed depends on them.", + "Other SALT-combined claims check out: SALT alone has Claude Sonnet 5.5 at 92.64 vs GPT-6 Luna at 92.61, so Sonnet passes Luna. Exclude+salt gives GPT-6 Sol 95.75 and Claude Opus 5.5 94.97 with exactly three adjacent exact swaps vs salt (#12/#13, #29/#30, #35/#36). Regenerate+salt gives 95.19 and 95.00 (94.9965).", + "No file in the checkout was modified (`git status --porcelain` empty). My scratch scripts, logs and per-variant CSVs are in /private/tmp/claude-501/-Users-maxghenis-Library-Application-Support-Claude-scratch-workspaces-ee3763f4-e7ea-4177-bcaf-1362266768c1-4457aa9d-2dc0-4bcf-a128-e50adae6be25-scratch-2026-09-22-c3da5e/224fe5b6-f6e8-4204-af8f-2508970193bc/scratchpad/review/differential-impact/." + ] + }, + "differential_sweep": { + "agrees": true, + "compared": "All 1,984 outputs (100 households, 1,928 scored and 56 excluded). (1) My zeroed-input values against the audit CSV's 'scoped' column: 1,984 of 1,984 within 1e-3 and 0 mismatches. They are bitwise equal on all 1,984 once the CSV is parsed with float_precision='round_trip'. With pandas' default parser, 4 differ by at most 5.7e-14 (CSV parse artifact): scenario_023 state_refundable_credits, scenario_042 state_income_tax_before_refundable_credits, scenario_112 federal_refundable_credits, scenario_112 payroll_tax. (2) My baseline (latest_final, no zero inputs) against the audit's 'final' column: 1,984 of 1,984, bitwise equal under round-trip parsing. (3) Outputs where 'scoped' != 'final': exactly 4, all payroll_tax, the same 4 where my zeroed run differs from my baseline. They are scenario_032 MN (2346.10009765625 to 2218.5), scenario_043 CO (330.7310791015625 to 312.74322509765625), scenario_081 MA (14192.662109375 to 13387.65234375) and scenario_082 NY (8108.02734375 to 7664.91748046875). The audit's moved_any count is also 4. (4) Audit 'final' against the published reference_outputs.csv (sha256 e8bbba8f..., equal to the sidecar's reference_csv_sha256): all 1,928 scored outputs exact. My own baseline also matches all 1,928. The CSV's excluded flag agrees with reference_exclusions.json on every row. Of the 56 excluded outputs, 19 differ from the reference, and each equals the sidecar's excluded_outputs_rechecked value_on_2_15_17 (19 entries, same set). (5) Additional checks: (a) employee_state_payroll_tax per household, baseline and zeroed, against sweep_payroll_scope_households.json: 100 of 100 within 1e-3. (b) proposed_regenerations.json and proposed_exclusions.json: frozen_value equals my baseline and regenerated_value/alternative_value equals my zeroed value, bitwise, for all 4.", + "mismatches": [], + "method": "The audit removes the five optional contributions from the employee_state_payroll_tax adds list through a Reform adapter (fixes/payroll_mandatory_scope.py) composed with latest_final. I used no adapter and no reform beyond latest_final, which I loaded from a scratch copy of reference_audit/2026-09-28/fixes/*.py with r19_irs_sales_tax_2025.json beside it; the module hashes match the sidecar's fix_modules. Instead, I set co_employee_famli_contribution, ma_employee_paid_leave_contribution, mn_employee_paid_leave_contribution, ny_employee_disability_benefits_contribution and ny_employee_paid_family_leave_contribution to {\"2026\": 0.0} as situation inputs on every person. Households were built with scenario_from_dict(...).to_pe_household(), with partnership_se_income renamed to partnership_self_employment_net_earnings. I used one CountryTaxBenefitSystem(reform=latest_final.reform) per worker and two Simulations per household (baseline and zeroed), on policyengine-us 2.15.17 / policyengine-core 3.32.8 with a 12-process pool. Outputs came from expand_programs_for_scenario(sidecar programs, scenario) with no filtering against the reference index; the resulting key set equals the reference CSV's and the audit CSV's 1,984 keys exactly. Values came from _pe_variable_for_output and _extract_person_value. Before relying on this, I grepped the 2.15.17 package (py and yaml, tests excluded): the only non-test readers of the five variables are the co/ma/mn/ny state aggregates, so zeroing them as inputs and removing them from the list should be equivalent, and the run confirms it. I also checked one property beyond the audit: in each household, the payroll_tax change equals minus the sum of the five baseline contributions, within 1e-3. The values are -127.6001 vs 127.6000, -17.9879 vs 17.9878, -805.0098 vs 805.0092 and -443.1099 vs 443.1100 (float32 rounding). Only those 4 households have any of the five contributions above 0. Scratch files are in /private/tmp/claude-501/-Users-maxghenis-Library-Application-Support-Claude-scratch-workspaces-ee3763f4-e7ea-4177-bcaf-1362266768c1-4457aa9d-2dc0-4bcf-a128-e50adae6be25-scratch-2026-09-22-c3da5e/224fe5b6-f6e8-4204-af8f-2508970193bc/scratchpad/review/differential-sweep/ (diff_sweep.py, diff_sweep_results.json, compare.py, compare.log, compare.csv).", + "commands_run": [ + "git -C log -1; git status --porcelain (HEAD 8714770979394463b193caddb366de8d18f54dc8, clean)", + "shasum -a 256 reference_audit/2026-09-28/fixes/*.py (match sidecar fix_modules) and fixes/payroll_mandatory_scope.py (74200522... = households json adapter_sha256)", + "grep -rn -E '' --include='*.py' --include='*.yaml' (only the state aggregates read them)", + "cp reference_audit/2026-09-28/fixes/*.py reference_audit/2026-09-22/fixes/r19_irs_sales_tax_2025.json /fixes/", + "PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH= /.venv-pe21517/bin/python diff_sweep.py (100 households, exit 0)", + "PYTHONPATH= /Users/maxghenis/PolicyEngine/policybench/.venv/bin/python compare.py", + "python: re-parse sweep_payroll_scope.csv with float_precision='round_trip' (0 non-bitwise-equal values in scoped and final)", + "python: bitwise check of proposed_regenerations.json and proposed_exclusions.json values against the independent run", + "git diff --stat 8b4c0ca1 HEAD -- reference_audit/2026-09-28 reference_audit/2026-09-22 policybench (empty)" + ], + "other_findings": [ + "Read with pandas' default C float parser, sweep_payroll_scope.csv gives 4 values that are off by up to 5.7e-14 (8 cells, the same 4 outputs in both 'final' and 'scoped'): scenario_023 state_refundable_credits 95.15400695800781 read as 95.1540069580078, scenario_042 state_income_tax_before_refundable_credits 464.18280029296875 read as 464.1828002929687, and scenario_112 federal_refundable_credits and payroll_tax 453.09600830078125 read as 453.0960083007813. The values written are exact; with float_precision='round_trip' all 1,984 are bitwise equal to my run. This is harmless at the 1e-3 tolerance, but any consumer that tests the CSV with exact == should use round-trip parsing.", + "The audit's sweep filters expanded outputs by membership in the reference index. That filter removes nothing: expand_programs_for_scenario with the sidecar's 18 programs gives exactly the 1,984 reference keys.", + "The branch leaves the frozen run directory, reference_audit/2026-09-28, reference_audit/2026-09-22 and the policybench package unchanged relative to base 8b4c0ca1. The diff is 29 added files and 16,990 insertions, all new." + ] + }, + "mechanism": { + "agrees": true, + "compared": "Hand-computed numbers: for all 12 outputs in the README table (scenario_005, 022, 023, 099, 028, 123, 120, 008, 032, 043, 081, 082) I computed the federal part (6.2% Social Security up to $184,500, 1.45% Medicare, 0.9% Additional Medicare Tax over $250k joint or $200k otherwise), the state part and the total from the scenarios.csv facts. That is 36 values, compared with verification/payroll_decomposition.csv (federal_total, employee_state_payroll_tax, reference) and reference_outputs.csv. Largest gap: $0.00033, which is float32 storage. All 16 per-program state amounts match the decomposition's state_programs. The 4 regenerated values match proposed_regenerations.json and the exclusions' alternative_value: hand 2218.5000, 312.7432, 13387.6530, 7664.9175 against audit 2218.5, 312.74322509765625, 13387.65234375, 7664.91748046875 (largest gap $0.00066). Mechanism, run separately: I re-swept all 164 outputs of the 9 households in MN, CO, MA and NY with my own code. Exactly the same 4 payroll_tax outputs move, to the same float values. Every other baseline value equals the published reference, except 2 excluded outputs (scenario_082 federal_income_tax_before_refundable_credits 9577.90 vs 9563.05; scenario_118 snap 2868.0 vs 2903.94); both equal the sidecar's excluded_outputs_rechecked values. Of the README's 'How the reference is built' section and the docstrings, these are confirmed against source: payroll_tax -> spm_unit_payroll_tax (benchmark_specs.json us[2]) -> sum_contained_tax_units('employee_payroll_tax'); employee_payroll_tax adds the 4 components; employee_state_payroll_tax adds 14 state aggregates and its documentation quote is accurate; the 18 leaves listed in decompose_payroll.STATE_PROGRAMS match the engine; the 'maximum permitted employee share' wording appears in exactly DE, MA, ME, MN and VT, and not in CO, NY, WA or anything else; latest_final has nine conventions plus the MD adapter and touches no payroll variable.", + "mismatches": [ + { + "item": "claim (README 'How the reference is built', last bullet): readers of the contributions and of employee_payroll_tax", + "audit_value": "Nothing else in the engine reads these contributions except through employee_payroll_tax, which also feeds spm_unit_paycheck_withholdings and household_tax_before_refundable_credits", + "your_value": "Not true for every contribution: ca_employee_state_disability_insurance_contribution is read directly by parameters/gov/local/ca/scc/general_assistance/countable_income/earned_deductions/person_sources.yaml (Santa Clara GA). The reader list is also incomplete. employee_payroll_tax is also read by parameters/gov/states/al/tax/income/deductions/itemized/sources.yaml (al_itemized_deductions), variables/contrib/taxsim/taxsim_tfica.py, variables/household/fica_marginal_tax_rate.py and reforms/federal/abolish_payroll_tax.py. spm_unit_payroll_tax feeds spm_unit_taxes and then spm_unit_net_income. spm_unit_paycheck_withholdings feeds the CA Marin and LA general relief programs.", + "note": "A text claim, not a computed value. The five optional leaves do have no direct reader, and none of the extra readers moves any benchmark output: my re-sweep of every MN, CO, MA and NY household moves only the 4 payroll_tax outputs. Fix the wording, e.g. 'the five optional contributions are read only through ...', and list every reader or say 'among others'." + }, + { + "item": "claim (README line 43; program_classification.json programs_without_a_benchmark_household.note): no household in Washington", + "audit_value": "no benchmark household lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington / No benchmark household lives in these states", + "your_value": "scenarios.csv has 2 WA households: scenario_002 (joint, ages 77/75) and scenario_111 (joint, ages 72/49). Both have employment_income 0 and a payroll_tax reference of 0.0, and the payroll_tax for both is scored (excluded=False).", + "note": "False for WA as written. The true statement is that no wage-earning household lives there, which is what README line 60 ('no benchmark household reaches') already gets right. No number changes." + }, + { + "item": "claim (README 'Related findings', Engine): no employer-size input", + "audit_value": "policyengine-us has no input for the employer's deduction choice, employer size (except Massachusetts' employer side) or a private plan", + "your_value": "variables/input/employer_payroll.py defines employer_headcount (Person, int, default_value 100). It is read by ma_employer_paid_leave_contribution, co_employer_famli_contribution, me_employer_paid_leave_contribution, or_employer_paid_leave_contribution, wa_employer_paid_leave_contribution, ny_mctmt_employer_quarterly_payroll_expense and employer_total_additional_state_payroll_tax. It is also read by de_paid_leave_contribution_rate, which sets the DE employee contribution (rate 0 under small_employer_threshold, parental only under full_coverage_threshold).", + "note": "The employer-size part is false. The deduction-choice part holds: I found no input for it. I did not check the private-plan part. No benchmark number is affected." + } + ], + "method": "I computed independently from law values in exact Python Decimal: SS base 184,500; CA SDI 1.3% uncapped; PA UC 0.07%; CT 0.5% to the SS base; NJ TDI 0.19% and FLI 0.23%; MN 0.44% to 185,000; CO 0.44% to the SS base; MA 0.18% + 0.4×0.70% = 0.46% to the SS base; NY PFL 0.432% capped at $411.91 and DBL 0.5% capped at $31.20. Inputs were the employment_income and self_employment_income values in each scenario_json. I first confirmed in the engine source that FICA wages are employment_income minus fica_pre_tax_contributions (pre_tax_health_insurance_premiums and HSA payroll contributions). Both are pure inputs that no scenario sets and that PE_INPUT_ALIASES does not map, so FICA wages equal employment_income here. The audit instead uses engine Simulation output (float32). For mechanism, I read policyengine_us/system.py and core's tax_benefit_system.py, simulation.py and variable.py, plus spm.py's _prepare_spm_system and share_spm_policy. I then built both systems in the 2.15.17 venv with an instrumented wrapper. The adapter is applied exactly twice: system.py line 136, 'if reform: self.apply_reform_set(reform)', and line 153, after 'reform = (reform, structural_reform)'. Core TaxBenefitSystem.__init__ does not apply it, because parameters_dir is None for US, and core Simulation built with tax_benefit_system= and no reform does not re-apply it. Call 1 rewrites the list from 14 state aggregates to 13 leaves; call 2 is a no-op, and a manual third call leaves the same object in place. Against the latest_final system, the only differing attribute of employee_state_payroll_tax is adds. Entity tax_unit, label, documentation, unit, defined_for=None and formulas={} are all the same. set_input, calculate_output and defined_for are not inherited by core's Variable, but the original declares none of them. Ignoring memory addresses, every other variable's fingerprint is identical, and all 102,807 parameter leaves are identical. Flattening keeps state gating because each of the 18 leaves carries the same defined_for StateCode as its aggregate, and core's adds path sums Person values to the TaxUnit with map_to in both versions. I did not run the audit's own scripts.", + "commands_run": [ + "git log -1; git status --porcelain (clean, HEAD 8714770979394463b193caddb366de8d18f54dc8)", + "cat CLAUDE.md, reference_audit/2026-10-05-payroll/README.md, fixes/payroll_mandatory_scope.py, scripts/{decompose_payroll,sweep_payroll_scope,propose_changes,model_answers}.py, head of scripts/leaderboard_impact.py", + "cat engine: spm_unit_payroll_tax.py, employee_payroll_tax.py, employee_state_payroll_tax.py, all 14 _employee_state_payroll_tax.py, all 18 leaf contribution files, taxable-wage variables, federal payroll variables, fica_pre_tax_contributions(.yaml), 2026 rate/cap parameter YAMLs", + "grep -rn -i 'maximum permitted' policyengine_us (5 hits: de, ma, me, mn, vt)", + "grep -rnw employee_payroll_tax / employee_state_payroll_tax / each state aggregate / each leaf across policyengine_us (excluding tests)", + "read policyengine_us/system.py lines 77-260, spm.py _prepare_spm_system/share_spm_policy, core tax_benefit_system.py 60-265, core simulation.py 144-294 and _run_formula adds path, core variable.py __init__/set/set_formulas", + "PYTHONDONTWRITEBYTECODE=1 OPENBLAS_NUM_THREADS=1 PYTHONPATH= .venv-pe21517/bin/python scratchpad/review/mechanism/diff_systems.py (application count, attribute diff, all-variable and all-parameter diff)", + "... diff_others.py (variable diff with addresses stripped: 0 differ)", + "python3 scratchpad/review/mechanism/hand_compute.py (12 outputs + 4 regenerations vs decomposition CSV, reference CSV, proposals)", + "... .venv-pe21517/bin/python scratchpad/review/mechanism/resweep.py (164 outputs in 9 MN/CO/MA/NY households, base vs scoped)", + "python3 summaries of payroll_decomposition.csv and sweep_payroll_scope.csv; scenarios.csv state counts; WA household inspection", + "shasum -a 256 fixes/payroll_mandatory_scope.py (74200522b823...3529, equal to proposals and sweep sidecar)", + "grep employer_headcount readers; cat de_paid_leave_contribution_rate.py; WA PFML and abolish_payroll_tax parameter YAMLs; latest_conventions PARTS (9)" + ], + "other_findings": [ + "Confirmed: CountryTaxBenefitSystem applies the reform exactly twice (system.py:136 and :153), and the adapter's second-pass guard makes it idempotent. A third application is also a no-op. Wording nit: README method step 3 says the system 'applies a reform twice ..., so the adapter is idempotent'. The causality is backwards: the adapter is written to be idempotent because of the double application.", + "Confirmed: the adapter changes nothing but the adds list of employee_state_payroll_tax. Before: 14 aggregates. After: the 13 leaves ca_sdi, ct, de, me, nj_tdi, nj_fli, or_pl, or_transit, pa_uc, ri_tdi, vt, wa_pl, wa_ltc. No formula, rate, parameter or other variable changes.", + "Scope caveat on the adapter docstring and DESCRIPTION ('Exclude employee shares the employer may but need not deduct'; 'This module removes those programs'). It drops only the five reviewed leaves. It keeps de_employee_paid_leave_contribution, me_employee_paid_leave_contribution and vt_employee_child_care_contribution, which the README and program_classification.json call optional pass-throughs, and wa_employee_paid_leave_contribution, classed as mixed. No benchmark effect, since no wage earners live in those states and program_classification.json says the adapter does not touch them. But used as an upstream fix it would not fully match its own definition.", + "README: 'employee_payroll_tax adds ...' leaves out the formula's branch that returns 0 when gov.contrib.ubi_center.flat_tax.abolish_payroll_tax is true. That parameter defaults to false, so there is no effect.", + "proposed_regenerations.json 'removed' amounts are cent-rounded strings taken from the decomposition CSV (CO 17.99 vs engine 17.98785; MA 805.01 vs 805.00922), not engine amounts. propose_changes.py's $0.01 check absorbs the difference. Cosmetic.", + "scenario_120's CT state part, shown as 827.98, is a float32 artifact. Exact 0.5% x 165,597 = 827.985, which rounds half-up to 827.99. Reference total 13,496.1552734 vs exact 13,496.1555. Not a mismatch.", + "Engine observation outside the audit's claims, with no effect on these outputs. additional_medicare_tax sums payroll_tax_gross_wages and taxable_self_employment_income over the whole tax unit, dependents included (scenario_123's 16-year-old has $45,000 of wages), and counts SE income inside payroll_tax. Every case here is far below the threshold.", + "Check counts all reproduce: payroll_decomposition.csv has 100/100 matches_reference, parts_add_up and leaves_add_up; 12 outputs with a state component, 11 scored; scenario_008 is excluded as reference_engine_defect; no household in a program state has wages and no state part. sweep_payroll_scope.csv has 1,984 rows, 1,928 scored, 1,965 final matches, and moved_any on exactly the 4 payroll_tax rows. The adapter sha256 74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529 matches the proposals and the sweep sidecar." + ] + }, + "recommendation": { + "advocate_briefs": [ + { + "option": "keep", + "strongest_case": "The four references follow from the stated facts under the prompt's own conventions. Each statute defines an employee share and treats the employer paying it as the employer's election: Minnesota employers pay at least half and employees \"must pay the remaining portion\"; Colorado deems an employer that deducts less to have \"elected to pay\" the employee share; Massachusetts' rule covers an employer that \"opts to deduct a lower percentage\", and its agency says employers \"must withhold\"; New York's regulator says employers \"shall collect\" PFL contributions, priced to fund the whole premium. The prompt says to treat unlisted facts as false and not to infer unlisted income, and Minnesota's agency calls an employer pickup \"additional compensation\" taxed as wages. With no election stated, the engine's amounts are the no-election amounts, not an arbitrary cap.\n\nThe exclusion rule needs a careful reader who could take the facts the other way. Of 184 answers on these outputs, one cites optional withholding (Grok 4.7, Minnesota), and the same model calls Colorado's and Massachusetts' optional shares mandatory. The federal-only answers behind the gains are guesses about the engine: Claude Opus 5.5 dropped Massachusetts \"on the assumption the model does not include it\" and Connecticut's mandatory contribution on the same ground. These items test program knowledge as the scored Pennsylvania, Connecticut and California items do, and the judges who annotated all 149 wrong answers raised no reference flag.\n\nExclusion is symmetric in rule, not in effect. It gives Claude Opus 5.5 90% of regeneration's gain, cuts GPT-6 Sol's lead over it from 1.31 to 0.78 points, drops GPT-6.1 Sol (right on all four) a place, and moves the average model's headline more than regeneration does (mean absolute change 0.44 vs 0.33 points). Keep, and state the convention in the next prompt version.", + "points": [ + { + "claim": "In all four states the law and the agencies name an employee share and describe the employer paying it as an election, so the default is the share the engine counts. Minnesota: the employer pays at least 50% and employees \"must pay the remaining portion, if any\", through deductions the employer \"must make\". Colorado: an employer that deducts less \"is considered to have elected to pay\" the employee share. Massachusetts: the rule covers an employer that \"opts to deduct a lower percentage\", and DFML says employers \"must withhold PFML contributions\". New York: DFS says \"Employers shall collect employee contributions consistent with this Decision\", and PFL is rated so that employees fund the whole premium.", + "evidence": "reference_audit/2026-10-05-payroll/verification/law_research.json: MN research.caveats and statute verdict (subd. 3, subd. 5a(b)(2)); CO research.can_employer_elect_to_pay_employee_share (7 CCR 1107-1 s. 1.4(6)(A)); MA statute and administration verdicts, citation_problems (DFML overview \"must withhold\", subtitle \"Most Massachusetts employers must make payroll withholdings\"); NY agency_guidance (DFS 2026 decision) and NY caveats (\"PFL is designed to be paid entirely by employees ... DFS sets the rate to cover the full premium\")" + }, + { + "claim": "The prompt's own conventions settle the employer choice it leaves unstated. Unlisted facts are false and no unlisted income may be inferred, and that wording was in the prompt before the run. An employer election to pay the employee share is an unlisted fact. In Minnesota it would also be unlisted income: DEED says that when the employer pays more than its minimum share, the excess is additional compensation taxed as wages.", + "evidence": "policybench/prompts.py:19-22 (TASK_PREFACE: \"Treat any unlisted numeric input as 0 and any other unlisted household fact, boolean, or status input as false ... Do not infer unlisted income\"), present since commit 1f0f7cb0 (2026-05-01), before the 2026-06-12 run; law_research.json MN guidance, pl.mn.gov/employers/taxes-and-paid-leave: \"If an employer pays more than the required minimum share of the premium, this is additional compensation to the employee and is included in the employee's federal gross income as wages.\"" + }, + { + "claim": "Minnesota has the structure the README uses to call California mandatory. In California an employer may pay the worker's SDI instead of deducting it, the payment counts as wages, and the contribution stays the worker's. DEED treats a Minnesota employer pickup the same way: as wages, with the 0.44% still the employee's for tax purposes.", + "evidence": "README.md line 67 (California bullet); law_research.json MN administration verdict reasoning: \"for federal tax purposes, an employer pickup is still treated as the employee's 0.44%, paid through extra wages\" (DEED taxes page modified 2026-05-08, before the freeze)" + }, + { + "claim": "Each engine amount is what the household owes when no election is made, not an arbitrary maximum. Minnesota: a 0.88% premium less the employer's 50% floor leaves $127.60 on $29,000. Colorado: 0.44% gives $17.99. Massachusetts: 0.46% gives $805.01. New York: PFL $411.91 plus DBL $31.20. Every reviewer re-ran the engine and got these figures.", + "evidence": "program_classification.json engine_amounts; law_research.json MN administration rate_or_amount_issues (\"Large employer: 0.88% x 29,000 = $255.20 total, with an employer minimum of 50%, which is $127.60\"); CO, MA and NY verdict rate_or_amount_issues" + }, + { + "claim": "In practice almost no model reads the facts the other way for the legal reason. Of 184 answers, two are classed as leaving the program out as optional or employer-paid. Grok 4.7 on Minnesota says \"paid-leave withholding is optional\", yet it calls Colorado's FAMLI share 0.44% and Massachusetts' PFML \"Mandatory ... at 0.46%\", though both are optional by the audit's own classification. Claude Sonnet 5 on Massachusetts calls PFML \"employer-side\" and submits 12,318.65, which matches neither reading. No model applies the optional reading consistently.", + "evidence": "verification/explanations.json counts excluded_as_optional_or_employer_paid = 1, 0, 1, 0 on scenario_032/043/081/082; predictions.csv.gz grok-4.7 rows for scenario_032 (2218.5), scenario_043 (330.72) and scenario_081 (14192.66); explanations.json claude-sonnet-5 on scenario_081 (12318.65)" + }, + { + "claim": "The leading federal-only answers are guesses about what the engine models, not readings of the law. Claude Opus 5.5 drops Connecticut's mandatory contribution on the same ground it gives for Massachusetts, and it misses both Pennsylvania mandatory items.", + "evidence": "predictions.csv.gz claude-opus-5.5: scenario_081 \"excluded on the assumption the model does not include it\"; scenario_120 (CT, mandatory) \"not included, on the assumption that PolicyEngine does not model it\"; scenario_082 \"no mandatory NY employee payroll tax is modeled\". claude-fable-5.1 scenario_032 \"no modeled mandatory MN employee payroll tax\"; explanations.json claude-sonnet-5.5 on scenario_028 \"No PA employee state payroll tax is modeled\". model_answers.csv: claude-opus-5.5 is not exact on scenario_028, 120 or 123" + }, + { + "claim": "These four outputs test program knowledge the same way the scored mandatory outputs do. Removing them drops working test items rather than fixing faulty ones.", + "evidence": "README.md lines 82-99 explanation tables: MN has 19 'no state tax' and 16 silent; PA scenario_028 has 17 and 15, scenario_123 16 and 11; CT has 11 and 9. model_answers_summary.json exact on the frozen reference: MN 6, CO 16, MA 9, NY 4, against PA scenario_028 10" + }, + { + "claim": "The repository's own review process never flagged these references, and their values have not changed since the first run. The judges annotated every wrong answer on the four outputs, 149 rows, and none marked the reference as suspect. The September 22 audit flagged only New Jersey's payroll output, and its r05 fix added more of New Jersey's employee contributions to payroll_tax.", + "evidence": "annotations/us_full_run_20260612_policyengine_4_16_1_populace/us_audit_row_annotations.csv: 149 rows (38+2, 30, 36+1, 41+1), all reference_suspect False; us_case_notes.csv reference_suspect False for all four; reference_audit/2026-09-28/verification/flagged_sept22_wave.json lists only scenario_008:payroll_tax; reference_audit/2026-09-22/root_causes.json r05_nj_worker_ui; README.md line 43 (values since 2026-06-15)" + }, + { + "claim": "Exclusion's rule is symmetric but its effect is not. It gives the models that answered federal tax alone on all four most of what regeneration would give them, and the one model right on all four is the only model whose headline falls.", + "evidence": "verification/leaderboard_impact_exclude_models.csv and _regenerate_models.csv, exact_delta under exclude vs regenerate: claude-opus-5.5 +0.620 vs +0.688 (90%); ox-alpha +0.594 vs +0.688 (86%); gemini-3.8-flash +0.586 vs +0.688 (85%); claude-fable-5.1 +0.555 vs +0.688 (81%); gpt-6.1-sol -0.067 under exclusion (rank 8 to 9), the only negative delta. GPT-6 Sol's lead over Claude Opus 5.5 goes from 1.306 to 0.776 points; on within-1% from 0.972 to 0.460" + }, + { + "claim": "On several measures exclusion moves the board more than regeneration. It moves less only on rank swaps. Keep moves nothing.", + "evidence": "Computed from leaderboard_impact_{exclude,regenerate}_models.csv: mean |exact delta| 0.439 vs 0.325; mean exact delta +0.437 vs +0.169; score-rank changes 4 vs 2; headline rank swaps 5 pairs vs 10. leaderboard_impact.json leader_margin_over_always_zero_exact: 25.02 to 24.57 under exclusion vs 24.65 under regeneration" + }, + { + "claim": "The Maryland precedent the audit borrows supports keeping the scope as published. That adapter undid an engine change made after the freeze, and the prompt explicitly excluded the item it removed. The payroll prompt names no paid-leave exclusion, and these four values have held across every engine version the board used.", + "evidence": "reference_audit/2026-09-28/README.md:21 (rule 3, \"The benchmark's output definitions hold\"; the adapter \"restores that scope\"); reference_audit/2026-09-28/fixes/latest_md_local_output_scope.py docstring (state output prompt says \"excluding local income and payroll taxes\"; 1.755.4 had no local entry); policybench/benchmark_specs.json payroll_tax prompt (no paid-leave exclusion); README.md line 43" + }, + { + "claim": "The proposed records' basis requires an engine input the prompt never listed, and the README says no such input exists. Precedent has already stretched this rule: the 'who paid for the coverage' and NIIT exclusions named no engine input. So this is a mismatch with the docstring wording, not a bar to exclusion.", + "evidence": "proposed_exclusions.json \"basis\" (\"depends on an engine input the prompt never listed\"); policybench/reference_exclusions.py:10 docstring; README.md line 170 (\"policyengine-us has no input for the employer's deduction choice\"); frozen reference_exclusions.json records for scenario_002/107/121 and scenario_020" + }, + { + "claim": "For Massachusetts, the amounts published before the freeze support $805.01. After Acts 2026 c. 101 was approved on 2026-06-12, DFML's 2026 rates page still showed a 0.46% employee maximum. The act's own s. 43(a) refers to impacts in calendar year 2027. The statute reviewer found that a literal sequential reading of ss. 25-26 gives small employers an incoherent 40%/40% split. Rule 1 prefers amounts published before the freeze.", + "evidence": "law_research.json MA agency_guidance, Wayback capture 20260617060228 of the DFML rates page (0.28% medical plus 0.18% family); MA statute verdict rate_or_amount_issues (\"line 33 must be 6(d) ... That would give small employers 40%/40%\"); MA administration verdict (\"The 2026-06-17 copy has the same text\"); reference_audit/2026-09-22/README.md:7 (rule 1)" + }, + { + "claim": "Limits of this case, stated plainly. All eight reviews of these four programs found the deduction legally optional, and the law sets the required employee amount at $0. Colorado's statute says \"may deduct\". New York's Workers' Compensation Board says collection is \"allowed, but not required\". Precedent excluded the 'who paid for the coverage' question. Keep rests on the prompt's default rule and on how the law frames the employer's choice, not on a statutory duty to withhold. Massachusetts also carries the open c. 101 question.", + "evidence": "law_research.json verdicts: refuted False on all 8 reviews for MN, CO, MA and NY; CO 507(5); NY agency_guidance (WCB employer page); frozen reference_exclusions.json scenario_002 record" + }, + { + "claim": "Keep fits the README's own long-term fix. The next prompt version can state that the employee share set by law counts unless the facts say the employer pays it. That aligns the prompt with the existing reference and changes no published score.", + "evidence": "README.md line 7 (\"In the next prompt version, state how optional employee shares count\")" + } + ], + "rebuttals_to_readme": [ + "Line 9, \"16 of 16 independent reviews agree\": this backs a claim about the four states, but only 8 reviews cover MN, CO, MA and NY. The other 8 (CT, CA, PA, NJ) found those programs mandatory. All 16 reviews and the explanation classification came from one Claude Code workflow (wf_cecd10dd-abe, per law_research.json and explanations.json 'method'). The September 22 audit verified its fixes on GPT-6 Astra lanes; this audit has no reviewer outside that workflow. The reviewers judged how to classify each program, not whether the reference fails the repo's rules.", + "Line 9, \"the prompt never states the employer's choice ... rests on a fact the prompt does not give\": the README never addresses the prompt's default rules (prompts.py:19-22: unlisted facts are false, and no unlisted income may be inferred). In all four states an employer paying the employee share is framed as an election, and Minnesota's DEED treats it as extra wages.", + "Lines 62-67, \"Points that cut the other way\", leave out evidence the reviewers themselves said should be disclosed. Minnesota: DEED says an employer pickup is \"additional compensation ... as wages\". Massachusetts: the DFML subtitle \"Most Massachusetts employers must make payroll withholdings\"; c. 175M s. 1, whose definition of 'Contributions' includes payments by an employee; and s. 4(a)(ii), which requires notice of \"the employee's contribution amount and obligations\". Colorado gets no bullet at all, though its FAQ says an employer is \"required to pay the 0.44% employer's share and collect and remit the 0.44% of the employee's share\" and \"Most Colorado businesses should be deducting\". New York: DFS's \"Employers shall collect\"; 12 NYCRR 380-2.6, under which employees \"shall make regular family leave benefit contributions\"; the WCB's \"fully paid for by employees\"; and the PFL site's \"not optional for eligible employees\" and \"Ensure that you are collecting the employee contributions\".", + "Line 67 distinguishes California because an employer's payment counts as wages and the contribution stays the worker's. The law record shows Minnesota works the same way under DEED's guidance. And since California also lets the employer pay instead of deducting (s. 986(a)(2)), California's reference assumes withholding just as much.", + "Line 10, \"a careful reader could take the stated facts the other way\": in the evidence, one of 184 answers cites optional withholding. That model, Grok 4.7, calls Colorado's and Massachusetts' optional shares mandatory. The other answer classed as optional or employer-paid (Claude Sonnet 5 on MA) calls PFML \"employer-side\" and submits 12,318.65, which matches neither reading.", + "Line 103, \"the readings split by family\", presents guesses about the engine as readings of the law. Claude Opus 5.5 says it leaves MA PFML out \"on the assumption the model does not include it\". It leaves out Connecticut's mandatory contribution \"on the assumption that PolicyEngine does not model it\" and misses both Pennsylvania mandatory items. The README notes that Fable 5.1 and Ox Alpha drop mandatory programs but not Opus 5.5's misses on Connecticut and Pennsylvania.", + "Lines 11 and 101, \"take credit from 35 answers that include the program at exactly the engine's amount\": 12 of the 35 are not the engine's amount. Nine Colorado answers of 331.13 used the 2025 rate of 0.45% and are $0.40 off. Three Massachusetts answers of 14,193 are $0.34 off. They count only because of the $1 tolerance.", + "Line 11, \"Regenerating would mostly reward answers that never engaged with the program\": exclusion also stops penalizing those same 82 answers. Claude Opus 5.5 gets 90% of its regeneration gain under exclusion, Ox Alpha 86% and Claude Fable 5.1 81%. The README turns this argument only against regeneration.", + "Line 12, \"Exclusion is the symmetric remedy and moves the board least\": the rule is symmetric, but the effect is not. GPT-6.1 Sol, exact on all four, is the only model whose headline falls (-0.07, rank 8 to 9). GPT-6 Sol's lead over Claude Opus 5.5 shrinks from 1.31 to 0.78 points; the README reports the narrowed gap for regeneration but not for exclusion. Exclusion moves the board less only on rank swaps. On mean absolute change in the headline rate (0.439 vs 0.325), on score-rank changes (4 vs 2) and on the leader's margin over the always-zero baseline (24.57 vs 24.65) it moves more than regeneration. Keep moves nothing.", + "Line 133, \"Payroll exact match across models rises from 78.5% to 81.0%\": this rise comes only from removing 184 rows where most models were wrong. The always-zero baseline rises too, from 69.98% to 70.53%.", + "Line 148 cites the Maryland adapter for its form. The substance of that precedent (2026-09-28 rule 3: the benchmark's output definitions hold) restored the published scope after an engine change, and the prompt explicitly excluded the item. Here the prompt has no such exclusion, and the scope has stood since 2026-06-15. The precedent favors keeping these four references.", + "proposed_exclusions.json 'basis' requires an engine input the prompt never listed, but README line 170 says no such input exists. The records rest on a looser reading of the rule than the one they quote.", + "Line 66 says the c. 101 question \"matters only if the reference is kept\" but leaves out the evidence that resolves it in keep's favor. After c. 101 was approved, DFML's 2026 rates page (Wayback capture of 2026-06-17) still published the 0.46% employee maximum. The act's s. 43(a) refers to 2027. The statute reviewer found that a literal reading gives small employers an incoherent 40%/40% split. Under rule 1's preference for amounts published before the freeze, $805.01 stands.", + "Line 169 says \"Both options need them reworded\" about the case notes. Under keep they largely stand. The exception is scenario_032's note, which is already inaccurate under any option. It says \"many\" models called the deduction optional, but explanations.json shows one. It says \"The one model that did include the paid-leave premium\", but seven did.", + "The README is incomplete as a decision record. Line 3 has the placeholder 'DECISION_ID', and lines 161 and 165 are the unfilled placeholders 'INDEPENDENT_REVIEW' and 'RELEASE_CHECKLIST'. Its board-impact discussion gives no separate column for keep, which is the published board, and it never states that keep moves no rank." + ] + }, + { + "option": "regenerate", + "strongest_case": "The prompt states the output's scope: \"mandatory employee state payroll taxes. Exclude employer payroll taxes.\" Rule 3 of the 2026-09-28 audit: the benchmark's output definitions hold. Its Maryland precedent restored a stated scope with a list-only adapter and kept the outputs scored, because \"with the adapter the output does not depend on the unlisted county\". The payroll adapter does the same: once the definition is applied, no unlisted fact moves the answer.\n\nThe audit's own law record settles what that covers. All 8 reviews of the Minnesota, Colorado, Massachusetts and New York programs (none refuted) find the premium is the employer's and put the employee's legal floor at $0. They weighed and rejected the \"must\" and \"shall\" passages the README cites. Both clauses of the definition give the federal-only value; the README concedes \"on that wording the frozen value is too high\".\n\nIf \"mandatory\" is read as turning on the employer's election, that election is unlisted, and the run's preface says to treat unlisted numeric inputs as 0 and other unlisted facts as false. Only the frozen value assumes something unstated: that the employer deducts the maximum. The regenerated value survives every caveat in the law record (Acts 2026 c. 101, private plans, collective agreements, employer size), because each moves only the cap.\n\nThe rules pick references by stated facts and law, not by whom they credit. Regeneration keeps all 1,928 outputs scored and the always-zero baseline unchanged, and moves the average headline exact rate less (mean absolute change 0.325 vs 0.439 points). Under the adapter, single earners at the four wage levels reproduce all four values; CA, PA, CT and NJ are unchanged.\n\nAgainst it: the 40-hour precedent excluded rather than applying the zero default, and the adapter keeps Delaware's, Maine's and Vermont's optional shares.", + "points": [ + { + "claim": "The output's definition is stated in the prompt and names exactly the two tests that remove these shares: 'mandatory employee state payroll taxes' and 'Exclude employer payroll taxes'. The same text was in force when the frozen run was generated.", + "evidence": "policybench/benchmark_specs.json:34; the identical text is present at commit 58b65105, the last commit before the run's scenarios were generated at 2026-06-12T05:15Z (git show 58b65105:policybench/benchmark_specs.json)" + }, + { + "claim": "The repo's governing precedent for a stated output definition that the engine's aggregate overshoots is an output-scope adapter that keeps the outputs scored, not exclusion. In the Maryland case the review accepted that adapter because applying the definition removed the dependence on the unlisted fact.", + "evidence": "reference_audit/2026-09-28/README.md:21 (rule 3, 'The benchmark's output definitions hold'); reference_audit/2026-09-28/clusters.json:119 ('Pinning Worcester gives the same value, so with the adapter the output does not depend on the unlisted county'); reference_audit/2026-09-28/fixes/latest_md_local_output_scope.py (list-only change, same form as fixes/payroll_mandatory_scope.py); proposed_regenerations.json:6 cites this basis" + }, + { + "claim": "The exclusion precedents that turn on wording apply when the definition is silent. Here it speaks.", + "evidence": "reference_audit/2026-09-22/root_causes.json: NIIT entry, unlisted_input 'whether federal income tax before refundable credits includes the net investment income tax, which the output's definition does not say'; reference_audit/2026-09-22/README.md:10 (rule 4: 'input or definition the prompt never states')" + }, + { + "claim": "The audit's own law record answers what 'mandatory' covers for these four programs. All 8 reviews (statute and administration lens for MN, CO, MA and NY) refuted nothing, classify each as an optional employer pass-through, and put the employee's legal floor at $0. Each researcher rated confidence 'high'.", + "evidence": "verification/law_research.json, programs MN/CO/MA/NY verdicts: MN admin 'the mandatory employee amount is $0'; CO statute 'The legally required employee amount for an ordinary private-sector W-2 employee is $0'; CO admin 'The legally mandatory employee amount is $0'; MA statute and admin 'legally mandatory employee amount of $0'; NY statute 'The legal minimum is $0 if the employer chooses not to collect'; NY admin 'ranges from $0 to $443.11'" + }, + { + "claim": "The reviewers considered and rejected each 'must' and 'shall' passage the README relies on. The MA 'must withhold' line is agency summary prose that the binding regulation contradicts.", + "evidence": "law_research.json MN statute verdict ('If any' expressly contemplates a zero remainder ... It does not require any positive employee amount); MA admin verdict ('The \"must withhold\" overview line is loose summary prose, contradicted by the page's own detailed text and by the regulation'); NY admin verdict ('The manner provided is permissive employer collection under s.209(4)')" + }, + { + "claim": "Even if 'mandatory' is read as conditional on the employer's election, the prompt resolves that unlisted fact. The preface tells models to treat unlisted numeric inputs as 0 and other unlisted facts as false, which means no deduction and the regenerated value. The exclusion record itself says the federal-only value follows 'with an employer that does not deduct it'.", + "evidence": "policybench/prompts.py:19-20 (also present at commit 58b65105); proposed_exclusions.json alternative_reading for each of the four records" + }, + { + "claim": "The regenerated value is the only one that no caveat in the law record moves. Every caveat (Acts 2026 c. 101, private or self-insured plans, collective agreements, employer size, NY's 53 Thursdays) changes the employee cap, not the $0 floor. The frozen value moves with several of them. MA under the literal c. 101 reading would be $1,351.02, not $805.01.", + "evidence": "program_classification.json MA rate_2026; law_research.json MN/CO/MA/NY caveats; README.md:66 ('matters only if the reference is kept')" + }, + { + "claim": "The exclusion reason code does not fit. reference_depends_on_unlisted_input is defined as dependence on 'an engine input the household facts never carried'. policyengine-us has no input for the employer's choice: the formulas apply the rate to taxable wages unconditionally.", + "evidence": "policybench/reference_exclusions.py:14-18; README.md:170; read in the 2.15.17 venv: variables/gov/states/mn/tax/payroll/paid_leave/mn_employee_paid_leave_contribution.py, ma/.../ma_employee_paid_leave_contribution.py, co/.../co_employee_famli_contribution.py (formula = rate * taxable wages)" + }, + { + "claim": "I checked the adapter independently on policyengine-us 2.15.17 with single-earner households at each scenario's wages. It gives exactly the four regenerated totals (MN 2,218.50, CO 312.74, MA 13,387.65, NY 7,664.92) and leaves every mandatory program unchanged (CA 4,710.83, PA 4,632.00, CT 13,496.16, NJ 2,162.76). The checked-in sweep also reports that the adapter moves only these four outputs.", + "evidence": "scratchpad/review/advocate-regenerate/check_adapter.py output; verification/sweep_payroll_scope.log ('adapter: 4 outputs move at all'); fixes/payroll_mandatory_scope.py sha256 74200522... matches proposed_regenerations.json" + }, + { + "claim": "How much each option moves the board depends on the metric. Regeneration moves the average model less on the headline and much less on score. It keeps the always-zero baseline and all 4,554 payroll rows.", + "evidence": "Computed from verification/leaderboard_impact_{exclude,regenerate}_models.csv. Headline exact mean |delta|: exclude 0.439, regenerate 0.325. Score max |delta|: 0.248 vs 0.040. outputGroupScore max |delta|: 0.055 vs 0.013. Score-rank changes: 4 vs 2. Payroll program exact: +2.50 vs +1.03 points. Always-zero baseline: 69.98 to 70.53 vs unchanged. Leader margin over the baseline: 24.57 vs 24.65. Payroll rows: n 4,554 to 4,370 vs unchanged (leaderboard_impact_*_programs.csv). Exclusion has fewer headline swaps (5 vs 10 pairs) and a smaller maximum headline |delta| (0.620 vs 0.688)." + }, + { + "claim": "Under regeneration the four outputs stay as the benchmark's only test of the mandatory-versus-optional distinction the prompt draws. Exclusion removes them until a new prompt and a fresh run.", + "evidence": "README.md:7 (recommendation to restate in the next prompt version); payroll_decomposition.csv (only these 4 scored outputs carry an optional program)" + }, + { + "claim": "Honest counter-evidence: an earlier audit excluded instead of applying the preface's zero default, in the weekly-hours case.", + "evidence": "reference_audit/2026-09-22/root_causes.json:104 (weekly_hours_worked_before_lsr: 'The prompt lists no hours worked and says to treat unlisted numeric inputs as 0; the reference assumed 40 hours'). It differs here: hours is an engine input, SNAP's output definition says nothing about hours, and the zero-hours alternative was 'not a certified entitlement'." + }, + { + "claim": "Honest counter-evidence: one NY reviewer framed the employee share as a range consistent with the prompt's facts. That supports the unlisted-fact framing.", + "evidence": "law_research.json NY administration verdict: 'Under NY law the employee share consistent with the prompt's facts ranges from $0 to $443.11'" + }, + { + "claim": "Honest counter-evidence: as a definition adapter, the regeneration module is incomplete. It keeps Delaware, Maine and Vermont shares that the same law research classifies as optional pass-throughs, and it leaves Washington PFML (mixed) undecided. No benchmark household is affected.", + "evidence": "fixes/payroll_mandatory_scope.py OPTIONAL lists only CO, MA, MN, NY; law_research.json OTHER classifies DE, ME and VT as optional_employer_pass_through; my 2.15.17 run at $50,000 of wages keeps DE 200.00, ME 250.00 and VT 55.00 under the adapter" + }, + { + "claim": "Honest counter-evidence: the answers that match the frozen value come from models stronger on the mandatory programs.", + "evidence": "Computed from explanations.json and model_answers.csv. On average, the 35 frozen-matching answers come from models that include a mandatory state program on 5.57 of 7 mandatory outputs, against 2.88 for the 82 answers regeneration would credit." + } + ], + "rebuttals_to_readme": [ + "README.md:9 '16 of 16 independent reviews agree' is attached to a claim about the four optional states, but only 8 reviews cover MN, CO, MA and NY. The other 8 review CT, CA, PA and NJ, which are classified mandatory (law_research.json).", + "README.md:9 leaves out that every review of the four programs puts the employee's legal floor at $0, and that six say outright that the mandatory or required employee amount is $0. That finding is the regenerated value, not a reason to exclude.", + "README.md:10 calls the readings 'a real split', yet the README's own law section (README.md:62-67) and all 8 reviews resolve each 'must'/'shall' passage against the mandatory reading. The MA 'must withhold' quote is agency overview prose that the MA administration reviewer called 'loose summary prose, contradicted by the page's own detailed text and by the regulation' (458 CMR 2.05(5)(c)). The README quotes it without that judgment.", + "README.md:10 concedes 'On that wording the frozen value is too high'. The prompt's wording is what 2026-09-28 rule 3 says must hold, so the concession grants the premise for regeneration.", + "README.md:10 treats '22 of 46 models included the Massachusetts and New York contributions' as evidence of ambiguity. It picks the two highest counts: inclusion is 7 (MN), 18 (CO), 22 and 22, or 69 of 184 overall. By the same test, Pennsylvania's mandatory contribution (12 and 15 include; 17 and 16 say none, README.md:97) would be 'a real split', yet it stays scored. The README itself says the explanations 'look the same' where the law requires the contribution, so model splits cannot tell ambiguous outputs from clear ones. Of the 22 MA includers only 9 match, with rates from 0.24% to 0.604% (explanations.json).", + "README.md:10 applies the 'careful reader could take the stated facts the other way' test from reference_exclusions.py to a dispute about law, not facts. The stated facts (wages, state) are complete. The one unstated fact, the employer's election, is covered by the preface's default for unlisted facts (prompts.py:19-20), which points to the regenerated value.", + "The proposed exclusion uses reason code reference_depends_on_unlisted_input, which reference_exclusions.py:14-18 defines as dependence on 'an engine input the household facts never carried'. README.md:170 itself says policyengine-us has no input for the employer's deduction choice. The class that fits is the 2026-09-28 rule 3 output-scope adapter.", + "README.md:11 'Regenerating would mostly reward answers that never engaged with the program' applies a criterion no repo rule contains. The rules choose a reference by stated facts and pre-freeze law, and the scorer reads numbers, not reasons, on every other output too.", + "README.md:11 is also overstated on its own terms. Four of the 6 'other' answers it would credit name the program and leave it out: Claude Opus 5.5 on MA ('about $805 at 0.46% ... is excluded'), Claude Sonnet 5.5 on MA, Claude Fable 5 on NY and Claude Opus 4.7 on CO. 66 of the 82 credited answers come from models that include a mandatory state program elsewhere. The README's paraphrase 'say the state has no employee payroll tax' drops the qualifier 'mandatory', which 32 of the 46 such quotes on the four outputs carry. Under the law record that qualified statement is true, although the same phrasing on PA (11 of 17 and 13 of 16) shows it often echoes the prompt.", + "README.md:11 and :101 say regeneration 'would take credit from 35 answers that include the program at exactly the engine's amount'. Nine of those 35 are Colorado answers of 331.13 that used the pre-2026 0.45% FAMLI rate ($18.40, not the engine's $17.99) and count as exact only through the $1 tolerance (explanations.json scenario_043). Four of the 35 call the contribution 'mandatory', which the law record says it is not. Exclusion also takes credit from all 35 answers.", + "README.md:12 'Exclusion is the symmetric remedy' turns a description of how exclusion works (reference_exclusions.py:35) into a reason to choose it. The rules assign the remedy by cause, and a regenerated reference applies equally to every model.", + "README.md:12 'moves the board least' holds only for rank swaps (5 vs 10 headline pairs) and maximum headline |delta| (0.620 vs 0.688). On the other measures exclusion moves the board more. Mean headline |delta| is 0.439 vs 0.325. Score max |delta| is 0.248 vs 0.040. outputGroupScore max |delta| is 0.055 vs 0.013. Payroll program exact rises 2.50 points vs 1.03. Exclusion also raises the always-zero baseline from 69.98 to 70.53 and cuts GPT-6 Sol's margin over it to 24.57, against 24.65 under regeneration (leaderboard_impact_*.csv/json).", + "README.md:12 cites 'puts Claude Opus 5.5 first on the within-1% metric' in the recommendation. Rank outcomes are not a criterion in any rule. Opus 5.5 gains nearly as much under exclusion (93.70 to 94.32) as under regeneration (94.39); the difference between the options falls mostly on GPT-6 Sol (95.09 vs 94.63).", + "README.md:103 'Claude Fable 5.1 and Ox Alpha leave out every state program, mandatory ones included' is wrong for Ox Alpha. It includes California SDI on scenario_022 ('mandatory California employee SDI at 1.2% = $632') and scenario_023 (explanations.json).", + "README.md:103 does not say that Claude Opus 5.5, the model regeneration helps most, matches all four California SDI references at the 2026 1.3% rate (model_answers.csv), so its federal-only answers on the optional programs are not a general failure to know state programs.", + "README.md:7 recommends stating in the next prompt version how optional employee shares count. The current prompt already does so through 'mandatory' and 'Exclude employer payroll taxes', and through its default for unlisted facts. Exclusion forfeits the four outputs until a fresh run.", + "README.md:66 says the Acts 2026 c. 101 question 'matters only if the reference is kept'. That robustness argues for regeneration: the federal-only value is the only one that no caveat in the law record (c. 101, private plans, collective agreements, employer size) changes.", + "The published case notes mark legally correct reasoning as error. scenario_032's note counts 'the paid-leave deduction is optional' as a model mistake, and scenario_081's note calls PFML 'mandatory' (annotations/.../us_case_notes.csv:294, :641). Regeneration lets the reworded notes state what the law record found. Exclusion records the question as one a careful reader could take either way, which contradicts the audit's own unanimous classification.", + "The README is incomplete. Placeholders DECISION_ID, INDEPENDENT_REVIEW and RELEASE_CHECKLIST are unfilled (README.md:3, :161, :165)." + ] + } + ], + "judge": { + "best_supported_option": "exclude", + "readme_recommendation_survives": true, + "reasons": "Exclusion is the best-supported option, and the README's recommendation survives. Several of the README's supporting arguments are wrong or overstated and need fixing before release.\n\nKEEP FAILS UNDER THE REPO'S OWN TEST. Rule 1 of 2026-09-22 and the reference_exclusions.py docstring say a scored reference must \"follow from the facts the prompt states under the law\".\n- All 8 reviews of MN, CO, MA and NY found the engine amounts to be legal maxima. None refuted the classification. Each put the legally required employee amount at $0.\n- I read the engine source this session. mn_ and ma_employee_paid_leave_contribution state \"assuming the employer withholds the maximum permitted employee share\". co_employee_famli_contribution is rate x taxable wages, with no input for the employer's choice.\n- So the frozen values (MN 127.60, CO 17.99, MA 805.01, NY 443.11) depend on an engine assumption, not on a stated fact.\n- Keep's appeal to the prompt's default (\"unlisted ... fact ... as false\") fails because the regenerate brief applies the same default with the opposite result. The default does not say which state is the \"fact\": the employer deducting, or the employer paying.\n- Keep's reading of the Maryland precedent is also wrong. 2026-09-28 rule 3 says the output definition holds. It does not say the published scope holds.\n\nREGENERATE IS THE STRONG ALTERNATIVE, BUT IT REQUIRES THE STRICT READING OF \"MANDATORY\" AS THE ONLY READING.\n- The law record itself treats the employee amount as depending on the employer's choice:\n - MN research caveat: \"Under a strict reading of the benchmark's 'mandatory employee state payroll taxes', the mandatory Minnesota employee amount is $0\".\n - MA summary: \"$0, unless one adopts the engine's assumption that the employer deducts the maximum\".\n - NY administration verdict: \"the employee share consistent with the prompt's facts ranges from $0 to $443.11\".\n- MN subd. 3 (\"Employees ... must pay the remaining portion, if any\") and NY WCL 209(1) (\"shall contribute\") place statutory duties on the employee. Once the employer withholds, the employee cannot opt out.\n- A careful reader could therefore count the statutory employee share as a \"mandatory employee state payroll tax\". That makes this a reference that turns on a fact or definition the prompt does not state. 2026-09-22 rule 4 (\"input or definition the prompt never states\") and 2026-09-28 rule 4 send that case to exclusion.\n\nTHE CLOSEST PRECEDENTS ALL EXCLUDED.\n- r24 (scenario_002, 107 and 121): who paid for the coverage, which is an employer-arrangement fact that decides the tax.\n- r25 NIIT (scenario_020): definition ambiguity.\n- r14 (scenario_056 and 112): the reference assumed 40 hours where the zero default arguably applied. The repo excluded rather than regenerating to the default.\n- The Maryland adapter, which the regenerate brief relies on, differs in two ways. Its definition was explicit (\"excluding local income and payroll taxes\"). And it reproduced the board's published values (scenario_068 equal; scenario_078 +0.539). Here regeneration would change values that have stood since 2026-06-15, by $17.99 to $805.01.\n- \"How the records follow\" #3 in 2026-09-22 prescribes exclusion with an alternative value for an unlisted-input reading, which is what proposed_exclusions.json does.\n\nREGENERATE'S REAL STRENGTHS. The README must answer them, not dismiss them:\n- The README concedes \"On that wording the frozen value is too high\".\n- The $0 floor is unchanged by every caveat in the law record (c.101, private plans, collective agreements, employer size).\n- The scope adapter is clean: the checked-in sweep log says the adapter moves 4 outputs and nothing else.\n\nTHE BOARD-IMPACT AND MODEL-BEHAVIOUR ARGUMENTS ON ALL SIDES ARE NOT RULE CRITERIA AND DO NOT DECIDE THIS.\n- Model inclusion rates are the same on mandatory programs: PA 12 and 15 include, CT 20.\n- Exclusion moves the board less on some metrics and more on others. Headline swaps are 5 vs 10 pairs, but mean |delta| is 0.439 vs 0.325 and score max |delta| is 0.248 vs 0.040.\n- The asymmetry argument cuts both ways. GPT-6.1 Sol, exact on all four, falls 0.067 under exclusion but 0.688 under regeneration.\n\nWHAT THE RECORD NEEDS BEFORE RELEASE.\n- The basis line in proposed_exclusions.json cites \"an engine input the prompt never listed\". README line 170 says no such input exists. The line should cite 2026-09-22 rule 4 and 2026-09-28 rule 4, and the r24 and r25 precedents.\n- The placeholders DECISION_ID, INDEPENDENT_REVIEW and RELEASE_CHECKLIST must be filled.\n- All 16 reviews ran in one Claude Code workflow (wf_cecd10dd-abe). The 2026-09-22 fixes were checked on GPT-6 Astra lanes; this audit has no outside-workflow review yet.\n\nI modified nothing in the checkout. My scratch files are in /private/tmp/claude-501/-Users-maxghenis-Library-Application-Support-Claude-scratch-workspaces-ee3763f4-e7ea-4177-bcaf-1362266768c1-4457aa9d-2dc0-4bcf-a128-e50adae6be25-scratch-2026-09-22-c3da5e/224fe5b6-f6e8-4204-af8f-2508970193bc/scratchpad/review/judge/ (stats.py, law_dump.txt).", + "readme_errors_or_overstatements": [ + "Line 9, \"16 of 16 independent reviews agree\": the claim is about the four optional states, but only 8 reviews cover MN, CO, MA and NY. The other 8 confirm CT, CA, PA and NJ as mandatory_employee_withholding. All 16 ran in one Claude Code workflow (law_research.json method: wf_cecd10dd-abe), so \"independent\" means separate agents in that workflow, not reviewers outside it.", + "Line 10, \"22 of 46 models included the Massachusetts and New York contributions. That is the case the exclusion rule covers\": model inclusion is not evidence of a careful reader's split. Line 89 itself says the explanations \"look the same\" where the law requires the contribution: PA 12 and 15 include, CT 20. Across the four outputs inclusion is 7, 18, 22 and 22 (69 of 184). Of the 22 MA includers only 9 are exact.", + "Line 10 cites MN \"must pay\", NY \"shall contribute\" and MA \"must withhold\" as making the split \"real\". It does not report that every reviewer weighed these passages and rejected them as setting any required amount. The MA administration reviewer called \"must withhold\" \"loose summary prose, contradicted by the page's own detailed text and by the regulation\". The defensible ground is narrower: the legal floor is $0, but the benchmark's word \"mandatory\" can be read as the statutory employee share the employer withholds by default, or as the legal floor. The README should say that, not imply the law itself is split.", + "Line 10 concedes \"On that wording the frozen value is too high\" and moves on. Unexplained, that concession grants the premise of 2026-09-28 rule 3 (output definitions hold) and supports regeneration.", + "Lines 11 and 101, \"35 answers that include the program at exactly the engine's amount\": 9 of the 35 are Colorado answers of 331.13 at the stale 0.45% rate ($18.40, not $17.99). They count as exact only through the $1 tolerance (explanations.json scenario_043). The 3 MA answers of 14,193 are whole-dollar roundings at the right 0.46% rate.", + "Lines 11 and 101, \"Regenerating would mostly reward answers that never engaged with the program\": no repo rule makes that a criterion. It is also overstated:\n- 4 of the 6 \"other\" answers name the program and leave it out (Claude Opus 5.5 MA, Claude Sonnet 5.5 MA, Claude Fable 5 NY, Claude Opus 4.7 CO).\n- 66 of the 82 newly credited answers come from models that include a mandatory state program elsewhere.\n- 25 of the 36 \"no state tax\" answers say \"no mandatory\", which is literally true under the law record. The README's paraphrase drops the qualifier.\n- Exclusion also stops penalizing the same answers: Claude Opus 5.5 gets 90% of its regeneration gain under exclusion (+0.620 vs +0.688), Ox Alpha 86%, Gemini 3.8 Flash 85%, Claude Fable 5.1 81%.", + "Line 12, \"moves the board least\": this holds only for headline rank swaps (5 vs 10 pairs) and max |delta| (0.620 vs 0.688). Exclusion moves more on:\n- mean |exact delta|: 0.439 vs 0.325\n- mean exact delta: +0.437 vs +0.169\n- score max |delta|: 0.248 vs 0.040\n- outputGroupScore max |delta|: 0.055 vs 0.013\n- score-rank changes: 4 vs 2\n- the always-zero baseline: 69.98 to 70.53, vs unchanged\n- GPT-6 Sol's margin over that baseline: 24.57 vs 24.65\nKeep moves nothing. Board movement is not a rule criterion either way.", + "Line 12 puts \"Claude Opus 5.5 first on the within-1% metric\" in the recommendation. Rank outcomes are not a rule criterion, and listing them invites a reading that the choice is outcome-driven. The README also omits that exclusion narrows the GPT-6 Sol lead over Claude Opus 5.5 from 1.306 to 0.776 points on exact match, and from 0.972 to 0.460 on within-1%. It reports the narrowed gap only for regeneration.", + "Line 103, \"Claude Fable 5.1 and Ox Alpha leave out every state program, mandatory ones included\": wrong for Ox Alpha. It includes CA SDI on scenario_022 (4,658, \"mandatory California employee SDI at 1.2% = $632\") and scenario_023 (1,544).", + "Line 103, \"the readings split by family\": this presents engine guesses as legal readings. Claude Opus 5.5 leaves out MA PFML \"on the assumption the model does not include it\", and CT's mandatory contribution \"on the assumption that PolicyEngine does not model it\". It also misses PA scenario_028 and scenario_123, while matching all four CA SDI references. The README notes the mandatory misses of Fable 5.1 and Ox Alpha but not those of Opus 5.5.", + "Lines 62-67, \"Points that cut the other way\", leave out counter-evidence the reviewers said should be disclosed:\n- MN: DEED's taxes page says an employer pickup is \"additional compensation ... as wages\", and the employee may deduct the amount \"that the employer pays on their behalf\".\n- CO: no bullet at all, despite the FAQ's \"required to pay the 0.44% employer's share and collect and remit the 0.44% of the employee's share\" and \"Most Colorado businesses should be deducting\".\n- MA: the subtitle \"Most Massachusetts employers must make payroll withholdings\"; c.175M s.1 (Contributions include employee payments); s.4(a)(ii) notice of \"the employee's contribution amount and obligations\".\n- NY: DFS \"Employers shall collect employee contributions\"; 12 NYCRR 380-2.6 \"shall make regular family leave benefit contributions\"; the PFL site's \"Ensure that you are collecting\"; WCB \"fully paid for by employees\".", + "Line 62, \"each engine amount equals the law's largest employee share to the cent\": overstated for NY DBL. The cap is $0.60 a week, and 2026 has 53 Thursdays, so a Thursday payroll's legal maximum is $31.80, not the engine's $31.20 (within $1). It is also overstated for MA under the literal c.101 reading ($1,351.02), though the README discloses that one.", + "Line 66 describes the c.101 question without the pre-freeze evidence for 0.46%:\n- DFML's rates page captured 2026-06-17, after c.101's approval, still shows the 0.46% maximum.\n- The act's s.43(a) refers to calendar year 2027.\n- The statute reviewer found that a literal sequential reading of ss.25-26 gives small employers 40%/40%.\nThis matters only to keep, but the record should be complete.", + "Line 169, \"Both options need them reworded\": understated. The scenario_032 note is already wrong under every option. It says \"The one model that did include the paid-leave premium used 0.35%\", but explanations.json has 7 includers and 6 exact matches at 2,346.10. The scenario_081 note's \"mandatory\" contradicts the law record whichever option is adopted.", + "proposed_exclusions.json \"basis\" says the reference \"depends on an engine input the prompt never listed\". README line 170 and the engine source (co_employee_famli_contribution is rate x co_famli_taxable_wages) show that no such input exists. The basis should cite 2026-09-22 rule 4 (\"input or definition the prompt never states\") and 2026-09-28 rule 4, and the r24 and r25 precedents. Those used reference_depends_on_unlisted_input with no engine input.", + "The decision record is incomplete. DECISION_ID (line 3), INDEPENDENT_REVIEW (line 161) and RELEASE_CHECKLIST (line 165) are unfilled placeholders. The README also never cites the closest precedent for its own recommendation: r24, \"who paid for the coverage\", an employer-arrangement fact the prompt omits, which was excluded." + ], + "arguments_readme_must_answer": [ + "Regenerate brief, rule 3: \"The benchmark's output definitions hold\" (2026-09-28). The README concedes the frozen value is too high on the prompt's wording, and all 8 reviews put the legally required employee amount at $0. The README must explain why this is a two-reading case under rule 4 and not a definition case under rule 3. It should also say why the Maryland adapter does not govern: that definition was explicit, and the adapter preserved the published values (scenario_068 equal, scenario_078 +0.539), where regeneration would change values that have stood since 2026-06-15.", + "Regenerate brief: the regenerated value is the only one no caveat in the law record moves (c.101, private and self-insured plans, collective agreements, employer size, NY's 53 Thursdays), because each caveat changes only the cap and not the $0 floor.", + "Regenerate brief and keep brief, mirror images: each says the preface (\"Treat ... any other unlisted household fact ... as false\"; \"Do not infer unlisted income\") resolves the employer's choice in its favor. The README never addresses the preface. Its best answer is that the default is indeterminate here: whether the deduction or the pickup is the \"fact\" depends on each statute's framing. That is the 40-hour precedent (r14), which excluded instead of applying the zero default.", + "Both briefs: the reason code reference_depends_on_unlisted_input is defined in reference_exclusions.py as dependence on \"an engine input the household facts never carried\", and README line 170 says no such input exists. The README must cite the precedents that used this code for a definition or arrangement question without an engine input (r25 NIIT scenario_020; r24 scenario_002, 107 and 121), or reword the basis.", + "Keep brief: the statutes frame the employer paying the employee share as an election. CO 7 CCR 1107-1 s.1.4(6)(A): \"considered to have elected to pay\". MN DEED: employers \"may choose to pay more than their required portion\". MA 458 CMR 2.05(5)(c): \"opts to deduct a lower percentage\". On that framing the no-election default is the maximum deduction.", + "Keep brief: DEED treats a Minnesota employer pickup as additional wages, with the 0.44% still the employee's for federal tax. That parallels the structure the README uses at line 67 to call California mandatory. The README's distinction has to rest on state law: California's worker contribution is still owed when the employer pays (s.986(a)(2), DE 231Q gross-up), while Minnesota's employee owes only \"the remaining portion, if any\".", + "Keep brief: exclusion's effect is not neutral. GPT-6.1 Sol, exact on all four, is the only model whose headline falls under exclusion (-0.067, rank 8 to 9). The README should report that, together with the exclusion-case narrowing of the GPT-6 Sol over Claude Opus 5.5 gap (1.306 to 0.776 exact; 0.972 to 0.460 within-1%). It should also note that regeneration costs GPT-6.1 Sol more (-0.688).", + "Regenerate brief: on the metrics listed above, exclusion moves the board more than regeneration on everything except headline rank swaps and max |delta|. The README must either drop \"moves the board least\" or qualify it.", + "Regenerate brief: the inclusion counts cannot separate ambiguous outputs from clear ones. The PA mandatory pattern (17 and 16 say there is no state tax) matches the optional outputs, so the \"22 of 46\" argument for a real split must be replaced with the definitional argument.", + "Regenerate brief: under exclusion the four outputs stop testing the mandatory-versus-optional distinction until a new prompt version and a fresh run. The README should state that cost and commit to the prompt change on line 7.", + "Regenerate brief: the published case notes count legally correct reasoning as model error. scenario_032 treats \"the paid-leave deduction is optional\" as a mistake, and scenario_081 calls PFML \"mandatory\". The README should say how the notes for excluded outputs will be reworded.", + "Keep brief: the judges annotated all 149 wrong answers on the four outputs and flagged no reference (reference_suspect False on every row). The README should note this, and note that the judges were not asked to research the employer-deduction law, so the lack of a flag is weak evidence.", + "Keep brief: pre-freeze evidence supports the 0.46% Massachusetts cap: the 2026-06-17 DFML capture, c.101 s.43(a), and the 40%/40% incoherence of the literal reading. This should be recorded even though it matters only to keep." + ] + } + }, + "release_checklist": { + "exclude": [ + { + "item": "Before installing anything, record Max's ruling. Set decided_on on the four records to the ruling date. Fill the README placeholders DECISION_ID, INDEPENDENT_REVIEW and RELEASE_CHECKLIST, which are still literal tokens at this commit.", + "where": "reference_audit/2026-10-05-payroll/proposed_exclusions.json:3 (status), README.md:5 and the INDEPENDENT_REVIEW / RELEASE_CHECKLIST lines" + }, + { + "item": "Insert the four records into reference_exclusions.json BEFORE the trailing scenario_023 head_medicaid_eligible record and write with json.dumps(indent=2)+'\\n'. The test requires the list's tail to equal final_actions.json audit_exclusions. Appending at the end fails it; observed on a scratch clone: tail [...'2026-10-05'] != [...'2026-09-29'].", + "where": "paper/snapshot/20260501/runs/us_full_run_20260612_policyengine_4_16_1_populace/reference_exclusions.json; tests/test_reference_upgrade.py:221-223, 229-232" + }, + { + "item": "Put any new 'derivation' text BEFORE the 'On 2026-09-29 the references moved to policyengine-us 2.15.17' marker. Otherwise change the test, which requires derivation == base + exclusion_derivation(...).", + "where": "tests/test_reference_upgrade.py:224-228; reference_audit/2026-09-28/scripts/build_references_latest.py:173-198" + }, + { + "item": "Decide whether to give the four records one shared unlisted_input string. As drafted they add four separate keys of count 1 each to the manifest's by_unlisted_input tally and to paper_results.excluded_outputs_by_input. #191 uses one shared string for its three.", + "where": "scripts/freeze_snapshot.py:330-337; policybench/paper_results.py:1125-1133" + }, + { + "item": "Add four developer adjudications to us_adjudications.json. Each needs every REQUIRED_FIELD plus: judge_model claude-opus-5-5, judge_failure_source llm_error, judge_failure_subtype state_local_rule, judge_reference_suspect false, adjudicated_failure_source prompt_ambiguity, excluded_from_scoring true, and reference_verdict unlisted_input with a reference_basis. The freeze refuses the run unless the adjudication set equals the exclusion set and the judge fields match each case's verdict.json.", + "where": "annotations/us_full_run_20260612_policyengine_4_16_1_populace/us_adjudications.json; policybench/adjudications.py:26-38, 84-121; scripts/freeze_snapshot.py:315-328, 358-403" + }, + { + "item": "Take judged_on_utc from the published audit tree: 2026-09-29 for 032, 081 and 082, and 2026-09-22 for 043. Point the freeze's AUDIT_CASES_DIR at results/local/gpt61sol-stage/gpt61sol-v1/audit/cases. The default unified_audit tree holds older verdicts for 032, 081 and 082 (dated 2026-09-22/23; sha a5c2b9ca, e4da4b03, 37043eaf, against the published 354482e6, b29fd367, b7a89645). The classes are the same, but the dates and judge-provenance tallies would come from a stale tree.", + "where": "scripts/freeze_snapshot.py:175, 237-313, 406-410" + }, + { + "item": "Add the new adjudication wave. Extend date_conventions, which names only 2026-09-05, 2026-09-22 and 2026-09-29. Add a wave_releases entry and four case entries to the judge evidence, or give the wave its own evidence file. Update the test, which raises KeyError '2026-10-05' (observed), looks up evidence['cases'][case] at :763, and asserts wave == '2026-09-29' at :766. While there, fill the 2026-09-29 wave's commit: it is still null although d616e67c merged.", + "where": "us_adjudications.json date_conventions; reference_audit/2026-09-28/verification/judge_verdicts.json; tests/test_adjudications.py:694-784" + }, + { + "item": "Scope the restatement test to release 20260929's cases. It asserts the current record's cases equal that release's exactly, so any added entry fails it (observed).", + "where": "tests/test_adjudications.py:639" + }, + { + "item": "Apply the adjudications with scripts/apply_adjudications.py. This relabels 145 llm_error rows to prompt_ambiguity: 38 on 032, 30 on 043, 36 on 081 and 41 on 082. 43 of them also change subtype to state_local_rule (27 thresholds_rates, 13 payroll_tax_base, 3 other). The 4 parse_contract_failure rows stay: glm-5.2 and kimi-k2.6 on 032, kimi-k2.6 on 081 and on 082. The four case notes take prompt_ambiguity and the appended adjudication sentence.", + "where": "annotations/.../us_audit_row_annotations.csv, us_case_notes.csv; policybench/adjudications.py:206-264, 267-354; tests/test_paper_results.py:364-380" + }, + { + "item": "Reword case note 032. It counts the optional reading as a model error: 'many said outright that Minnesota has no mandatory employee payroll tax or that the paid-leave deduction is optional'. It also lists 'Minnesota's Paid Leave employee contribution of $127.60' as a fixed piece of the reference.", + "where": "us_case_notes.csv scenario_032 payroll_tax" + }, + { + "item": "Reword case note 043. It says 'Colorado's FAMLI program charges employees half of a 0.88% premium' and treats 'dropped FAMLI on purpose' as an error. It also gives FICA as $312.75 in one sentence and $312.73 in the next; the engine's federal part is $312.74.", + "where": "us_case_notes.csv scenario_043 payroll_tax" + }, + { + "item": "Reword case note 081, which says 'the mandatory Massachusetts Paid Family and Medical Leave (PFML) employee contribution' and treats 'treated it as employer-only' as an error.", + "where": "us_case_notes.csv scenario_081 payroll_tax" + }, + { + "item": "Reword case note 082, which says 'The requested payroll_tax output includes mandatory employee state payroll taxes. In New York those are Paid Family Leave (PFL) and disability benefits (DBL) contributions.'", + "where": "us_case_notes.csv scenario_082 payroll_tax" + }, + { + "item": "Reword the row annotations that state the program is mandatory. A regex finds at least 47 of the 149 (032: 8, 043: 14, 081: 9, 082: 16), for example grok-4.7 on 032: 'Minnesota Paid Leave premiums are mandatory from 2026'. Others call the claim that a state has no mandatory employee tax 'wrong', for example claude-opus-4.8 on 082.", + "where": "annotations/.../us_audit_row_annotations.csv (149 rows on the four outputs)" + }, + { + "item": "Optionally fix scenario_032's frozen narrative. It calls the premium a 'contribution requirement' and names mn_paid_leave_contribution, which does not exist in policyengine-us 2.15.17 (the variable is mn_employee_paid_leave_contribution). The r33 exclusion set the precedent of rewriting a misstated frozen narrative.", + "where": "annotations/.../us_case_reference_explanations.csv; tests/test_reference_audit.py:382-401" + }, + { + "item": "Extend the record-date rule in the reference-audit test so records decided after the 2026-09-22 wave carry their own date. The filter excludes only 2026-09-05 and 2026-09-29, so the new records fail it (observed: '2026-10-05' == '2026-09-22').", + "where": "tests/test_reference_audit.py:288-300" + }, + { + "item": "Let the sweep-table test accept exclusions decided after the upgrade. sweep_moves.csv marks the four outputs excluded=False, and the test requires that column to match the record (observed failure on scenario_032).", + "where": "tests/test_reference_upgrade.py:94-108; reference_audit/2026-09-28/sweep_moves.csv rows 558, 714, 1304, 1326" + }, + { + "item": "Change len(scored) == 1928 and its comment to 1,924, or 1,921 with #191.", + "where": "tests/test_reference_upgrade.py:358-360" + }, + { + "item": "Retire or re-scope the committed-snapshot test in the gpt61sol suite. It pins release 20260929's reference_exclusions.json sha bf4e6a24..., BASE_EXCLUSIONS 56, the 28/28 reason split and BASE_SCORED (observed: 'does not match its pin'). Read the base files from git at BASE_COMMIT instead.", + "where": "tests/test_finish_gpt61sol.py:400-418; scripts/finish_gpt61sol.py:133-150" + }, + { + "item": "Build a release driver. freeze_gpt61sol.py refuses any change to the exclusions ('this release has no reference revision') and any incumbent drift. freeze_snapshot.py's SOURCE_RUN, PUBLISHED_DASHBOARD_SOURCE and PUBLISHED_DASHBOARD_ARTIFACT still name the 42-model 20260922c bundle, so do not run its main() as-is. Precedent: 20260922b (ba886b4c) edited those constants directly; 20260930 overrode them in-process.", + "where": "scripts/freeze_gpt61sol.py:205-232, 600-617; scripts/finish_gpt61sol.py:564-571; scripts/freeze_snapshot.py:101-129, 1760-1785" + }, + { + "item": "Refreeze the payload, the analysis CSVs and the manifest. This sets scored=false on 184 rows; n becomes 1,924 and every modelStats row moves by -0.07 to +0.62 points. The freeze re-pins the hashes the snapshot tests check (observed failing before refreeze: manifest source-run and annotation hashes, bootstrap estimates, impact summaries, and test_manuscript_views.py:71).", + "where": "paper/snapshot/20260501/manifest.json and runs/.../data.json.gz, analysis/*; tests/test_snapshot_artifacts.py:32-151, 1028, 1048; tests/test_manuscript_views.py:71" + }, + { + "item": "Update the release pointers: app/src/data.artifact.json, PUBLISHED_DASHBOARD_ARTIFACT, and app/src/paperSnapshot.json after re-rendering. Rewrite data.versions.json's description: 'the 56 excluded outputs ... 52 from policyengine-us 1.755.4, 4 from 2.15.17' becomes 60 (52, 8), or 63 (52, 11) with #191. Keep snapshotLabel 'Snapshot 2026-09-30', since the models and answers are unchanged (20260922b precedent). Uploading the release asset is a publish step that needs Max.", + "where": "app/src/data.versions.json; app/src/data.artifact.json; tests/test_disclosures.py:690 (observed failing)" + }, + { + "item": "Change the expand page's leader pin from 95.003 to 95.093 and the copy from '95.0%' to '95.1%'. With #191 the values are 95.749 and '95.7%'.", + "where": "app/tests/expandPage.test.ts:73-83" + }, + { + "item": "Rescore the sensitivity evidence with scripts/sensitivity_by_variable.py and then scripts/rescore_sensitivity_summaries.py --release . Fable 5.1 auto goes from 91.728 to 92.349 and Fable 5 thinking from 91.521 to 92.129 (observed). Update n 1928, 'fifty-six' and the doc's table, and copy the new autoExact values into servingSensitivity.ts.", + "where": "sensitivity/data/claude-fable-5-1-thinking.json:23-24,38; sensitivity/data/claude-thinking-2026-08.json; sensitivity/claude-thinking-2026-08.md:50,59,89,150; app/src/lib/servingSensitivity.ts:38-61; tests/test_sensitivity_evidence.py:127,217,239" + }, + { + "item": "Revise paper/index.qmd in three places. Add the employer-deduction reading to 'The unlisted inputs behind the unlisted-input exclusions...' (line 1087). Extend 'moved on 1.755.4 or, for the state and local tax refund reading, on 2.15.17' to cover the paid-leave sweep on 2.15.17. Add a reference-review table row for the October payroll audit beside 'US payroll and overtime' (line 888). Then re-render and run freeze_snapshot.py --rendered-only. The abstract sentence is generated: 'the 28 ... and the 32 whose references depend on an input' (35 with #191).", + "where": "paper/index.qmd:886-893, 970, 1087; tests/test_disclosures.py:617-634 (observed failing on the stale render)" + }, + { + "item": "Update the benchmark card: line 73 (56 → 60; '4 with 2.15.17' → 8), 105-109 (audit counts), 122 (69 → 73), 144 ('Fifty-six outputs in 39 households' → 'Sixty outputs in 41 households'), 156 ('Twenty-eight' → 'Thirty-two', and add the paid-leave reading to the list), 173 (1,928 → 1,924) and 175 ('55 answers that never parsed' → 59).", + "where": "docs/benchmark_card.md:73-74,105-109,122,144,156-160,173,175; tests/test_disclosures.py:767 (observed failing)" + }, + { + "item": "Update the paper guide's audit counts and its exclusion and scored counts.", + "where": "docs/paper.md:70-71, 77-78" + }, + { + "item": "Notes: no note carries dashboard-data-20260930, so SUPERSEDED_RELEASES and CURRENT_RELEASE_SNAPSHOT change only if a release note is added. Decide whether the 2026-09-29 Sonnet note, which says Sonnet is #5 and GPT-6 Luna #4, gets a later-release sentence as cb312fd7 added to superseded notes. Under exclusion Sonnet passes Luna, 92.52 to 92.34.", + "where": "tests/test_notes.py:78-96; app/src/notes/2026-09-29-claude-sonnet-5-5-debuts-fifth.json" + }, + { + "item": "If #191 lands in the same release, reword its scenario_081 record. It says the household's 'Massachusetts paid family and medical leave contributions ($805.01)' are left out of SALT as deductible taxes, but this audit classifies that share as an optional employer pass-through: up to $805.01, and only if the employer deducts it.", + "where": "git show 8af912a0:reference_audit/2026-10-05/proposed_exclusions.json (scenario_081 alternative_reading); 8af912a0 README 'Related engine issues'" + }, + { + "item": "Optionally add a later-revision line to the 2026-09-28 README, which states '1,928 scored outputs and 56 exclusions'. The 09-22 README set the precedent for recording later revisions.", + "where": "reference_audit/2026-09-28/README.md:13" + } + ], + "regenerate": [ + { + "item": "Before installing anything, record Max's ruling. Set the revision date to the ruling date and fill the README placeholders DECISION_ID, INDEPENDENT_REVIEW and RELEASE_CHECKLIST.", + "where": "reference_audit/2026-10-05-payroll/proposed_regenerations.json:3; README.md" + }, + { + "item": "Write the four values (2218.5, 312.74322509765625, 13387.65234375, 7664.91748046875) into BOTH reference CSVs. The top-level copy must equal the run copy.", + "where": "paper/snapshot/20260501/runs/.../reference_outputs.csv; paper/snapshot/20260501/us_reference_outputs.csv; tests/test_snapshot_artifacts.py:629-642" + }, + { + "item": "Fix propose_changes.py's docstring. It says a release 'adds one output_scope revision to the reference sidecar, as the Maryland adapter was recorded', which is false. The sidecar has no output_scope kind; the Maryland adapter is a fix_modules entry inside the single engine_upgrade revision.", + "where": "reference_audit/2026-10-05-payroll/scripts/propose_changes.py:7-11; runs/.../reference_outputs.csv.meta.json (engine_upgrade fix_modules)" + }, + { + "item": "Choose how to record the revision. Option (a): a trailing revision of a new kind. That needs read_reference_engine_setup and the upgrade-test helper relaxed, since both require the single engine_upgrade revision to be the last one. On a scratch clone that helper broke 8 tests by itself. Option (b): add the adapter to the engine_upgrade revision. That rewrites the 2026-09-29 record, whose modules must live in reference_audit/2026-09-28/fixes.", + "where": "scripts/freeze_snapshot.py:1414-1442; tests/test_reference_upgrade.py:46-51, 88-91" + }, + { + "item": "Update reference_csv_sha256 in the sidecar. Give the revision a convention or root_cause key; regenerated_references_by_source keys on those and would key on None otherwise. Add a paper_results accessor if the paper reports this regeneration, because regenerated_reference_keys counts only convention and upstream_fix revisions.", + "where": "runs/.../reference_outputs.csv.meta.json; policybench/paper_results.py:1290-1302, 1530-1537" + }, + { + "item": "Decide whether to advance regenerated_at_utc. The freeze dates the references by it, the test pins it to 2026-09-29, and the paper prints r.reference_rebuilt_date.", + "where": "scripts/freeze_snapshot.py:1360-1385; tests/test_reference_upgrade.py:403-405; tests/test_snapshot_artifacts.py (refresh-date test); tests/test_reference_provenance.py:191 (observed failing)" + }, + { + "item": "Add payroll_mandatory_scope.py to OUTPUT_SCOPE_ADAPTERS and to the module check, and rewrite the reproducibility note 'the Maryland output-scope adapter'. The test pins the module set {latest_conventions.py, latest_md_local_output_scope.py} and both sentences verbatim.", + "where": "scripts/freeze_snapshot.py:1408, 1434, 1541-1543; tests/test_snapshot_artifacts.py:897-935; docs/paper.md:59-62" + }, + { + "item": "Re-run the publication check on the newest policyengine-us at release with latest_final plus payroll_mandatory_scope, and commit the sweep and log. latest_final_2170.csv has no payroll adapter, so the test that 'recomputes every scored reference' fails for the four outputs (observed). The engine_upgrade rule says that at publication PolicyBench checks the newest release gives the same values.", + "where": "reference_audit/2026-09-28/verification/latest_final_2170.csv; tests/test_reference_upgrade.py:347-379" + }, + { + "item": "Scope two upgrade tests to outputs outside the new revision. The sweep-table test compares sweep_moves.csv reference/final values to the CSV (observed failure). The changed-reference test requires the outputs that differ from the 20260922c board to equal the engine_upgrade revision's 'changed' list.", + "where": "tests/test_reference_upgrade.py:94-108, 111-130" + }, + { + "item": "Add a test for the new revision, modeled on the 09-22 one. Check that the module is committed with sha 74200522..., that sweep_payroll_scope.csv moves exactly the four outputs, that the CSV holds the regenerated values, and that none of the four is excluded.", + "where": "tests/test_reference_audit.py:141-181 (pattern); reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope.csv" + }, + { + "item": "Rewrite the four reference narratives (reference_value and explanation) as federal-only derivations. Add them to a REGENERATED_NARRATIVES-style table with required figures and test it.", + "where": "annotations/.../us_case_reference_explanations.csv; tests/test_reference_audit.py:404-438 (pattern)" + }, + { + "item": "Re-derive the audit rows. Delete the 82 annotations that become exact (032: 33, 043: 26, 081: 9, 082: 14). Annotate the 35 new misses: 032 deepseek-v4.1-flash, gpt-5.5, gpt-5.6-sol, gpt-6-astra, gpt-6.1-sol, kimi-k3; 043 sixteen models; 081 nine; 082 gpt-6-luna, gpt-6-sol, gpt-6.1-sol, kimi-k3. Set wrong_model_count from 40/30/37/42 to 13/20/37/32. Both auditUniverse.ts and paper_results.audit_selection_rule raise when the annotated rows differ from the legacy-threshold rows.", + "where": "annotations/.../us_audit_row_annotations.csv, us_case_notes.csv; app/src/lib/auditUniverse.ts; policybench/paper_results.py:1624-1630; tests/test_snapshot_artifacts.py:717, 803 (observed failing)" + }, + { + "item": "Re-judge the four cases on the new reference and narrative with claude-opus-5-5, through scripts/run_audit_claude.sh as 20260922b did. Rewrite all four case notes from the new verdicts. Refresh judge_provenance: the dates change, and by_judge stays 298/116/260 only if Opus 5.5 judges. Add an adjudication only if a verdict comes back non-final or flagged.", + "where": "results/local//cases/us__scenario_0{32,43,81,82}__payroll_tax; tests/test_paper_results.py:303-311; tests/test_judge_provenance.py" + }, + { + "item": "Retire or re-scope the gpt61sol committed-reference pin test. It pins release 20260929's CSV sha e8bbba8f... and sidecar sha 816fef53... (observed failing).", + "where": "tests/test_finish_gpt61sol.py:400-418; scripts/finish_gpt61sol.py:133-150" + }, + { + "item": "Build a release driver that accepts the reference revision and incumbent drift. Every modelStats row moves by -0.69 to +0.69 points, and n stays 1,928. freeze_gpt61sol.py and finish_gpt61sol.py refuse both. finish_adds0928.py accepted only an engine_upgrade revision.", + "where": "scripts/finish_gpt61sol.py:201-209; scripts/freeze_gpt61sol.py:226-245; scripts/finish_adds0928.py" + }, + { + "item": "Refreeze the payload, the analysis CSVs and the manifest, and update the pointers, data.versions.json and paperSnapshot.json. Uploading the release asset is Max's step.", + "where": "paper/snapshot/20260501/manifest.json; app/src/data.artifact.json; app/src/data.versions.json; tests/test_snapshot_artifacts.py:32-151, 1028, 1048 (observed failing)" + }, + { + "item": "Change the expand page's leader pin from 95.003 to 94.629 and the copy from '95.0%' to '94.6%'. With #191 the value is 95.192 and the copy '95.2%'.", + "where": "app/tests/expandPage.test.ts:73-83" + }, + { + "item": "Rescore the sensitivity evidence. Fable 5.1 auto goes from 91.728 to 92.416 and Fable 5 thinking from 91.521 to 92.209 (observed); n is unchanged. Copy the new autoExact values into servingSensitivity.ts.", + "where": "sensitivity/data/*.json; sensitivity/claude-thinking-2026-08.md; app/src/lib/servingSensitivity.ts:38-61; tests/test_sensitivity_evidence.py:127,217,239" + }, + { + "item": "Revise the paper. Add the payroll adapter to the reference-construction paragraph ('an adapter restores that scope'). Fix the publication-check sentence 'gives the same value ... for all 1,984 outputs under the same conventions and adapter'. Add a reference-review table row. Then re-render and re-pin.", + "where": "paper/index.qmd:886-893, 1083; tests/test_disclosures.py:487-511 (observed failing)" + }, + { + "item": "Update the benchmark card's adapter and publication-check paragraph and its audit counts. The board grid on the leaderboard is 7,813 / 7,809 / 4 / 2,125.", + "where": "docs/benchmark_card.md:84-90, 105-109; docs/paper.md:59-62, 70-71" + }, + { + "item": "Leave the exclusion record and the adjudication count (69) unchanged, but note one inconsistency if #191 also lands. #191's 081 record treats the $805.01 MA PFML share as paid, while the regenerated payroll reference says it is not employee tax. Reword that record.", + "where": "git show 8af912a0:reference_audit/2026-10-05/proposed_exclusions.json" + } + ], + "pinned_counts": [ + { + "name": "Excluded outputs", + "now": "56", + "exclude": "60", + "exclude_with_salt": "63", + "regenerate": "56", + "where": "tests/test_paper_results.py:327,328('56 outputs'),414; tests/test_reference_exclusions.py:156; tests/test_adjudications.py:231,298; tests/test_finish_gpt61sol.py:411; docs/benchmark_card.md:73,106,144; docs/paper.md:77; app/src/data.versions.json; sensitivity md:50 and data JSON 'fifty-six'" + }, + { + "name": "Unlisted-input exclusions", + "now": "28", + "exclude": "32", + "exclude_with_salt": "35", + "regenerate": "28", + "where": "tests/test_paper_results.py:330; tests/test_reference_exclusions.py:158-160; tests/test_finish_gpt61sol.py:414; docs/benchmark_card.md:156; paper abstract via r (tests/test_disclosures.py:617-634)" + }, + { + "name": "Households with an exclusion", + "now": "39", + "exclude": "41 (+032, 043)", + "exclude_with_salt": "42 (+114)", + "regenerate": "39", + "where": "tests/test_paper_results.py:329; docs/benchmark_card.md:144; unlisted-input households 22 -> 26/28 (paper_results.excluded_output_households_by_reason, unpinned)" + }, + { + "name": "Scored outputs per model", + "now": "1,928", + "exclude": "1,924", + "exclude_with_salt": "1,921", + "regenerate": "1,928", + "where": "tests/test_paper_results.py:358,382; tests/test_reference_upgrade.py:360; tests/test_finish_gpt61sol.py:418; docs/benchmark_card.md:173; docs/paper.md:77; sensitivity md:50,59,89,150; sensitivity/data/*.json n; comment test_paper_results.py:80-81 (88,688 -> 88,504 / 88,366)" + }, + { + "name": "Exclusions decided on 2.15.17 (by engine)", + "now": "{1.755.4: 52, 2.15.17: 4}", + "exclude": "{52, 8}", + "exclude_with_salt": "{52, 11}", + "regenerate": "{52, 4}", + "where": "tests/test_paper_results.py:411,413; docs/benchmark_card.md:73-74; app/src/data.versions.json; tests/test_disclosures.py:690,767 derive the sentence" + }, + { + "name": "Developer adjudications", + "now": "69", + "exclude": "73", + "exclude_with_salt": "76", + "regenerate": "69 (unless a re-judge needs one)", + "where": "tests/test_adjudications.py:221,276; docs/benchmark_card.md:122; manifest developer_adjudications.cases" + }, + { + "name": "Adjudications to prompt_ambiguity / judge llm_error", + "now": "28 / 59", + "exclude": "32 / 63", + "exclude_with_salt": "35 / 66", + "regenerate": "28 / 59", + "where": "tests/test_adjudications.py:222-224, 285-290" + }, + { + "name": "Audited rows (legacy-threshold universe)", + "now": "7,860", + "exclude": "7,711", + "exclude_with_salt": "7,573", + "regenerate": "7,813", + "where": "tests/test_paper_results.py:87-88; tests/test_snapshot_artifacts.py:651; tests/test_disclosures.py:268; app/tests/auditUniverse.test.ts:22-23; docs/benchmark_card.md:105; docs/paper.md:70" + }, + { + "name": "Exact-match misses (= annotated misses)", + "now": "7,856", + "exclude": "7,707", + "exclude_with_salt": "7,569", + "regenerate": "7,809", + "where": "tests/test_paper_results.py:90-93; tests/test_snapshot_artifacts.py:652-653; tests/test_disclosures.py:270-271; app/tests/auditUniverse.test.ts:24-25; docs/benchmark_card.md:108; docs/paper.md:71" + }, + { + "name": "Annotated exact hits", + "now": "4", + "exclude": "4", + "exclude_with_salt": "4", + "regenerate": "4", + "where": "tests/test_paper_results.py:94-95 (unchanged)" + }, + { + "name": "Unannotated scored rows below full bounded score", + "now": "2,107", + "exclude": "2,072", + "exclude_with_salt": "2,072", + "regenerate": "2,125", + "where": "tests/test_paper_results.py:96-97; tests/test_snapshot_artifacts.py:656; tests/test_disclosures.py:273; app/tests/auditUniverse.test.ts:27; docs/benchmark_card.md:109; docs/paper.md:71" + }, + { + "name": "Scored rows below full bounded score", + "now": "9,967", + "exclude": "9,783", + "exclude_with_salt": "9,645", + "regenerate": "9,938", + "where": "tests/test_snapshot_artifacts.py:655" + }, + { + "name": "Scored annotation sources llm_error / parse_contract_failure", + "now": "7,208 / 652", + "exclude": "7,063 / 648", + "exclude_with_salt": "6,928 / 645", + "regenerate": "7,161 / 652", + "where": "tests/test_snapshot_artifacts.py:658-660" + }, + { + "name": "Parse-contract failures (scored)", + "now": "652 (kimi-k2.6 390, glm-5.2 133, glm-5.3 71, kimi-k3 58)", + "exclude": "648 (kimi-k2.6 387, glm-5.2 132)", + "exclude_with_salt": "645 (kimi-k2.6 384, glm-5.2 132)", + "regenerate": "652", + "where": "tests/test_paper_results.py:69-79 (pct stays 0.7)" + }, + { + "name": "Contract violations", + "now": "712", + "exclude": "708", + "exclude_with_salt": "705", + "regenerate": "712", + "where": "tests/test_paper_results.py:110" + }, + { + "name": "Annotated rows on excluded outputs", + "now": "2,111", + "exclude": "2,260", + "exclude_with_salt": "2,398", + "regenerate": "2,111", + "where": "tests/test_paper_results.py:361; docs/benchmark_card.md:106" + }, + { + "name": "prompt_ambiguity_row_count", + "now": "820", + "exclude": "965", + "exclude_with_salt": "1,100", + "regenerate": "820", + "where": "tests/test_paper_results.py:363" + }, + { + "name": "Never-parsed rows on excluded outputs", + "now": "55", + "exclude": "59", + "exclude_with_salt": "62", + "regenerate": "55", + "where": "docs/benchmark_card.md:175" + }, + { + "name": "Leader (GPT-6 Sol) weighted exact", + "now": "95.003", + "exclude": "95.093", + "exclude_with_salt": "95.749", + "regenerate": "94.629", + "where": "app/tests/expandPage.test.ts:74,81-83; tests/test_snapshot_artifacts.py:1028 (bootstrap, via refreeze)" + }, + { + "name": "Always-zero exact / leader margin (paper @tbl-baselines)", + "now": "70.0 / 25.0", + "exclude": "70.5 / 24.6", + "exclude_with_salt": "71.0 / 24.7", + "regenerate": "70.0 / 24.6", + "where": "policybench/paper_results.py:870-936 (rendered, no test pin found)" + }, + { + "name": "Payroll exact across models (n)", + "now": "78.5% (4,554)", + "exclude": "81.0% (4,370)", + "exclude_with_salt": "81.0% (4,370)", + "regenerate": "79.5% (4,554)", + "where": "payload programStats; analysis/summary_by_variable.csv" + }, + { + "name": "Sensitivity auto exact: Fable 5.1 / Fable 5 thinking", + "now": "91.728 / 91.521", + "exclude": "92.349 / 92.129", + "exclude_with_salt": "not computed", + "regenerate": "92.416 / 92.209", + "where": "app/src/lib/servingSensitivity.ts:38,61; sensitivity/data/*.json; tests/test_sensitivity_evidence.py:127,217" + }, + { + "name": "wrong_model_count on 032/043/081/082", + "now": "40/30/37/42", + "exclude": "40/30/37/42", + "exclude_with_salt": "40/30/37/42", + "regenerate": "13/20/37/32", + "where": "annotations/.../us_case_notes.csv; tests/test_snapshot_artifacts.py:803" + } + ], + "notes": [ + "Verdict: both options are installable, but neither is a records-only change. Either one needs a new release driver, a payload refreeze and changes to tests that pin release 20260929 or 20260930 bytes. Regenerate also needs a new publication-check sweep, four judge re-runs and a decision on how the sidecar records a revision after engine_upgrade.", + "How the numbers were produced: counts come from subclassing policybench.paper_results.PaperResults with the extra exclusion records and relabelled rows. The regenerate row sets come from scoring a scratch copy with the audit's own stage() and analyze(). The README's figures reproduce: always-zero 70.53 (exclude), 69.98 (regenerate), 71.03 (exclude with SALT); GPT-6 Sol 95.09 / 94.63 / 95.75.", + "Test-run evidence: the 'observed' tags come from scratch shared clones of 8714770 with the records installed mechanically: exclusions appended, adjudications applied, regenerate rows swapped. The payload was not refrozen and the prose was not edited. On the targeted files, the exclude clone gave 27 failed / 245 passed and the regenerate clone 27 failed / 245 passed. On the unmodified clone, the same record and snapshot test files passed in an earlier partial run. The exclude-with-SALT run was stopped at 52%, with the same failures as exclude up to that point. Payload-derived pins (test_disclosures.py:264-274 and 487-511, app auditUniverse) pass on the exclude clone only because nothing was refrozen; they will need the values above after the refreeze.", + "Regenerate together with #191: excluded 59, households 40, scored 1,925, adjudications 72, audited rows 7,675, misses 7,671, unannotated below full 2,125, below full bounded 9,800, llm_error/parse 7,026/649, GPT-6 Sol 95.192, always-zero 70.4.", + "#191's own checklist (8af912a0 README) misses several things a combined release hits: parse failures fall 652 -> 649 (kimi-k2.6 390 -> 387), contract violations 712 -> 709, and the scored-source counts in tests/test_snapshot_artifacts.py:658-660 change. It also misses tests/test_adjudications.py:639 and the wave logic at 709-784, the exclusion-list tail order at tests/test_reference_upgrade.py:221-223, and tests/test_finish_gpt61sol.py:400-418. Its judged_on_utc for 022, 081 and 114 must be 2026-09-30 from the published tree, not 2026-09-23 from unified_audit.", + "proposed_regenerations.json says to apply the adapter together with latest_final on policyengine-us 2.15.17. The engine rule in the sidecar says references come from the newest release when the sweep begins, and that the newest release is checked at publication. A regenerate release in October has to either re-sweep on the newest release or state why 2.15.17 stands.", + "There is no release_facts file in the repo at this commit.", + "Test environment: lstat on the system TMPDIR (/private/var/folders/.../T) hung on this machine and stalled pytest in uninterruptible state, so the runs used TMPDIR and --basetemp inside the scratch directory. No checkout file was modified. Scratch is under the review/release-checklist/ scratchpad: counts.json, regen_rows.json, *_targeted2.txt and the clones in x/." + ] + }, + "claim_audit": { + "claims_checked": 131, + "problems": [ + { + "readme_text": "no benchmark household lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington.", + "problem": "Two benchmark households live in Washington: scenario_002 and scenario_111, both WA with $0 employment income (scenarios.csv; payroll_decomposition.csv state column). program_classification.json repeats the error (\"No benchmark household lives in these states\").", + "correct": "No household with wages lives in Delaware, Maine, Oregon, Rhode Island, Vermont or Washington. Two Washington households (scenario_002, scenario_111) have no wages, and none lives in DE, ME, OR, RI or VT.", + "severity": "error" + }, + { + "readme_text": "It would take credit from 35 answers that include the program at exactly the engine's amount. (also in the Recommendation bullet)", + "problem": "The 35 answers are those within the $1 exact tolerance of the frozen reference. Only 15 equal it to the cent: MN 2,346.10 x6, MA 14,192.66 x6, NY 8,108.03 x3. The other 20 do not: CO 331.13 x9 (implied state part $18.39 against the engine's $17.99), CO 330.72 x7, MA 14,193 x3, NY 8,107.98 x1. No CO answer equals 330.73.", + "correct": "It would take credit from 35 answers that include the program and score as exact (within $1) against the frozen reference. 15 of them match it to the cent.", + "severity": "error" + }, + { + "readme_text": "Claude Fable 5.1 and Ox Alpha leave out every state program, mandatory ones included.", + "problem": "Ox Alpha includes California SDI on scenario_022 (answer 4,658; explanations.json quote \"plus mandatory California employee SDI at 1.2% = $632\") and on scenario_023 (answer 1,544; \"mandatory CA employee SDI 1.2% x $17,443 = $209\"). Both are categorized included_state_component. It also answers 31,084 on scenario_005, which is not the federal-only amount. Only Claude Fable 5.1 answers federal-only on all 11 scored outputs.", + "correct": "Claude Fable 5.1 leaves out every state program, mandatory ones included. Ox Alpha leaves out the four optional programs and PA, CT and CA SDI on scenario_099, but includes CA SDI at the 2025 rate of 1.2% on scenario_022 and scenario_023.", + "severity": "error" + }, + { + "readme_text": "policyengine-us has no input for the employer's deduction choice, employer size (except Massachusetts' employer side) or a private plan.", + "problem": "policyengine-us 2.15.17 has an employer-size input, employer_headcount (variables/input/employer_payroll.py, Person, default 100). It is read by co_employer_famli_contribution, ma_employer_paid_leave_contribution, me_employer_paid_leave_contribution, or_employer_paid_leave_contribution, wa_employer_paid_leave_contribution, employer_total_additional_state_payroll_tax and the NY MCTMT payroll-expense proxy. It also feeds de_paid_leave_contribution_rate, which sets Delaware's employee contribution.", + "correct": "policyengine-us has an employer_headcount input. It drives the employer side in CO, MA, ME, OR and WA, and Delaware's contribution rate. It reaches none of the four benchmark employee shares (MN, CO, MA, NY). There is no input for the employer's deduction choice or for a private plan.", + "severity": "error" + }, + { + "readme_text": "Its records alone reproduce #191's figures here (GPT-6 Sol 95.62%; Claude Sonnet 5.5 passes GPT-6 Luna).", + "problem": "The reproduction holds: this audit's salt CSV equals #191's leaderboard_impact_models.csv to 0.0 on all 29 shared columns. The printed figure is mis-rounded, though. GPT-6 Sol's exact_proposed is 95.6146 in both files, which rounds to 95.61. The 95.62 comes from #191's README line 95, which carries the same rounding error.", + "correct": "GPT-6 Sol 95.61% (95.6146)", + "severity": "error" + }, + { + "readme_text": "22 of 46 models included the Massachusetts and New York contributions.", + "problem": "This reads as one set of 22 models that included both. In explanations.json, 22 models include MA and a different 22 include NY. 17 include both and 27 include at least one. MA only: gemini-3.5-flash-lite, gpt-5.6-sol, gpt-5.6-terra, grok-4.5, grok-4.7. NY only: claude-haiku-4.5, claude-opus-4.7, claude-sonnet-5, gemini-3.6-flash, gpt-6-luna.", + "correct": "22 of 46 models included the Massachusetts contribution and 22 the New York one (17 included both).", + "severity": "imprecise" + }, + { + "readme_text": "Under each state's law the premium is the employer's, and the employee pays only what the employer chooses to deduct, from zero up to a cap (16 of 16 independent reviews agree).", + "problem": "Only 8 of the 16 reviews concern MN, CO, MA and NY. The other 8 reviewed CA, CT, PA and NJ and agreed those are mandatory. law_research.json has two verdicts for each of the 8 programs, all refuted=false.", + "correct": "(8 of 8 independent reviews of these four programs agree; none of the 16 reviews refuted any classification)", + "severity": "imprecise" + }, + { + "readme_text": "Exclusion is the symmetric remedy and moves the board least.", + "problem": "This holds for rank swaps only: 5 headline pairs against 10, and 9 within-1% rank changes against 22. On levels exclusion moves the board more. Mean headline delta is +0.437 for exclusion against +0.169 for regeneration, and mean absolute delta is 0.439 against 0.325. Exclusion moves 4 bounded-score ranks against 2, and moves the always-zero baseline 69.98 to 70.53 while regeneration leaves it unchanged. GPT-6 Sol's lead over always-zero shrinks more under exclusion (24.57) than under regeneration (24.65).", + "correct": "Exclusion swaps fewer headline ranks (5 adjacent pairs against 10) and keeps the within-1% leader, though it shifts rates and the always-zero baseline more than regeneration does.", + "severity": "imprecise" + }, + { + "readme_text": "Where the law requires the employee contribution, the explanations look the same (mandatory programs, all scored):", + "problem": "Only the PA and CT rows resemble the optional programs. On the four CA SDI outputs, 29 to 31 of 46 models include the program and only 10 to 13 say there is no tax or never mention it. On MN the figures are 7 include and 35 no-tax-or-silent; on CO, 18 and 25.", + "correct": "On Pennsylvania's and Connecticut's mandatory contributions the explanations look the same. California SDI is included far more often (29 to 31 of 46).", + "severity": "imprecise" + }, + { + "readme_text": "Nothing else in the engine reads these contributions except through `employee_payroll_tax`, which also feeds `spm_unit_paycheck_withholdings` and `household_tax_before_refundable_credits`.", + "problem": "The list of what employee_payroll_tax feeds is incomplete. It is also an Alabama itemized-deduction source (parameters/gov/states/al/tax/income/deductions/itemized/sources.yaml), and it is read by fica_marginal_tax_rate and taxsim_tfica. One contribution is also read directly: ca_employee_state_disability_insurance_contribution appears in the Santa Clara County General Assistance earned-income deductions (parameters/gov/local/ca/scc/general_assistance/countable_income/earned_deductions/person_sources.yaml). The five optional contributions are read only by their state aggregates. No benchmark output moves through these paths: the sweep confirms this for the optional five.", + "correct": "The four optional contributions are read only through their state aggregates and employee_payroll_tax. employee_payroll_tax also feeds spm_unit_paycheck_withholdings, household_tax_before_refundable_credits, Alabama's itemized deductions, fica_marginal_tax_rate and taxsim_tfica. California SDI also feeds Santa Clara County General Assistance.", + "severity": "imprecise" + }, + { + "readme_text": "Every 2026 rate, share and cap above was published before the 2026-07-03 reference freeze, and each engine amount equals the law's largest employee share to the cent.", + "problem": "For CT, the law amount is 0.005 x 165,597 = $827.985, which law_research.json shows as $827.99. The engine returns 827.98498 (float32), which shows as $827.98. For MA, the README's own bullet says Acts 2026 c. 101 s. 45 could put the 2026 maximum at $1,351.02. The claim needs both qualifications.", + "correct": "each engine amount equals the law's largest employee share within a cent (CT: engine $827.98, law $827.985), with the MA maximum subject to the Acts 2026 c. 101 question below", + "severity": "imprecise" + }, + { + "readme_text": "The two reviewers read its scope differently, and its s. 45 applies it to taxable years beginning on or after 2026-01-01 ... If it reached 2026, the most a large employer could deduct for scenario_081 would be $1,351.02 (0.772%) rather than $805.01.", + "problem": "$1,351.02 holds only under the administration reviewer's reading, in which ss. 25-26 amend s. 6(c), (d) and (e). The statute reviewer reads both sections as amending only \"line 33\", which is s. 6(d), the remittance rule for small employers. On that reading the large-employer deduction cap in s. 6(c) stays at 0.46% ($805.01) even for 2026.", + "correct": "Under the administration reviewer's reading (ss. 25-26 amend s. 6(c)), the 2026 large-employer maximum would be $1,351.02 (0.772%). Under the statute reviewer's reading (only s. 6(d)), it stays $805.01.", + "severity": "imprecise" + }, + { + "readme_text": "The four proposed references have had these values since the first run (2026-06-15).", + "problem": "2026-06-15 is the date of the commit that added the run (e4563713). The run's reference sidecar at that commit records generated_at_utc 2026-06-12T05:15:20Z, which matches the run name us_full_run_20260612. The values have been unchanged since that commit.", + "correct": "since the first run (references generated 2026-06-12, committed 2026-06-15 in e4563713)", + "severity": "imprecise" + }, + { + "readme_text": "| NY PFL and DBL | Covered employer provides the coverage and pays its cost beyond what it collects | ... | [N.Y. WCL s. 209](...)(3)-(4), s. 210(1) |", + "problem": "The s. 210(1) text in law_research.json says the employer contributes \"the cost of providing disability benefits in excess of the contributions collected\". It covers DBL only. For PFL, s. 209(3)(b) says \"No employer shall be required to fund any portion of the family leave benefit.\" The NY bullet under points that cut the other way leaves out further sources: the DFS 2026 decision (\"Employers shall collect employee contributions consistent with this Decision\") and the WCB press release (\"fully paid for by employees\"). Both are recorded in law_research.json.", + "correct": "Covered employer provides the coverage and pays the DBL cost beyond what it collects (s. 210(1)); for PFL no employer is required to fund any portion (s. 209(3)(b)), but it may", + "severity": "imprecise" + }, + { + "readme_text": "Four scored references count an employee share of a state paid-leave premium that the law lets the employer deduct but does not require.", + "problem": "scenario_082 also counts the New York Disability Benefits (DBL) contribution of $31.20, which is a disability program and not a paid-leave premium. The adapter drops five contributions, ny_employee_disability_benefits_contribution among them.", + "correct": "an employee share of a state paid-leave or disability premium", + "severity": "imprecise" + }, + { + "readme_text": "| MN Paid Leave | Premiums \"accrue and become payable by each employer\"; employer pays at least 50% |", + "problem": "The 50% floor in subd. 3 applies to large employers only. For small employers (subd. 5a), the total is 0.66% and the employer minimum is 0.22% of wages, one-third of the premium. The maximum employee share is still 0.44% (law_research.json rate_2026).", + "correct": "employer pays at least 50% (small employers: at least 0.22% of a 0.66% premium); employee share at most 0.44% either way", + "severity": "imprecise" + }, + { + "readme_text": "But the same laws speak of the employee share in mandatory terms: ... and Massachusetts tells employers they \"must withhold PFML contributions\".", + "problem": "The Massachusetts quote is not from the law. It comes from the DFML rates-page overview, an agency web page (law_research.json MA verdicts[0].evidence[12]). The README's own MA bullet attributes it correctly.", + "correct": "and Massachusetts's Department of Family and Medical Leave tells employers they \"must withhold PFML contributions\"", + "severity": "imprecise" + }, + { + "readme_text": "scenario_032's and scenario_043's count the optional-deduction reasoning as a model error.", + "problem": "scenario_032's case note does say \"the paid-leave deduction is optional\". scenario_043's says models \"either said Colorado has no employee payroll tax or dropped FAMLI on purpose\" and never mentions optional deduction. explanations.json records 0 of 46 CO answers giving the optional or employer-paid reason.", + "correct": "scenario_032's counts the optional-deduction reasoning as a model error, and scenario_043's counts dropping FAMLI \"on purpose\" as one", + "severity": "imprecise" + }, + { + "readme_text": "Model by model, the readings split by family.", + "problem": "Only partly supported. The Claude family matches the frozen reference on 1 of 40 optional answers (Sonnet 5.5, once). The GPT family matches on 19 of 40, but GPT-5.6 Terra, GPT-5.4 Mini and GPT-5.4 Nano match on none. The four models the README names as federal-only on all four include Ox Alpha and Gemini 3.8 Flash, which are not Claude models.", + "correct": "Claude models almost never answer the frozen reference (1 of 40 answers); the newer GPT models mostly do (GPT-6.1 Sol 4 of 4; GPT-6 Sol, GPT-6 Astra and GPT-5.6 Sol 3 of 4).", + "severity": "imprecise" + }, + { + "readme_text": "`CountryTaxBenefitSystem` applies a reform twice to the same system, so the adapter is idempotent.", + "problem": "The premise is true: policyengine_us/system.py calls apply_reform_set at lines 136 and 153, and a probe reform was applied 2 times. But the sentence states the consequence backwards. Applying the reform twice does not make the adapter idempotent. The adapter was written to be idempotent because the reform is applied twice: its second pass returns early.", + "correct": "`CountryTaxBenefitSystem` applies a reform twice to the same system, so the adapter is written to be idempotent (its second pass leaves the already-flattened list unchanged).", + "severity": "style" + }, + { + "readme_text": "These files hold for policyengine-us 2.15.17, as installed in the reference venv, plus `latest_final`, the system that built the published references.", + "problem": "\"These files hold\" does not parse. The bullets that follow describe engine facts, not files.", + "correct": "These facts hold for policyengine-us 2.15.17, as installed in the reference venv, plus `latest_final`, the system that built the published references.", + "severity": "style" + } + ] + } +} diff --git a/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json index e1ed4865..28d558b4 100644 --- a/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json +++ b/reference_audit/2026-10-05-payroll/verification/sweep_payroll_scope_households.json @@ -1,5 +1,5 @@ { - "adapter_sha256": "74200522b823abf237a00a5cd6c61038cf1a8015b12bd505c536c743c3713529", + "adapter_sha256": "e139fe6f4dded9723721def83b03facc466bff19930532024d6262166846dc54", "households": { "scenario_000": { "employee_state_payroll_tax_final": 0.0, From 049f4f091397e9c3ec2d9b21f3acbeb9c07c9764 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Mon, 5 Oct 2026 12:54:11 -0400 Subject: [PATCH 3/3] Name the payroll decision (cos d972) in the audit README Co-Authored-By: Claude Opus 5.5 --- reference_audit/2026-10-05-payroll/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/reference_audit/2026-10-05-payroll/README.md b/reference_audit/2026-10-05-payroll/README.md index ed0dd568..74a1277c 100644 --- a/reference_audit/2026-10-05-payroll/README.md +++ b/reference_audit/2026-10-05-payroll/README.md @@ -1,6 +1,6 @@ # Optional employer pass-through in the payroll tax references, October 5, 2026 -This directory audits the state part of PolicyBench's US `payroll_tax` references in release `dashboard-data-20260930`. Four scored references count an employee share of a state paid-leave or disability premium that the law lets the employer deduct but does not require. The directory proposes two alternative records for those four outputs, excluding them or regenerating them, and changes no reference. Either one changes published scores, so they wait for Max's ruling (cos decision DECISION_ID). +This directory audits the state part of PolicyBench's US `payroll_tax` references in release `dashboard-data-20260930`. Four scored references count an employee share of a state paid-leave or disability premium that the law lets the employer deduct but does not require. The directory proposes two alternative records for those four outputs, excluding them or regenerating them, and changes no reference. Either one changes published scores, so they wait for Max's ruling (cos decision d972). ## Recommendation