Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ us,scenario_030,snap,46,llm_error,categorical_eligibility,False,,"The SNAP unit
us,scenario_030,ssi,1,parse_contract_failure,missing_output,False,,All wrong responses were missing or unparseable predictions.
us,scenario_030,tanf,2,llm_error,categorical_eligibility,False,,Texas TANF requires a family with a dependent child; pregnancy alone does not turn the unborn child into a qualifying dependent child or create a two-person assistance unit for this annual TANF calculation. The single adult therefore has no qualifying child and receives zero TANF.
us,scenario_031,federal_income_tax_before_refundable_credits,4,llm_error,taxable_income_or_deductions,False,,"The $1,165 alimony expense reduces $14,408 of IRA and pension income to $13,243 of adjusted gross income, while Social Security remains nontaxable. The applicable single-filer standard deduction, including the age-65 addition, exceeds that income, producing zero taxable income and therefore zero federal income tax before refundable credits."
us,scenario_031,head_medicaid_eligible,43,llm_error,taxable_income_or_deductions,False,,"At 67, the head qualifies as aged, so the engine tests Medi-Cal through the optional senior-or-disabled pathway (California's Aged & Disabled FPL program). This is a non-MAGI pathway with a 138% FPL income limit. Countable income is gross income minus health insurance premiums, including the Medicare Part B standard premium (about $202.90/month, or about $2,435/year in 2026). That brings the $23,853 gross down to about $21,400, below 138% FPL (about $21,600). The $4,200 in bank assets easily passes California's asset test. Almost every model compared gross or near-gross income with the limit and never subtracted the Part B premium. Others used the wrong threshold (100% FPL or the SSI rate), applied the $2,000 SSI resource limit, or denied that an aged person has a Medicaid category."
us,scenario_031,head_medicaid_eligible,43,llm_error,taxable_income_or_deductions,False,,"At 67 the head is aged (65 or older), which alone meets the category condition of the engine's optional senior-or-disabled pathway; it needs no disability or SSI receipt. The adult expansion group (ages 19 to 64, no Medicare eligibility) does not apply, but in the engine's Medicaid rules Medicare eligibility bars only that group. The pathway counts income under SSI rules: Social Security ($9,445.47, in full), the $13,608 pension and the $800 IRA distribution give $23,853.47; the $1,165 of alimony paid is not subtracted. In place of SSI's $20 monthly general income exclusion, the engine applies California's $230 monthly disregard ($2,760 a year), leaving $21,093.47, under the $22,024.80 limit (138% of the 2026 poverty guideline of $15,960). The $4,200 in bank assets is far below California's $130,000 asset limit for the pathway. The engine also models a $2,434.80 Medicare Part B premium for the head, but countable income does not subtract it: with the premium at zero, or the head not enrolled in Medicare, countable income is still $21,093.47. Most models (29 of 43) miscounted income: most of those compared about $23,853 with the limit, a few applied only SSI's $20 exclusion or an exclusion of unstated size, and several asserted without figures that income was too high. The rest used the wrong threshold (100% of the poverty guideline or an SSI income limit), applied SSI's $2,000 resource limit or otherwise judged the $4,200 too much, or denied the head a category, by treating Medicare eligibility or the lack of SSI receipt or a disability as disqualifying, or by testing only the adult expansion group."
us,scenario_031,snap,6,llm_error,categorical_eligibility,False,,"The head is 67, so this is an elderly household. PolicyEngine does not let California's 200% FPL broad-based categorical eligibility skip the net income test for this household. Monthly gross income is $1,987.75 (Social Security, pension and IRA). The only deduction is the $209 standard deduction. There are no shelter costs, and the $50/yr OTC medical expense is below the $35/mo threshold, so net income is about $1,779/mo. That is well above the 1-person net limit of about $1,305/mo (100% FPL), so the household is not SNAP-eligible. The one/two-person minimum benefit only applies to eligible households, so SNAP is $0. Five models treated BBCE as automatic eligibility and paid the minimum benefit. The sixth undercounted income."
us,scenario_031,state_income_tax_before_refundable_credits,1,llm_error,credit_phaseout,False,,"California first computes $75.37 of tax on $7,537 of taxable income, then applies $312.93 of nonrefundable credits, including the aged-or-blind exemption credit. Those credits fully eliminate the liability, so state income tax after nonrefundable credits and before refundable credits is $0."
us,scenario_032,child1_chip_eligible,13,llm_error,health_coverage,False,,"CHIP is residual by statute: 42 U.S.C. 1397jj(b)(1)(A) and 42 CFR 457.310 exclude from ""targeted low-income child"" any child who is eligible for Medicaid, so the CHIP question is settled by first running the Medicaid test. Minnesota's Medical Assistance covers children ages 6-18 — PolicyEngine's OLDER_CHILD category — far above this household's $42,664 for three people (~160% FPL), so the child is Medicaid-eligible and thereby barred from CHIP. Every wrong model instead applied a standalone CHIP income ceiling, most of them quoting Minnesota's 275%/283% FPL child limits, which are the Medical Assistance and MinnesotaCare bands rather than CHIP bands. They read ""income below the child-coverage limit"" as qualifying when it is precisely what disqualifies: several stated the Medicaid fact outright and then used it backwards."
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1809,7 +1809,7 @@ us,scenario_099,child2_medicaid_eligible,0.0,1,"PolicyEngine determined that chi
us,scenario_020,head_medicaid_eligible,0.0,7,"PolicyEngine determined that the head of household is not eligible for Medicaid in Texas for 2026, resulting in `is_medicaid_eligible = False`. The person's `medicaid_category` evaluated to 9, indicating they qualify through none of the standard Medicaid eligibility pathways. While the person met the immigration status requirement (`is_medicaid_immigration_status_eligible = True`), having been in the US for 5 years, this alone was insufficient for eligibility. Additionally, the person did not qualify for California's FFYP program (`ca_ffyp_eligible = False`) or Illinois's HBI program (`il_hbi_eligible = False`), which might have provided alternative pathways. With a MAGI income level of 18.81 times the federal poverty line and no qualifying category, the household falls outside Texas's Medicaid eligibility criteria for 2026.",
us,scenario_057,head_medicaid_eligible,1.0,1,"PolicyEngine determined that the head of household is eligible for Medicaid (is_medicaid_eligible = True) based on their status as an SSI recipient. Under Louisiana's Medicaid rules for 2026, individuals receiving Supplemental Security Income (SSI) are automatically enrolled in the SSI_RECIPIENT medicaid category, which carries automatic Medicaid eligibility regardless of income level. Although the household's Modified Adjusted Gross Income (MAGI) is measured at 0.45 times the Federal Poverty Level, this income threshold is not the binding constraint for SSI recipients—categorical eligibility through SSI receipt itself is sufficient to establish Medicaid eligibility. Therefore, PolicyEngine returned a value of 1.0 (True) for the head's Medicaid eligibility status.",
us,scenario_123,spouse_medicaid_eligible,0.0,1,"PolicyEngine determined that the spouse is not eligible for Medicaid in Pennsylvania for 2026. The spouse, age 39, has a Modified Adjusted Gross Income (MAGI) of 5.31 times the Federal Poverty Level, which exceeds the income thresholds for all available Medicaid eligibility pathways in the state. With household income around $145,002, the spouse does not qualify through any of Pennsylvania's Medicaid categories—including those for parents, pregnant individuals, children, or disabled individuals—as their income substantially surpasses the relevant limits for each pathway. Consequently, the engine assigned an eligibility status of False with no qualifying category.",
us,scenario_031,head_medicaid_eligible,1.0,13,"PolicyEngine determined that the head of household is eligible for Medicaid in California as a senior-or-disabled individual. The person, age 67, qualifies through the optional senior-or-disabled category (a non-MAGI pathway) because they meet three conditions: they are aged or disabled (is_ssi_aged_blind_disabled = True), they satisfy the income test under this category (is_optional_senior_or_disabled_income_eligible = True), and they meet the asset limits (is_optional_senior_or_disabled_asset_eligible = True). Additionally, the person satisfies California's immigration status requirements, having been in the United States for 5 years. Under the senior-or-disabled category, countable income is calculated by deducting health insurance premiums, including Medicare Part B, from gross income, which results in the household falling within California's income limits for this eligibility group.",
us,scenario_031,head_medicaid_eligible,1.0,13,"PolicyEngine determined that the head of household is eligible for Medicaid in California through the optional senior-or-disabled category. At 67 the head is aged, which meets the category's first condition (is_ssi_aged_blind_disabled = True) without SSI receipt or a disability; the adult expansion group (ages 19 to 64) does not apply, and in the engine's Medicaid rules Medicare eligibility bars only that group. Income follows SSI rules: Social Security ($9,445.47, counted in full), the private pension ($13,608) and the IRA distribution ($800) total $23,853.47; the $1,165 of alimony paid is not subtracted. California's $230 monthly income disregard ($2,760 a year), which takes the place of SSI's $20 general income exclusion, leaves countable income of $21,093.47, within the $22,024.80 limit, 138% of the 2026 federal poverty guideline for one person (is_optional_senior_or_disabled_income_eligible = True). Countable resources are the $4,200 bank balance, under California's $130,000 limit for an individual (is_optional_senior_or_disabled_asset_eligible = True). The household lists no immigration status, so the engine's default status (citizen) applies and the immigration test passes.",
us,scenario_071,head_medicaid_eligible,0.0,7,"PolicyEngine determined that this 76-year-old New York resident is not eligible for Medicaid (is_medicaid_eligible = False) because they do not qualify through any of the available eligibility pathways. While the person meets the immigration status requirement for Medicaid (is_medicaid_immigration_status_eligible = True, having been in the US for 5 years), they fail to qualify under the primary eligibility categories tracked by the engine. Specifically, the person does not qualify for California's FFYP program (ca_ffyp_eligible = False) or Illinois's HBI program (il_hbi_eligible = False), and their medicaid_category is assigned as NONE, indicating no qualifying pathway. With a MAGI income level of 2.28 times the Federal Poverty Level, the household's income and other circumstances do not align with any of New York's Medicaid eligibility categories for this household composition and age.",
us,scenario_085,head_medicaid_eligible,0.0,7,"PolicyEngine determined that the head of household is not eligible for Medicaid in Pennsylvania in 2026, despite meeting the immigration status requirement. The person qualifies as immigration-eligible (having been in the US for 5 years), but this alone is insufficient for Medicaid eligibility. The engine evaluated two state-specific pathways—California's FFYP (Family Planning) program and Illinois's HBI (Health Benefits for Immigrants) program—both of which returned false, indicating the household does not qualify through these mechanisms. With a MAGI income of 3.69 times the federal poverty level and no qualifying Medicaid category identified, the person falls outside Pennsylvania's Medicaid eligibility pathways for 2026, resulting in an eligibility determination of False.",
us,scenario_084,head_medicaid_eligible,0.0,7,"PolicyEngine determined that the head of household is not eligible for Medicaid in North Carolina in 2026, despite meeting the immigration status requirement. The person qualifies as immigration-eligible (having been in the US for 5 years), but this alone is insufficient for Medicaid eligibility. The engine assigned a medicaid_category value of 9, indicating no qualifying eligibility pathway, which directly resulted in is_medicaid_eligible being set to False. With a Modified Adjusted Gross Income (MAGI) of 1.56 times the Federal Poverty Level and no categorical eligibility through either the California FFYP or Illinois HBI programs, the household falls outside North Carolina's Medicaid coverage groups for this income and household composition.",
Expand Down
6 changes: 3 additions & 3 deletions paper/snapshot/20260501/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@
},
"files": {
"us_adjudications.json": "42d6a90fe402b6dc8a37422999ea05cd0e481a5ead690ee4045f649d682270c6",
"us_audit_row_annotations.csv": "3b27bfa643a5725935e90bea39f2584dc507572418a8e1474e981b841078d3ad",
"us_case_notes.csv": "2bef75c043ec08338038276e018f300ed86fcb937c44ee26d0daadb58d7f2c08",
"us_case_reference_explanations.csv": "6cccc5586815f2ecc4f1b527e79e141f858d0e75cc30b3a867d107909cbb38c4"
"us_audit_row_annotations.csv": "ef393cc0604fe3ef57a1dc0eb10395405c818521ec172e29a00748b6e8657f0f",
"us_case_notes.csv": "fa26630bcdace072230beb23f4699e3f52f1ed6dba45cd8a89e745d4d4de00b5",
"us_case_reference_explanations.csv": "c5905eff49c28bf7fec436e49b5608f298eb8b3fb110cf35e214f4daacb0b275"
},
"judge_provenance": {
"by_judge": {
Expand Down
Loading
Loading