Repository navigation
v2 prompt and data: program-specific disability facts, integer inputs, and the take-up boundary #165
Description
v2 prompt and data: make disability facts program-specific, expose integer inputs, state the take-up boundary
Problem (found on the 39-model board, 2026-09-05)
The prompt lists a single undefined fact, - is disabled, and tells models that unlisted status facts are false and to assume program take-up. The engine's disability-linked gates are program-specific and read other inputs:
| Program | Engine gate | Present in the certified June build? |
|---|---|---|
| SSI | meets_ssi_disability_criteria and the SGA screen (42 USC 1382c) |
false for all 160,858 persons; never appears in any prompt |
| Medicare under 65 | social_security_disability > 0 and months_receiving_social_security_disability >= 24 (42 USC 426) |
months absent in every build, and integer-typed, which the prompt builder excludes |
| SNAP elderly/disabled | receipt-based is_usda_disabled |
works (SSI-disabled, SSDI income, veteran statuses) |
| IRC 152 / 22 | is_permanently_and_totally_disabled |
never populated in any build |
| IRC 21 CDCC | is_incapable_of_self_care |
absent until the 2026-07-23 builds (2,067 from PEDISDRS) |
is_disabled itself (any of six CPS difficulty items) feeds 128 variables: 113 state child-care, TANF and state-tax provisions, HUD, Medicaid work-requirement exemptions, VITA, and the SSI working-student exclusion (which should read is_ssi_disabled).
Consequences on the 39-model board: 34 SSI model-rows across five households paid SSI on a disabled under-65 member against a $0 reference fixed by the data; 18 of them are pure disability-gate disagreements (within 10% of the clean alternative amount), 16 carry independent errors (resource limit exceeded, one-third reduction from inferred shelter, wrong rates). One Medicare row (scenario_074) rests on the unlisted SSDI duration. See annotations//us_adjudications.json and the paper's audit section.
The preamble also does not state the take-up boundary: the engine deems the TANF non-cash benefit received when eligible (SNAP BBCE) while models are told not to infer unlisted benefit receipt.
Proposed preamble (Astra draft, 2026-09-05, to edit)
Estimate the requested outputs under the PolicyEngine-US version and calculation conventions identified with this question. All listed people live together in the stated household group. Unless dated explicitly, facts apply throughout the tax-benefit year without changes in status or income volatility; wage amounts are annual totals including overtime, and hourly rates are straight-time rates. Unlisted numeric inputs, including integer inputs, are zero, and unlisted boolean inputs are false, subject to the filing and take-up assumptions below. "General disability indicator" is a survey characteristic and does not establish SSI disability, tax-specific disability, SSDI entitlement, or Medicare eligibility. SSI disability uses the separately stated "meets SSI disability criteria before the substantial-gainful-activity test"; apply the earnings test separately. Tax-specific disability and self-care conditions are also stated separately. Medicare eligibility is evaluated on January 1; qualifying disability-benefit months are measured as of that date. Assume filing and full take-up of eligible requested benefits, plus eligible TANF/MOE noncash benefits used for SNAP categorical eligibility, and apply those computed benefits in downstream calculations. This exception permits computed receipt; it does not create substantive eligibility or historical entitlement. Social Security, SSDI, and veterans payments and histories are supplied inputs only. Infer no other income, expenses, assets, receipt, rent, or coverage. The SSI output includes federal SSI only; state supplements require a separate requested output.
Data and builder requirements
- Draw v2 households from a microcosm build that carries the SIPP-imputed
meets_ssi_disability_criteria(builds from 2026-07-17) andis_incapable_of_self_care(from 2026-07-23). - Populate
is_permanently_and_totally_disabled(or state that IRC 152/22 disability pathways are out of scope). - Carry SSDI entitlement duration (or start date) with a defined reference date, or state the assumption; allow integer inputs to be promptable.
- Label
employer_sponsored_insurance_premiumsas employer-paid (never in the stated wages). - Record provenance per disability fact: observed, imputed, or unknown. Unknown must not print as an observed negative.
- Engine nit:
is_ssi_blind_or_disabled_working_student_exclusion_eligiblereadsis_disabled; it should readis_ssi_disabled.
Related: #139 (v2 board condition), the prompt-ambiguity adjudication record introduced in #164.
Additions from the 2026-09-05 work (PR #164)
- The prompt builder exposes only float and boolean inputs;
months_receiving_social_security_disabilityis integer-typed and can never appear. Make integer inputs promptable. Scenario.to_pe_household()defined no marital units; policyengine-core lumped every member into one, deeming the whole household's income to any SSI-eligible adult. Fixed in Add six September 2026 models (GPT-6 Astra #2 of 39), exclude eleven ambiguous outputs from scoring, publish dashboard-data-20260905c #164 (head and spouse as the only couple); v2 should carryperson_marital_unit_idfrom the data instead of inferring it.- Regenerating today's references on policyengine-us 1.822.5 moves 36 of 1,984 outputs; one change (weekly hours default 0 instead of 40, so missing hours fail the SNAP work-requirement test) turns SNAP into an unlisted-input dependency for any household whose prompt lists no hours. v2 must list weekly hours whenever the engine reads them, or state the default in the preamble.
- Standing gate for v2 and every later board: recompute every reference under each unlisted-input reading before scoring; exclude what moves (the
reference_exclusions.jsonmechanism from Add six September 2026 models (GPT-6 Astra #2 of 39), exclude eleven ambiguous outputs from scoring, publish dashboard-data-20260905c #164), rather than waiting for a judge to notice.
Activity
Add opt-in v2 household prompt contract for #165 [skip ci]
Specify October 5 v2 fact and output-contract regressions (#165)
State October 5 audit facts in the unactivated v2 contract (#165)
Report required-fact gaps by replaying PR #196 across frozen househol…
Preserve v1 fingerprints and close October 5 contract review gaps (#165)
Bound v2 conventions to verified contexts and expose unknown readers (#…
MaxGhenis commentedon Oct 10, 2026
Two more prompt labels for v2, from a check of release dashboard-data-20261010's scenario_039 (VA) federal income tax. Six strong models agree against that reference; the reference holds, and these two labels are what misled the models or left the question open. Evidence: #213 (
reference_audit/2026-10-10-held-references/). Neither label is in this issue yet.1. "is a surviving spouse" is the Code's name for a filing status; the data flag is only "widowed"
is_surviving_spouseprints as "is a surviving spouse" (policybench/prompts.py). 26 U.S.C. 2(a) uses that term for the status that gets the joint rate table and standard deduction. It requires a dependent child living in the home and a death in one of the two preceding tax years.person.A_MARITL == 4(policyengine_us_data/datasets/cps/cps.pyline 1212 at main42ed5d45);marital_status == 4(packages/microcosm-build/src/microcosm/build/us_runtime/relationship_inputs.pyline 181 at mainaca40a69).surviving_spouse_eligible), so the reference files a lone widow as single.2. "estate income" states no character
estate_incomeprints as "estate income". The engine's input is ordinary income: Schedule E Part III income carried to Schedule 1 line 5, with an estate's interest, dividends and capital gains in separate inputs (the variable's documentation in policyengine-us 2.38.6)."employer sponsored insurance premiums", the label behind the other cell in #213 (OH scenario_025), is already in this issue's data requirements.
Record Max's d1252 and d1263 rulings: keep both, state the rate-sched…
MaxGhenis commentedon Oct 14, 2026
Two rulings from Max on 2026-10-13 add items for the v2 prompt:
reference_audit/2026-10-10-tax-table/README.md("What each option would take"). The benchmark card and paper state the convention from release dashboard-data-20261014 (PR Add Gemini 4 Argon and Muse Spark 1.3 (release, 50 models) [WIP] #221).reference_audit/2026-10-10-held-references/README.md("Prompt labels" and the reviewers' dissents).