Repository navigation
Fix fused reference URLs repo-wide and correct Arkansas DFA page anchors - #9609
Merged
Merged
Conversation
Python joins adjacent string literals, so reference tuples written without commas between URLs collapsed into one string that renders as a single dead link. Add the missing commas across 208 variable and reform files, split the Kansas and FLSA references that were already fused inside one literal, and strip stray whitespace and inline annotations from reference URLs. Move the doubled URL in az_taxable_income's documentation string into its reference. Add a code-health test that fails when any reference entry is not a single bare URL: baseline variables (at runtime), reform variables (from source, resolving module-level constants and dict() calls), and parameter references, including those under a dated value's own metadata. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every changed anchor was checked by extracting the anchored PDF page: - Low income tax tables: 2023 and 2024 booklet anchors used printed page numbers; point them at the tables (2023 p29/p30, 2024 p28/p29) and name the standalone TaxTables_FI PDFs as tax tables, not instructions. - Retirement/disability cap: 2021 booklet LINE 18A is on p12, not p13. - Exemptions: add each booklet's item 12 page next to the item 1-7 page. - Itemized deduction sources: 2021 AR3 instructions are on p20. - AR AGI: cite the 2022 booklet LINE 23-25 page (p14), not the proration page (p22). - Miscellaneous deduction: cite AR3 lines 20-25 and its instructions instead of the AR1075 tuition form. - Disabled dependent credit: the 2023 and 2024 $500 amount is on the AR1000TC instructions (LINE 8 and LINE 7), not the 2023 AR1000F page 1 or the 2024 booklet LINE 7 page. - Non-refundable credits: the credit lines 34-37 are on AR1000F p2 and booklet p15. - 2025 booklet: the rate schedule anchor used the tax table's printed page number (26, now PDF p30), and the additional credit reduction start pointed at the e-file page (p5) instead of the worksheet (p25). - Fix three titles that cite the wrong document or line and the "Tution"/"eduction" typos, replace a third-party AR1000D mirror with DFA's copy, and add the 2025 AR1000D to the capital gains exemption parameters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
policyengine-core only reads `reference` from a parameter's `metadata` block (or a dated value's `metadata`); a top-level or per-value `reference` key is accepted for legacy compatibility and then dropped. Move these under `metadata` so they load and are covered by the reference URL test: the Arizona family tax credit cap and college savings cap, the Indiana aged low-AGI exemption threshold, the Massachusetts HECS prior-year cost threshold, and the 14 Rev. Proc. citations on dated values of the federal exemption amount. No values change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
fix/ar-reference-hygiene
branch
from
September 25, 2026 02:45
f225183 to
6a7b430
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9609 +/- ##
============================================
- Coverage 100.00% 64.38% -35.62%
============================================
Files 4 236 +232
Lines 76 3656 +3580
Branches 2 18 +16
============================================
+ Hits 76 2354 +2278
- Misses 0 1302 +1302
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Oct 3, 2026
The fused-URL half of this branch (0ccab71) landed on main as #9786, with its own test_no_concatenated_reference_urls.py. Every file that commit touched now carries main's version: 189 were already identical, 46 differ and take main's. The only file kept from it is this branch's runtime test, tests/code_health/test_reference_urls.py, which also covers reform modules and parameter references. The Arkansas DFA anchor commit (e11a9d2) and the parameter-metadata commit (6a7b430) are re-applied on top of main: - tuition weighted-average files: main's new 2015-2020 values and references, plus this branch's "Tuition"/"education" title fixes. - low-income tables (two or more dependents): main (#9599) already has the 2023 p29 and 2024 p28 anchors; re-applied the 2025 TaxTables_FI title. - rates/main/{rate,reduction}.yaml: main already points the 2025 booklet at p30; main's version. - ar_misc_deduction_{indiv,joint}.py: main (#9625) already replaced the AR1075 citation with the statute and the 2025 AR3 Lines 20-25 instructions; main's version. - MA HECS prior-year cost threshold: moved main's reference block, with main's archived FY2023/FY2025 URLs (#9765), under metadata. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#9656 added this parameter after the branch was cut, with one href ending "#page=1, 20", which this branch's reference test rejects. Cite the Summary (p1) and Ordering Paragraph 2 (p20), the two places the decision raises the CARE discount from 15% to 20%, as separate entries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
That half of the branch landed on main as #9786, which has its own changelog entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Merged by the US + core hub on its merge gates. Audit:
|
This was referenced Oct 6, 2026
This was referenced Oct 6, 2026
MaxGhenis
added a commit
that referenced
this pull request
Oct 9, 2026
…in documentation (#9849) * Catch reference keys policyengine-core drops and URLs filed in documentation policyengine-core reads a parameter's reference and unit only from metadata, and it accepts both keys elsewhere without a warning (LEGACY_COMPAT_KEYS). That is how 14 IRS exemption citations went missing until #9609. Every other key it does not recognize raises a ParameterKeyWarning, but policyengine_us.model_api ignores all warnings, so the Pell Grant 2024-08-01 threshold citation, filed under a misspelled refrence: key, never loaded either. - Add test_parameter_yaml_keys.py: a raw-YAML scan that fails, naming the file and line, on any reference or unit key outside metadata, and a load of the parameter tree with ParameterKeyWarning recorded. - Move the Pell citation under the dated value's metadata. - Check variable documentation (baseline at runtime, reforms from source): documentation that is only URLs fails, as does any URL it cites that reference does not list. Move the URLs of 39 variables into reference, including az_taxable_income's doubled URL, keeping each inline comment on its URL. - Accept a URL after "=" only as a query value (?name= or &name=), in both test_reference_urls.py and test_no_concatenated_reference_urls.py, so x.pdf#page=https://... is reported as two fused URLs. No parameter value or formula changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fix plural citation metadata and raw YAML guard --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #9494 (merged), which moved these Arkansas references to DFA's
wp-content/uploadspath. This PR fixes the pre-existing reference problems an adversarial review of #9494 found. It also adds a runtime test that every variable, reform and parameter reference entry is a single bare URL.Summary
This PR only changes reference metadata and adds a code-health test. No formulas or parameter values change. Comparing all 30 parameter files in the diff with
main, withmetadata(references, labels) anddescriptionstripped, gives identical values. The only description change is the "eduction" typo fix intuition/rate.yaml.1. Reference-URL test (
policyengine_us/tests/code_health/test_reference_urls.py)Every reference entry that contains a URL must be exactly one bare URL. The test covers three things:
WATCA_REFERENCESanddict(title=…, href=…)calls, and anything it can't evaluate fails rather than being skipped.hrefs, including references under a dated value's ownmetadata.Wayback snapshot URLs (
web.archive.org/web/<ts>/https://…) and query-embedded URLs (?docName=https://…) are allowed.#9786's
test_no_concatenated_reference_urls.pyscans variable classes statically. This test adds runtime variable references, reform modules and parameter references.On current
mainit found one violation, which #9656 added after this branch was cut.gov.states.ca.cpuc.care.gas_discountcited CPUC D.01-06-010 as…/7673.pdf#page=1, 20. Commitfff6e635c7splits it into the Summary (p1) and Ordering Paragraph 2 (p20). Both pages state the CARE discount increase from 15% to 20%; I checked them withpdftotext.2. Arkansas DFA
#page=anchors and titles (e11a9d2e2e)For every changed anchor, I extracted the target page from the DFA PDF with
pdftotext -f N -l N, and it shows the cited content. Where a parameter value is involved, that year's value appears on the page. Most of the wrong anchors used the booklet's printed page number instead of the PDF page index.rates/low_income_tax_tables/*(5 files)#page=25→29(single/HoH/SS) and26→30(joint). 2024#page=24→28and25→29. In the two HoH/SS "2 or more dependents" filesmainalready has these anchors (#9599).rates/low_income_tax_tables/*(all 7 files)TaxTables_FI_2024/2025.pdfnow say "Arkansas Tax Tables" instead of "Income Tax Instructions".exemptions/retirement_or_disability_benefits/cap.yaml#page=13→12exemptions/exemptions.yamldeductions/itemized/sources/{individual,joint}.yaml#page=21→20ar_agi_{indiv,joint}.py#page=22→14credits/personal/amount/disabled_dependent.yamlcredits/non_refundable.yaml#page=1→2. The 2024 entry had no anchor and a title naming the return, but it links the booklet, so it now points to#page=15with a matching title.credits/additional_tax_credit_for_qualified_individuals/reduction/start.yaml#page=5→25credits/personal/{age_threshold,amount/base}.yaml,contrib/states/ar/dependent_credit/amount.yamlgross_income/capital_gains/{exempt/cap,exempt/rate,loss_cap}.yaml,ar_taxable_capital_gains_{indiv,joint}.pyexempt/capandexempt/rate, since the 2025 booklet doesn't state $10,000,000 or 50%.deductions/itemized/tuition/**main's new 2015–2020 AR1075 values and references (#9615).Two changes from the original commit drop out because
mainalready has them:rates/main/{rate,reduction}.yaml:mainalready points the 2025 booklet at#page=30.ar_misc_deduction_{indiv,joint}.py: Use Arkansas's own miscellaneous and casualty loss itemized deduction rules #9625 replaced the AR1075 (tuition form) citation with Ark. Code § 26-51-437(a) and the 2025 AR3 Lines 20–25 instructions.3. Parameter references that never loaded (
6a7b430ef8)policyengine-core reads
referenceonly from a parameter'smetadatablock or from a dated value'smetadata. A top-level or per-valuereferencekey is accepted for legacy compatibility and then dropped. This commit moves such keys undermetadataso they load and the test above covers them:referencemoved intometadatain 4 files:az/…/family_tax_credits/amount/cap.yaml,az/…/subtractions/college_savings/cap.yaml,in/…/exemptions/aged_low_agi/threshold.yaml,ma/doer/liheap/hecs/eligibility/prior_year_cost_threshold.yaml. The MA block keepsmain's archived FY2023 and FY2025 URLs (Replace dead Massachusetts reference URLs #9765).irs/income/exemption/amount.yaml: 14 Rev. Proc. citations moved into their values'metadata.Checked and deliberately left alone:
gross_income/sources/{individual,joint}.yaml, 2022/2025 booklet#page=13: the income section begins on p12 (LINE 8), but p13 holds most of the listed lines, including LINE 21 (unemployment), which these entries were added for in Exclude Arkansas unemployment compensation from gross income in 2021 #9451.deductions/itemized/tuition/rate.yaml, 2022 booklet#page=21: it shows "LINE 19 … attach Form AR1075(s)", not the 50% rate. No page of the 2022 booklet states the rate. The AR1075 entries in the same file do.The four non-metadata problems the original audit found are all fixed on
mainnow:medical_expense/income_floorparameters.Overlap with open PRs
main, the one-line conflict this PR predicted in the two HoH/SS "2 or more dependents" files was resolved as planned:main's identical 2024 href plus this PR's 2025 title.ar_agi_joint.py, in different hunks, andgit merge-treemerges that file cleanly. Fix AR, MT and NM income tax gaps found in TAXSIM comparison sweep #9585's own conflicts are withmain, not with this PR:gross_income/sources/*.yamlandar_gross_income_indiv.yaml.Verification (head
f065f0113b, after mergingmainat8c93745c89)test_reference_urls.py,test_no_concatenated_reference_urls.pyandtest_known_dead_reference_urls.py: 93 passed.policyengine-core test policyengine_us/tests/policy/baseline/gov/states/ar: 880 passed.policyengine-core teston AZ income subtractions and credits, IN exemptions, MA LIHEAP,contrib/states/ar, CA CPUC and IRS income exemptions: 332 passed.ruff format --checkon the PR's five Python files passes.mainfor all 30 parameter files in the diff.Before the merge, an independent review recomputed the meaning-preservation check for the fused-URL commit, re-verified every new anchor against its PDF page, and confirmed every new or changed DFA URL returns
200 application/pdf.Microsimulation impact
None. The PR changes reference metadata (URLs, page anchors and where
referenceentries sit so policyengine-core loads them) and adds a code-health test. No parameter value, formula or dataset changes, so no model output changes.axiom: n/a: reference metadata only
🤖 Generated with Claude Code