Skip to content

Replace dead casetext.com references with live sources - #9627

Merged
MaxGhenis merged 3 commits into
mainfrom
fix/replace-casetext-references
Oct 3, 2026
Merged

Replace dead casetext.com references with live sources#9627
MaxGhenis merged 3 commits into
mainfrom
fix/replace-casetext-references

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

casetext.com shut down: every casetext URL now returns HTTP 410 Gone. This PR removes all 199 casetext hrefs in policyengine_us/ (55 distinct casetext URLs, covering 53 sections across 23 jurisdictions). 198 now point to live sources for the same sections. The 199th was a duplicate, so it was deleted (see North Dakota below). After this PR, grep -rn casetext policyengine_us returns nothing.

Source choice, in order of preference:

  1. The official legislature or code site, where it serves a stable per-section URL: AZ, CA, DE, IL, KS, MD, MT, NC, ND, VT, AL (ALISON renders the section in a browser), the GA and NM administrative codes, the Colorado Secretary of State CCR, and IDAPA. Oregon's official site only has chapter pages, so those links go to the chapter and the title keeps the section number.
  2. law.justia.com, where the state has no free official per-section code: AR, CO, GA statutes, MS, NJ statutes, NM statutes (NMOneSource links are opaque JS fragments), PR. This follows each state's existing convention in the repo.
  3. Cornell LII for NYCRR, N.J.A.C. and the Oklahoma Administrative Code:
    • Westlaw's free NYCRR labels itself "Unofficial".
    • N.J.A.C. is only on LexisNexis.
    • rules.ok.gov renders blank at a deep link, and the repo already cites LII for OAC in 31 files.

Version-qualified casetext slugs map to matching editions where the source keeps them:

  • Colorado § 39-22-104: the "until proclamation" version goes to the 2022 Justia edition, the "upon proclamation" version to 2023.
  • Georgia § 48-7-26: 2022 edition, which predates the 2024 split.
  • New Mexico § 7-2-5.13: 2024 edition, before the 2025 amendment.
  • Montana § 15-30-2131: the official 2021 MCA archive, since the current page only says "Repealed".

Some official sites keep no past editions (Maryland, Arizona, Arkansas on Justia's undated path). Their version-qualified slugs link to the current text, and every subsection the titles cite was checked against that text.

Verification

Every replacement was checked for liveness and for content: the page must show the named section, not just return HTTP 200. codes.ohio.gov, for example, answers a missing section with a 200 "number-not-found" page.

  • 35 distinct URLs checked with curl (browser user agent, redirects followed). The script matched the section number in the page text, the PDF #page=N index (pdftotext), or the PDF #nameddest= target (pypdf). Maryland prints "§10–704" with an en dash, so dashes were normalized before matching.
  • 18 Justia URLs checked in a real browser, because Justia sits behind a Cloudflare challenge. Each page title names the section and the full section text renders.
  • ALISON (Alabama) is a JavaScript app, so it was checked in a browser too: § 40-18-19 renders, including (a)(13), the retirement exemption.

Other fixes in the touched references

Titles that misidentified the linked law:

  • Kansas oasdi/agi_limit: the title cited § 79-32,271 (a food-purchase credit). The Social Security subtraction is § 79-32,117(c)(xviii).
  • Idaho (5 files): "IDAHO STATUTES || Section 35.01.01.075" is a rule, so the title is now IDAPA 35.01.01.075.
  • New Mexico: 3.3.1.12 is NMAC, not the Income Tax Act.
  • Colorado charitable subtraction (1 YAML file, 2 Python files): the link is 1 CCR 201-2, Rule 39-22-104(4)(m), not C.R.S.
  • Georgia (28 files): O.C.G.A. 48-7 is Chapter 7, but titles called it "CH17 Income Taxes". Every title carrying the label is fixed, not just the three next to casetext links, so the Georgia titles stay consistent.
  • North Dakota stillborn: removed the "(Casetext rendering)" entry. Its replacement would exactly duplicate the official t57c38.pdf#nameddest=57-38-30p3 entry directly above it.
  • Ohio adoption credit (5 files): the casetext link was the "Repealed" page for R.C. 5747.37. The same files already cite the 2022 text on Justia, so this entry now cites the repealing act instead: H.B. 45 (134th G.A.) §§ 105.10 and 107.10, PDF page 100.

Subsection pins checked against the official text and corrected:

File Old pin New pin
CA amt/tentative_min_tax_rate § 17062 (b)(3)(iv) (b)(3)(A)(iv)
CA foster_youth/phase_out/start ($25,000 threshold) § 17052.2(a)(C)(i) (c)(2)
MD max_two_income_subtraction § 10-207 (r)(2)(a) (r)(2)
MD aged/age, aged/amount ($1,000 for a filer 65 or older) § 10-211(b)(2) (b)(3)
MD ctc/amount ($500) § 10-751 (a)(2)(ii.) (b)
CO cdcc/low_income/rate (25%) § 39-22-119.5 (3), (c), (I) (3)(b)
CO senior housing age_limit § 39-22-544 (4)(a) (3)(a)(II)(A), (3)(b)(II)(A)
CO senior housing income_threshold § 39-22-544 (4)(a) (3)(a)(II)(B), (3)(b)(II)(B)
MT CDCC phase_out § 15-30-2131.(v) (1)(c)(v)

URLs that Python fused into one string:

  • md_dependent_care_subtraction and both ms_itemized_deductions_* files lacked commas between tuple elements. These carry the same fix as Fix fused reference URLs repo-wide and correct Arkansas DFA page anchors #9609.
  • al_retirement_exemption, al_retirement_exemption_eligible_person, co_low_income_cdcc and co_low_income_cdcc_eligible put comma-less URLs in documentation. They now sit in a proper reference tuple; the modeling notes are kept as comments.
  • ca_renter_credit kept its statute URL in documentation; it is now reference.
  • ny_additions and ny_agi_subtractions held their link in a bare dict(title=..., href=...) statement that was never assigned. It is now reference = dict(...), so the link finally reaches the variable metadata.
  • or_retirement_credit_household_income now names ORS 316.157 next to its chapter-page reference.
  • co_low_income_cdcc_eligible: the modeling note moved out of documentation cited (3)(a)(V)-(VI), which doesn't exist. Provider and dependent identification are in § 39-22-119.5(5)(a)-(c).

Interaction with #9609

#9609 adds tests/code_health/test_reference_urls.py and fixes fused references repo-wide. It also adds commas on three lines this PR rewrites (the MD dependent-care and two MS itemized-deduction tuples). Whichever PR merges second gets a trivial conflict there: keep this PR's lines, which contain #9609's commas plus the new URL.

On a trial merge of this branch with #9609, with those three conflicts resolved this way, test_reference_urls.py passes 16/16. A mutation check showed the test catches a fused URL planted in a parameter this PR edits. On this branch alone, the test's remaining failures are all fused references in files this PR does not touch; #9609 fixes them. policyengine_us/tests/code_health passes 45/45 on this branch.

Review

Two independent Opus 5.5 reviewers (legal mapping, and code structure) approved with no blocking findings. Their nits are addressed in the second commit. They also found a Maryland title citing § 10-207(r) for a rule that lives in § 10-208(e); that section-level change is deferred to a follow-up, along with other pre-existing cross-section citation issues.

Mapping

Section Source Refs Replacement
1 CCR 201-2, Rule 39-22-104(4)(m) official 3 https://www.sos.state.co.us/CCR/GenerateRulePdf.do?fileName=1+CCR+201-2&ruleVersionId=11214#page=15
20 NYCRR § 112.2 Cornell LII 1 https://www.law.cornell.edu/regulations/new-york/20-NYCRR-112.2
20 NYCRR § 112.3 Cornell LII 6 https://www.law.cornell.edu/regulations/new-york/20-NYCRR-112.3
20 NYCRR § 132.1 Cornell LII 1 https://www.law.cornell.edu/regulations/new-york/20-NYCRR-132.1
3.3.1.12 NMAC official 1 https://www.srca.nm.gov/parts/title03/03.003.0001.html
32 V.S.A. § 5822 official 4 https://legislature.vermont.gov/statutes/section/32/151/05822
32 V.S.A. § 5830f official 6 https://legislature.vermont.gov/statutes/section/32/151/05830f
35 ILCS 5/244 official 2 https://www.ilga.gov/Documents/legislation/ilcs/documents/003500050K244.htm
Ala. Code § 40-18-19 official 2 https://alison.legislature.state.al.us/code-of-alabama?section=40-18-19
Ariz. Rev. Stat. § 43-1088 official 2 https://www.azleg.gov/ars/43/01088.htm
Ark. Code Ann. § 26-51-201 Justia 1 https://law.justia.com/codes/arkansas/title-26/subtitle-5/chapter-51/subchapter-2/section-26-51-201/
Ark. Code Ann. § 26-51-404 Justia 4 https://law.justia.com/codes/arkansas/title-26/subtitle-5/chapter-51/subchapter-4/section-26-51-404/
Ark. Code Ann. § 26-51-501 Justia 3 https://law.justia.com/codes/arkansas/title-26/subtitle-5/chapter-51/subchapter-5/section-26-51-501/
Ark. Code Ann. § 26-51-503 Justia 1 https://law.justia.com/codes/arkansas/title-26/subtitle-5/chapter-51/subchapter-5/section-26-51-503/
C.R.S. § 39-22-104 Justia 4 https://law.justia.com/codes/colorado/2022/title-39/article-22/part-1/section-39-22-104/
C.R.S. § 39-22-104 Justia 6 https://law.justia.com/codes/colorado/2023/title-39/specific-taxes/income-tax/article-22/part-1/section-39-22-104/
C.R.S. § 39-22-119.5 Justia 3 https://law.justia.com/codes/colorado/title-39/specific-taxes/income-tax/article-22/part-1/section-39-22-119-5/
C.R.S. § 39-22-129 Justia 6 https://law.justia.com/codes/colorado/title-39/specific-taxes/income-tax/article-22/part-1/section-39-22-129/
C.R.S. § 39-22-544 Justia 3 https://law.justia.com/codes/colorado/title-39/specific-taxes/income-tax/article-22/part-5/section-39-22-544/
C.R.S. § 39-3-203 Justia 1 https://law.justia.com/codes/colorado/title-39/property-tax/exemptions/article-3/part-2/section-39-3-203/
Cal. Rev. & Tax. Code § 17052.1 official 1 https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=RTC&sectionNum=17052.1
Cal. Rev. & Tax. Code § 17052.2 official 2 https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=RTC&sectionNum=17052.2
Cal. Rev. & Tax. Code § 17053.5 official 3 https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=RTC&sectionNum=17053.5
Cal. Rev. & Tax. Code § 17062 official 4 https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=RTC&sectionNum=17062
Cal. Rev. & Tax. Code § 17073.5 official 1 https://leginfo.legislature.ca.gov/faces/codes_displaySection.xhtml?lawCode=RTC&sectionNum=17073.5
Del. Code tit. 30, § 1114 official 1 https://delcode.delaware.gov/title30/c011/sc02/index.html#1114
Ga. Comp. R. & Regs. r. 560-7-4-.04 official 1 https://rules.sos.ga.gov/gac/560-7-4-.04
IDAPA 35.01.01.075 official 5 https://adminrules.idaho.gov/rules/current/35/350101.pdf#page=17
Kan. Stat. Ann. § 79-32,117 official 1 https://ksrevisor.gov/statutes/chapters/ch79/079_032_0117.html
Kan. Stat. Ann. § 79-32,121 official 6 https://ksrevisor.gov/statutes/chapters/ch79/079_032_0121.html
Md. Code, Hum. Servs. § 5-501 official 3 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=ghu&section=5-501&enactments=false
Md. Code, Tax-Gen. § 10-207 official 2 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-207&enactments=false
Md. Code, Tax-Gen. § 10-207 official 2 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-207&enactments=false
Md. Code, Tax-Gen. § 10-208 official 1 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-208&enactments=false
Md. Code, Tax-Gen. § 10-211 official 14 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-211&enactments=false
Md. Code, Tax-Gen. § 10-704 official 12 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-704&enactments=false
Md. Code, Tax-Gen. § 10-716 official 9 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-716&enactments=false
Md. Code, Tax-Gen. § 10-751 official 7 https://mgaleg.maryland.gov/mgawebsite/Laws/StatuteText?article=gtg&section=10-751&enactments=false
Miss. Code Ann. § 27-7-17 Justia 2 https://law.justia.com/codes/mississippi/title-27/chapter-7/article-1/section-27-7-17/
Mont. Code Ann. § 15-30-2131(1)(c)(v) (2021) official 1 https://mca.legmt.gov/bills/2022/mca/title_0150/chapter_0300/part_0210/section_0310/0150-0300-0210-0310.html
N.C. Gen. Stat. § 105-153.5 official 2 https://www.ncleg.gov/EnactedLegislation/Statutes/HTML/BySection/Chapter_105/GS_105-153.5.html
N.D. Cent. Code § 57-38-01.28 official 2 https://ndlegis.gov/cencode/t57c38.pdf#nameddest=57-38-01p28
N.D. Cent. Code § 57-38-30.3 official 11 (+1 duplicate removed) https://ndlegis.gov/cencode/t57c38.pdf#nameddest=57-38-30p3
N.J. Admin. Code § 18:35-2.9 Cornell LII 1 https://www.law.cornell.edu/regulations/new-jersey/N-J-A-C-18-35-2-9
N.J. Stat. § 54A:3-1 Justia 7 https://law.justia.com/codes/new-jersey/title-54a/section-54a-3-1/
N.J. Stat. § 54A:3-1.1 Justia 2 https://law.justia.com/codes/new-jersey/title-54a/section-54a-3-1-1/
N.M. Stat. § 7-2-14 Justia 11 https://law.justia.com/codes/new-mexico/chapter-7/article-2/section-7-2-14/
N.M. Stat. § 7-2-5.13 Justia 2 https://law.justia.com/codes/new-mexico/2024/chapter-7/article-2/section-7-2-5-13/
O.C.G.A. § 48-7-26 Justia 2 https://law.justia.com/codes/georgia/2022/title-48/chapter-7/article-2/section-48-7-26/
Ohio R.C. 5747.37 (repealed) official 5 https://search-prod.lis.state.oh.us/api/v2/general_assembly_134/legislation/hb45/07_EN/pdf/#page=100
Okla. Admin. Code § 710:50-15-49 Cornell LII 3 https://www.law.cornell.edu/regulations/oklahoma/OAC-710-50-15-49
Or. Rev. Stat. § 315.273 official 1 https://www.oregonlegislature.gov/bills_laws/ors/ors315.html
Or. Rev. Stat. § 316.157 official 1 https://www.oregonlegislature.gov/bills_laws/ors/ors316.html
P.R. Laws tit. 13, § 30211 Justia 1 https://law.justia.com/codes/puerto-rico/title-thirteen/subtitle-17/part-ii/chapter-1007/subchapter-b/30211/
P.R. Laws tit. 13, § 30212 Justia 9 https://law.justia.com/codes/puerto-rico/title-thirteen/subtitle-17/part-ii/chapter-1007/subchapter-b/30212/

axiom: n/a: reference metadata only

🤖 Generated with Claude Code

@MaxGhenis @claude
Replace dead casetext.com references with live sources
9e57652 
casetext.com shut down and every URL now returns HTTP 410 Gone. Repoint
all 199 casetext hrefs (55 distinct sections across 23 jurisdictions) to
live sources for the same sections: official legislature/code sites where
they offer per-section pages, Justia where the state has no free official
code site, and Cornell LII for state regulations without usable official
deep links. Every replacement was verified to resolve and to show the
named section.

Also fix titles that misidentified the linked law (a regulation labelled
as a statute, a wrong section or chapter number, a Casetext attribution),
subsection pins verified wrong against the official text, and URL strings
that Python fused into one because they lacked commas or sat in
`documentation` instead of `reference`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the fix/replace-casetext-references branch from 5b6ad21 to 9e57652 Compare September 25, 2026 17:43
@MaxGhenis @claude
Address review nits on casetext reference replacement
69f079d 
- Oklahoma 710:50-15-49: cite Cornell LII's per-section page (the repo's
  convention for OAC) instead of the year-stamped Tax Commission PDF,
  whose page anchor would drift.
- New York 112.2/112.3: the repaired links sat in bare dict() statements
  that were never assigned, so assign them to `reference`.
- Oregon: name ORS 316.157 next to the chapter-page reference.
- Colorado low-income CDCC note: provider and dependent identification
  rules are in 39-22-119.5(5)(a)-(c), not (3)(a)(V)-(VI).
- Georgia: O.C.G.A. 48-7 is Chapter 7, so fix the "CH17 Income Taxes"
  label in every title that carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis mentioned this pull request Sep 26, 2026
Merged
@MaxGhenis @claude
Merge remote-tracking branch 'upstream/main' into fix/replace-casetex…
718bf5d 
…t-references

Conflicts: main had already replaced the casetext links in md_ctc.py and
md_ctc_eligible.py (kept main's versions) and added references to the
Arkansas rate schedule (kept main's additions with this branch's Justia
link for Ark. Code 26-51-201).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit c1e7f8c into main Oct 3, 2026
68 of 70 checks passed
@MaxGhenis
MaxGhenis deleted the fix/replace-casetext-references branch October 3, 2026 11:55
@MaxGhenis

MaxGhenis commented Oct 3, 2026

Copy link
Copy Markdown
Contributor Author

Merged on gates under Max's 2026-10-02 rule that obvious bug fixes merge without his approval, whatever they do to aggregates (cos card closed with "merge on gates").

  • Head merged: 718bf5d49a083888584a27659a320c5f80bc5dc9 (squash, --match-head-commit).
  • CI: gh pr checks exited 0 on this head: 35 passed, 0 skipped (Pull request run 37062902039 (attempt 2)). The earlier failures were runner out-of-memory deaths in states-shard-2/3 and ssa-usda on policyengine-core 3.32.12, re-run after 3.32.13 (policyengine-core#578) was released.
  • Mergeable: MERGEABLE, not a draft, no changes requested.
  • Independent review: an Opus 5.5 Subfleet review approved this exact head: ~/reviews/pe-us-merge-backlog-2026-10-02/reviews/9627-r1.md (local review folder).
  • Impact: stated in the PR body from real microsimulation runs (~/reviews/pe-us-merge-backlog-2026-10-02/impact/).
  • Admin merge: the repo requires one approving review and this PR is from Max's account, so no one else's approval was available; every other gate above held.

This was referenced Oct 3, 2026
Merged
Merged
MaxGhenis added a commit that referenced this pull request Oct 6, 2026
@MaxGhenis @claude
Record the 2026-10-06 casetext.com check and the verified link count
9ac1fc8 
A redirect-following GET of every casetext.com URL #9627 removed (46
statute, 9 regulation), four case pages, three index pages and four
made-up paths returned HTTP 410 with the same shutdown notice; only the
root redirects, to a Thomson Reuters product page. Update `verified` to
that check. #9627's base had 197 casetext.com links (55 distinct URLs),
not 199, by git grep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 6, 2026
@MaxGhenis @claude
Say #9627 removed the 197 casetext.com links, not replaced them
864967e 
#9627 deleted one duplicate North Dakota entry instead of replacing it, so
"removed" is the accurate verb. The 197 links and 55 distinct URLs are the
casetext.com URLs on the removed lines of #9627's diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis mentioned this pull request Oct 6, 2026
Merged
MaxGhenis added a commit that referenced this pull request Oct 6, 2026
@MaxGhenis @claude
List casetext.com in the known-dead reference URL guard (#9768)
47ab53e 
* List casetext.com in the known-dead reference URL guard

casetext.com shut down: statute, regulation and case pages return HTTP
410 Gone, and the home page redirects to a Thomson Reuters product page.
#9627 replaced all 199 references; this entry keeps new ones out of the
package and docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test that the casetext.com entry flags statute, regulation and case URLs

The casetext.com entry is the guard's only whole-host prefix. Pin what it
covers: a statute URL over http with www., a regulation URL and a case
URL in a markdown link are each flagged once, and a Wayback copy of a
casetext page is not. Without the entry the three flagged examples fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Record the 2026-10-06 casetext.com check and the verified link count

A redirect-following GET of every casetext.com URL #9627 removed (46
statute, 9 regulation), four case pages, three index pages and four
made-up paths returned HTTP 410 with the same shutdown notice; only the
root redirects, to a Thomson Reuters product page. Update `verified` to
that check. #9627's base had 197 casetext.com links (55 distinct URLs),
not 199, by git grep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Say #9627 removed the 197 casetext.com links, not replaced them

#9627 deleted one duplicate North Dakota entry instead of replacing it, so
"removed" is the accurate verb. The 197 links and 55 distinct URLs are the
casetext.com URLs on the removed lines of #9627's diff.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MaxGhenis