Skip to content

Replace dead reference URLs for SC, KY, OK, FL, OH, MO, NC and SD - #9773

Merged
MaxGhenis merged 5 commits into
mainfrom
dead-urls-southeast
Oct 5, 2026
Merged

MaxGhenis merged 5 commits into
mainfrom
dead-urls-southeast

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Part of the repo-wide audit of dead reference URLs outside tax.ny.gov (follow-up to #9711 and #9730). This PR covers South Carolina, Kentucky, Oklahoma, Florida, Ohio, Missouri, North Carolina and South Dakota, plus the retired efile.com pages.

  • 31 dead URLs, now point to a live copy of the same document, a live official page with the same content, or a pinned Wayback capture of the dead URL with the same #page.
  • Only references change. A script checked that every changed line under parameters/, variables/ and docs/ differs from the old line only inside URLs. The exceptions: four South Dakota Python reference assignments that ruff re-wrapped (their ASTs match apart from the URL strings), and the South Carolina and Wisconsin reference lists changed after review.
  • policyengine_us/tests/code_health/known_dead_reference_urls/southeast.toml lists 31 of the dead prefixes, so the offline guard from Replace dead NY tax department reference URLs #9711 fails if they return.

How each URL was checked

  1. Dead: GET with curl -sL -A "Mozilla/5.0" on 2026-09-30, rechecked with a Chrome user agent. A 403, 429 or bot challenge counted as unknown, not dead, and was rechecked inside a real browser page; URLs still behind a bot wall were left unchanged. A 200 that landed on a home, search, login or not-found page counted as dead.
  2. Replacement: one research agent per batch looked for the same document on the agency's current site, then for an official page stating the same content for the same period, then for a pinned Wayback capture. Each mapping records the quoted content it checked.
  3. Independent verification: a second agent re-checked every live replacement and a sample of the Wayback ones against the values the citing file uses (pdftotext on the cited #page for PDFs). Disputed items went to an adjudicator under fixed rules: the agency's own site, then another official publisher, then a Wayback capture of the dead URL. Third-party mirrors (taxformfinder.org, NBER and the like) were not accepted for official citations.
  4. Mechanical: every replacement URL returned 200 with the expected content type. Wayback URLs use 14-digit timestamps and replay the same original URL, and every #page exists in its PDF. A few replacements behind bot walls were confirmed in a browser instead.

Changes

Dead URL Dead because Kind New target Refs
https://codes.ohio.gov/assets/laws/revised-code/authenticated/4141/4141.01/3-31-2024/4141.01%203-31-2024.pdf 404 live, fragment updated https://codes.ohio.gov/assets/laws/revised-code/authenticated/41/4141/4141.01/10-3-2023/4141.01-10-3-2023.pdf 1
https://codes.ohio.gov/assets/laws/revised-code/authenticated/4141/4141.25/3-31-2024/4141.25%203-31-2024.pdf 404 live, fragment kept https://codes.ohio.gov/assets/laws/revised-code/authenticated/41/4141/4141.25/3-28-2017/4141.25-3-28-2017.pdf 1
https://codes.ohio.gov/ohio-administrative-code/rule-5180:2-16-01 redirects live https://codes.ohio.gov/assets/laws/administrative-code/authenticated/5180/2/16/5180$2-16-01_20221211.pdf 9
https://codes.ohio.gov/ohio-administrative-code/rule-5180:2-16-03 redirects live https://codes.ohio.gov/assets/laws/administrative-code/authenticated/5180/2/16/5180$2-16-03_20220227.pdf 8
https://codes.ohio.gov/ohio-administrative-code/rule-5180:2-16-05 redirects live https://codes.ohio.gov/assets/laws/administrative-code/authenticated/5180/2/16/5180$2-16-05_20221211.pdf 4
https://codes.ohio.gov/ohio-administrative-code/rule-5180:2-16-09 redirects live https://codes.ohio.gov/assets/laws/administrative-code/authenticated/5180/2/16/5180$2-16-09_20231007.pdf 2
https://dese.mo.gov/sites/dese/files/media/file/2025/12/2025%20Rates%20Held%20Harmless%202.0.xlsx 404 live https://dese.mo.gov/sites/g/files/zuston521/files/media/file/2025/12/2025%20Rates%20Held%20Harmless%202.0.xlsx 17
https://dese.mo.gov/sites/dese/files/media/pdf/2025/10/10.2025%20Income%20Eligibility%20Table%20%282%29.pdf 404 live, fragment kept https://dese.mo.gov/sites/g/files/zuston521/files/media/pdf/2025/10/10.2025%20Income%20Eligibility%20Table%20%282%29.pdf 29
https://dese.mo.gov/sites/dese/files/media/pdf/2026/04/ACF-118%20CCDF%20FFY%202025-2027%20For%20Missouri%20-%20Approved%203.26.2026.pdf 404 live, fragment updated https://dese.mo.gov/sites/g/files/zuston521/files/media/pdf/2026/04/ACF-118%20CCDF%20FFY%202025-2027%20For%20Missouri%20-%20Approved%203.26.2026.pdf 11
https://dor.sc.gov/forms-site/Forms/SC1040ES_2023.pdf 404 Wayback, fragment kept https://web.archive.org/web/20240309102037/https://dor.sc.gov/forms-site/Forms/SC1040ES_2023.pdf 1
https://dor.sc.gov/forms-site/Forms/TC60_2021.pdf 404 live https://dor.sc.gov/sites/dor/files/forms/TC60_2021.pdf 3
https://dor.sc.gov/forms-site/Forms/TC60_2022.pdf 404 live https://dor.sc.gov/sites/dor/files/forms/TC60_2022.pdf 1
https://dor.sc.gov/rebate-2022 404 Wayback https://web.archive.org/web/20251003201253/https://dor.sc.gov/rebate-2022 1
https://dss.mo.gov/fsd/iman/medasst/0865-010-15.html 404 Wayback https://web.archive.org/web/20250311062044/https://dss.mo.gov/fsd/iman/medasst/0865-010-15.html 2
https://dss.sd.gov/docs/childcare/assistance/Subsidy_Manual.pdf 404 Wayback, fragment kept https://web.archive.org/web/20251031171652/https://dss.sd.gov/docs/childcare/assistance/Subsidy_Manual.pdf 15
https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/511-EIC.pdf 404 live, fragment kept https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/511-EIC-2024.pdf 1
https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/511-Pkt.pdf 404 live, fragment kept https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/511-Pkt-2024.pdf 25
https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/538-H.pdf 404 live https://oklahoma.gov/content/dam/ok/en/tax/documents/forms/individuals/past-year/2024/538-H-2024.pdf 4
https://oklahoma.gov/okdhs/library/policy/current/oac-340/chapter-10/subchapter-3/parts-3/earned-income-disregard.html 404 Wayback https://web.archive.org/web/20250808211623/https://oklahoma.gov/okdhs/library/policy/current/oac-340/chapter-10/subchapter-3/parts-3/earned-income-disregard.html 3
https://policies.ncdhhs.gov/wp-content/uploads/chapter-8-parental-fees-7.pdf 404 live, fragment kept https://policies.ncdhhs.gov/wp-content/uploads/Chapter-8-CN26-03-Parent-Fees-2-BLACK.pdf 2
https://revenue.ky.gov/Forms/1999_42a740p.pdf 404 Wayback https://web.archive.org/web/20161221194043/https://revenue.ky.gov/Forms/1999_42a740p.pdf 1
https://revenue.ky.gov/Forms/2000_42a740p.pdf 404 Wayback https://web.archive.org/web/20161221171528/https://revenue.ky.gov/Forms/2000_42a740p.pdf 1
https://revenue.ky.gov/Forms/740%20Packet%20Instructions%20(2025).pdf 404 live, fragment kept https://revenue.ky.gov/Forms/740%20Packet%20Instructions.pdf 1
https://revenue.ky.gov/Forms/740%20instructions%20packet%20(2025).pdf 404 live, fragment kept https://revenue.ky.gov/Forms/740%20instructions%20packet%20(2024).pdf 4
https://www.efile.com/oregon-tax-brackets-rates-and-forms/ redirects live, fragment updated https://www.oregon.gov/dor/forms/FormsPubs/publication-or-17_101-431_2023.pdf 5
https://www.efile.com/pdf/5988.pdf efile.com closed Wayback, fragment updated https://web.archive.org/web/20230220132302/https://www.michigan.gov/taxes/-/media/Project/Websites/taxes/Forms/2022/2022-IIT-Forms/BOOK_MI-1040CR-7.pdf?rev=d765b421bc7343c0a54793499dc550db&hash=0B10BE7DD0B91D7F609230E8F636EBC8 6
https://www.efile.com/south-carolina-tax-rates-brackets-and-forms/ efile.com closed live official schedule https://dor.sc.gov/sites/dor/files/forms/SC4972_2024.pdf#page=2 1
https://www.efile.com/wisconsin-tax-brackets-rates-and-forms/ efile.com closed Wayback for 2021, plus official DOR sources for 2022–2025 https://web.archive.org/web/20250119123144/https://www.efile.com/wisconsin-tax-brackets-rates-and-forms/ (see "Changes after review") 9
https://www.myflfamilies.com/sites/default/files/2023-02/2600.pdf 404 live, fragment kept https://ffic.myflfamilies.com/manual/2600.pdf 1
https://www.myflfamilies.com/sites/default/files/2025-05/Appendix%20A-12%20-%20State%20Funded%20Programs%20Eligibility%20Standards.pdf 404 Wayback https://web.archive.org/web/20251030232040/https://www.myflfamilies.com/sites/default/files/2025-05/Appendix%20A-12%20-%20State%20Funded%20Programs%20Eligibility%20Standards.pdf 16
https://www.senate.mo.gov/25info/bts_web/bill.aspx?SessionType=R&BillID=4957943 404 live https://www.senate.mo.gov/BillTracking/Bills/BillInformation?year=2025&billid=4957943 1

Left unchanged

These URLs are not changed here; each needs more than a URL swap or turned out not to be dead.

  • https://www.myflfamilies.com/sites/default/files/2024-06/Appendix%20A-12.pdf: unresolved (The cited edition, 'effective 2024-07-01' per the repo titles (columns Jan 2023, Jan 2024, Jul 2024), cannot be found anywhere. (1) Live: DCF's ESS manual page )

Changes after review

The independent review (GPT-6.1 Sol, tier hard) of 6959175b3a requested changes. Commit 11f538dbeb addresses them. Each new page was checked with pdftotext.

  • Blocker, South Carolina: sc/tax/income/rates.yaml cited an eFile capture that shows six 2022 brackets with a 7% top rate, against the file's three brackets and 6.5%. It now cites the issuing agency's SC4972 (2024) Tax Rate Schedule, page 2: 0% to ,460, 3% to ,330, 6.2% above, matching the 2024 values. The "third party source" comment is gone, and the guard replacement points at SC4972. The other years keep their existing official citations.
  • Blocker, Wisconsin (9 files): the eFile capture is dated January 2025 and has no 2025 values. Each file now adds official sources by year:
    • 2022: DOR's Tax Exemption Devices 2023–25, page 13, with ,790 / ,990–,240 / 12% and 3.54% /bin/zsh–,760 for single filers.
    • 2024: Tax Exemption Devices 2025–27, page 9, with ,230 / ,070–,320 and brackets ,320 / ,640 / ,310.
    • 2023 and 2025, deduction files only: the 2023 Form 1-ES instructions, page 2 (,760 less 12% over ,400) and the 2025 Form 1-ES instructions, page 3 (,560 / ,550, ,110 / ,210, ,930 / ,390, ,520 with the ,210 crossover).
    • The rate files already cite the 2023–2026 Form 1 and 1-ES instructions.
    • 2021: no live official page states the phase-out starts. The 2021 Form 1-ES instructions are 404, and DOR publishes the Tax Exemption Devices report only every other year (21sumrpt covers 2020). So the eFile capture stays for 2021 only, titled as a third-party archive.
  • Should-fix, Michigan exception: the efile.com/pdf/5988.pdf row is a deliberate exception to "Wayback of the same URL". CDX has no captures of the eFile PDF, and michigan.gov no longer serves the 2022 MI-1040CR-7 booklet (IIT/TY2022/BOOK_MI-1040CR-7.pdf is 404). The replacement is therefore an archive of the issuing agency's own booklet. Its page numbers differ from eFile's copy (11→9, 13→11). The reviewer confirmed that pages 9 and 11 carry the 90%/50% rates, the …,433 allowances, the exemption step and the Table B caps.
  • Should-fix, South Dakota period: the capture is the same dead URL (the June 2025 Child Care Assistance manual), and its values match. The parameters' 2026-08-01 start date was already on main before this PR, which changes only the URL. Whether a later edition changed the rule is a policy-period question for a separate follow-up, not a reference swap.
  • Nits: the Summary now lists the formatting exceptions and drops the reference count.

Verification

  • pytest policyengine_us/tests/code_health/test_known_dead_reference_urls.py passes locally, including the new entries.
  • ruff format --check . (locked ruff 0.15.5) passes.
  • No parameter values, dates, titles or labels change.

axiom: n/a: reference-only

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 3, 2026 14:48
Repoints dead reference URLs to the same document at a live official URL, an official page with the same content, or a pinned Wayback capture of the dead URL. Only URLs change. Lists the dead prefixes in known_dead_reference_urls/southeast.toml for the offline guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These mappings were verified earlier but skipped by the first build because their mechanical liveness check had not run yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.05%. Comparing base (909176a) to head (50546b4).
⚠️ Report is 177 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##              main    #9773       +/-   ##
============================================
- Coverage   100.00%   61.05%   -38.95%     
============================================
  Files            4       34       +30     
  Lines           76      606      +530     
  Branches         2        4        +2     
============================================
+ Hits            76      370      +294     
- Misses           0      236      +236     
Flag Coverage Δ
unittests 61.05% <100.00%> (-38.95%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MaxGhenis and others added 3 commits October 3, 2026 22:11
Review found the archived eFile South Carolina page shows six 2022
brackets with a 7% top rate, which contradicts the parameter, and that
the archived eFile Wisconsin page stops at 2024 while nine Wisconsin
files carry 2025 values.

- South Carolina rates cite the issuing agency's 2024 SC4972 rate
  schedule (page 2) instead of the eFile capture.
- The Wisconsin standard deduction, phase-out and rate files add DOR's
  Tax Exemption Devices reports (2022 and 2024 tax years) and, for the
  deduction files, the 2023 and 2025 Form 1-ES instructions. The eFile
  capture stays only for 2021, labelled as a third-party archive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	policyengine_us/parameters/gov/states/wi/tax/income/deductions/standard/phase_out/head_of_household.yaml
#	policyengine_us/parameters/gov/states/wi/tax/income/deductions/standard/phase_out/joint.yaml
#	policyengine_us/parameters/gov/states/wi/tax/income/deductions/standard/phase_out/separate.yaml
#	policyengine_us/parameters/gov/states/wi/tax/income/deductions/standard/phase_out/single.yaml
The merge of main took this branch's side of each conflicted file, which put
back URLs main had already replaced. Restore main's replacements; the branch's
own reference changes in these files stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit da4cda0 into main Oct 5, 2026
37 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the US + core hub at head 50546b41042ab6a9b5c78496059da0401da3f695 (squash, --admin --match-head-commit). The only block was the required approving review, which no one else can give on Max's PRs.

Gates:

  • Earlier review: an independent review approved the old head 11f538dbeb.
  • Main merge: main then took Separate concatenated URLs in Variable reference and documentation strings #9786, which adds commas between concatenated reference URLs and a guard against them. The hub merged main into this branch and re-applied the comma fix. The hub also added 50546b4. In four Wisconsin phase-out files, taking this branch's side of a conflict had put back the statute URL main had already moved to /2023/; that commit restores main's URL.
  • Delta review: an independent Opus 5.5 batched delta review approved this exact head (~/reviews/us-hub/deadurl/batch-delta-review-r3.md). It confirmed:
    • the same multiset of URL changes as the approved head;
    • the history is the old head plus the merge of main;
    • no non-attribute AST change in any touched .py file;
    • none of main's edits is lost in any conflicted file;
    • both URL guard tests pass.
  • CI: gh pr checks exits 0, with every check run on 50546b41042ab6a9b5c78496059da0401da3f695 successful.
  • State: MERGEABLE against current main, not a draft, no CHANGES_REQUESTED review.
  • Impact: reference URLs only, with no formula or value change.

@MaxGhenis
MaxGhenis deleted the dead-urls-southeast branch October 5, 2026 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant