Skip to content

Replace dead Northeast and California local reference URLs - #9766

Merged
MaxGhenis merged 3 commits into
mainfrom
dead-urls-northeast_ca
Oct 4, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
dead-urls-northeast_ca

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Part of the repo-wide audit of dead reference URLs outside tax.ny.gov (follow-up to #9711 and #9730). This PR covers Vermont, Connecticut, New York Labor, Maine and California county and city sources (Vermont Tax and DCF, CT General Assembly and OEC, NY DOL, Maine Legislature, San Francisco, Santa Clara, Contra Costa, Alameda and San Diego counties, University of Hawaii).

  • 22 dead URLs, cited by 63 references, now point to a live copy of the same document, a live official page with the same content, or a pinned Wayback capture of the dead URL with the same #page.
  • Only references change. A script checked that every changed line under parameters/, variables/ and docs/ differs from the old line only inside URLs. The one exception is the max_au_size.yaml reference title, changed after review to name CDSS ACL 23-74.
  • policyengine_us/tests/code_health/known_dead_reference_urls/northeast_ca.toml lists 22 of the dead prefixes, so the offline guard from Replace dead NY tax department reference URLs #9711 fails if they return.

How each URL was checked

  1. Dead: GET with curl -sL -A "Mozilla/5.0" on 2026-09-30, rechecked with a Chrome user agent. A 403, 429 or bot challenge counted as unknown, not dead, and was rechecked inside a real browser page; URLs still behind a bot wall were left unchanged. A 200 that landed on a home, search, login or not-found page counted as dead.
  2. Replacement: one research agent per batch looked for the same document on the agency's current site, then for an official page stating the same content for the same period, then for a pinned Wayback capture. Each mapping records the quoted content it checked.
  3. Independent verification: a second agent re-checked every live replacement and a sample of the Wayback ones against the values the citing file uses (pdftotext on the cited #page for PDFs). Disputed items went to an adjudicator under fixed rules: the agency's own site, then another official publisher, then a Wayback capture of the dead URL. Third-party mirrors (taxformfinder.org, NBER and the like) were not accepted for official citations.
  4. Mechanical: every replacement URL returned 200 with the expected content type. Wayback URLs use 14-digit timestamps and replay the same original URL, and every #page exists in its PDF. A few replacements behind bot walls were confirmed in a browser instead.

Changes

Dead URL Dead because Kind New target Refs
https://dcf.vermont.gov/benefits/ccfap/act76-faqs 404 live, fragment updated https://outside.vermont.gov/dept/DCF/Shared%20Documents/CDD/Act76/ACT-76-FAQs.pdf 1
https://dol.ny.gov/system/files/documents/2025/01/p832-how-your-weekly-ui-benefits-are-calculated-1-25.pdf 404 live, fragment kept https://dol.ny.gov/system/files/documents/2025/02/p832s-how-your-weekly-ui-benefits-are-calculated-1-25es-us.pdf 1
https://ehsd.org/wp-content/uploads/2024/08/GA-Brochure_ENGLISH_July2024_FA_Digital.pdf 404 live, fragment kept https://ehsd.org/wp-content/uploads/2025/05/GA-Brochure_ENGLISH_Apr2025_FA_Digital.pdf 3
https://hhsaprogramguides.sandiegocounty.gov/CalWORKS/44-300/CalWORKs_Payment_Standards/G_CalWORKs_Payment_Standards.pdf redirects live official letter (after review) https://www.cdss.ca.gov/Portals/9/Additional-Resources/Letters-and-Notices/ACLs/2023/23-74.pdf#page=6 1
https://info.healthconnect.vermont.gov/sites/hcexchange/files/documents/2026-Eligibility-Income-Thresholds-for-QHP-Subsidies.pdf 404 live, fragment kept https://info.healthconnect.vermont.gov/sites/vhc/files/documents/2026-Eligibility-Income-Thresholds-for-QHP-Subsidies.pdf 1
https://legislature.maine.gov/legis/bills/getPDF.asp?paper=HP0892&item=3&session=131 200 'Webpage Error, live https://legislature.maine.gov/legis/bills/getPDF.asp?paper=HP0592&item=6&snum=131 1
https://portal.ct.gov/-/media/DRS/Forms/2024/Income/CT-1040-Instructions_1224.pdf redirects live, fragment kept https://portal.ct.gov/-/media/drs/forms/2024/income/2024-ct-1040-instructions_1224.pdf 1
https://portal.ct.gov/dss/archived-folder/health-programs-overview/husky-b 404 live https://portal.ct.gov/dss/knowledge-base/articles/healthcare-coverage/husky-b 1
https://sfmohcd.org/sites/default/files/Documents/MOH/BMR%20Ownership/2022%20AMI-IncomeLimits.pdf 404 live https://api.sf.gov/documents/48907/2022_AMI-IncomeLimits.pdf 2
https://stgenssa.sccgov.org/debs/program_handbooks/general_assistance/assets/09Income/Potential_Inc_Res.htm 404 live https://stgenssa.sccgov.org/debs/program_handbooks/general_assistance/assets/09Income/Potential_Income_Resources.htm 1
https://stgenssa.sccgov.org/debs/program_handbooks/general_assistance/assets/09Income/Verification_Income.htm 404 live https://stgenssa.sccgov.org/debs/program_handbooks/general_assistance/assets/09Income/Verification_of_Income.htm 2
https://tax.vermont.gov/document/2023-renter-credit-website-calculator 404 Wayback https://web.archive.org/web/20250526044955/https://tax.vermont.gov/document/2023-renter-credit-website-calculator 1
https://tax.vermont.gov/document/2024-renter-credit-website-calculator 404 live https://tax.vermont.gov/sites/tax/files/documents/2024%20Renter%20Credit%20Website%20Calculator.xlsx 1
https://tax.vermont.gov/sites/tax/files/documents/Income%20Booklet-2023.pdf 404 live, fragment kept https://tax.vermont.gov/sites/tax/files/documents/Income-Booklet-2023.pdf 1
https://www.alamedacountysocialservices.org/acssa-assets/PDF/GA-Policies/GA-Regulations.pdf redirects live, fragment kept https://socialservices.alamedacountyca.gov/acssa-assets/PDF/GA-Policies/GA-Regulations.pdf 14
https://www.cga.ct.gov/2022/act/pa/pdf/2022PA-00118-R00SB-00005-PA.pdf 404 live https://www.cga.ct.gov/2022/ACT/PA/PDF/2022PA-00118-R00HB-05506-PA.PDF 1
https://www.cga.ct.gov/2023/act/pa/pdf/2023PA-00031-R00HB-06680-PA.pdf 404 live https://www.cga.ct.gov/2023/ACT/PA/PDF/2023PA-00031-R00HB-06733-PA.PDF 1
https://www.ctoec.org/care-4-kids/c4k-policies/ 404 live, fragment updated https://www.ctoec.org/care-4-kids-regulations/ 1
https://www.ctoec.org/care-4-kids/c4k-providers/c4k-rates/ 404 live, fragment updated https://www.ctoec.org/care-4-kids-regulations/ 6
https://www.sfhsa.org/sites/default/files/media/document/2023-01/form_wfc_ty22_eng_code41.pdf 404 Wayback, fragment kept https://web.archive.org/web/20240630043054/https://www.sfhsa.org/sites/default/files/media/document/2023-01/form_wfc_ty22_eng_code41.pdf 1
https://www.sfhsa.org/sites/default/files/media/document/2024-01/form_wfc_english_1.26.24.pdf 404 live, fragment updated https://www.sf.gov/news--san-francisco-launches-free-tax-preparation-assistance-and-local-tax-credit-program-and 2
https://www.sfhsa.org/sites/default/files/media/document/2026-06/manual_caap_eligibility_6_9_2026_v2.pdf 404 live, fragment updated https://www.sfhsa.org/sites/default/files/media/document/2026-07/manual_caap_eligibility_7_6_2026.pdf 19

Left unchanged

These URLs are not changed here; each needs more than a URL swap or turned out not to be dead.

  • https://www.hawaii.edu/bridgetohope/downloads/UPDATED%20STANDARD%20OF%20ASSISTANCE%20AND%20DESK%20AID%20%20EFF.%2001-01-20%202.pdf: unresolved (No replacement verified. (1) Live search: the UH Bridge to Hope site was rebuilt in 2026 and its /downloads/ page now holds only monthly FTW forms. Web searches)

Changes after review

An independent review (GPT-6.1 Sol) requested one change, now made: the CalWORKs maximum assistance unit size (max_au_size.yaml, effective October 2023) now cites the live CDSS All County Letter 23-74, page 6 (effective October 1, 2023; both tables end at "10 or more"), instead of an archived San Diego County chart from October 2022. That reference's title is updated to name the letter; it is the only non-URL line this PR changes.

Verification

  • pytest policyengine_us/tests/code_health/test_known_dead_reference_urls.py passes locally, including the new entries.
  • ruff format --check . (locked ruff 0.15.5) passes.
  • No parameter values, dates, titles or labels change.

axiom: n/a: reference-only

🤖 Generated with Claude Code

Repoints dead reference URLs to the same document at a live official URL, an official page with the same content, or a pinned Wayback capture of the dead URL. Only URLs change. Lists the dead prefixes in known_dead_reference_urls/northeast_ca.toml for the offline guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.09%. Comparing base (909176a) to head (b7b9cbe).
⚠️ Report is 162 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##              main    #9766       +/-   ##
============================================
- Coverage   100.00%   69.09%   -30.91%     
============================================
  Files            4       26       +22     
  Lines           76      398      +322     
  Branches         2        0        -2     
============================================
+ Hits            76      275      +199     
- Misses           0      123      +123     
Flag Coverage Δ
unittests 69.09% <100.00%> (-30.91%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Cite the live CDSS All County Letter 23-74 (page 6, effective October 1,
2023, tables ending at "10 or more") for the CalWORKs maximum assistance
unit size instead of an archived county chart from October 2022, and retitle
that reference to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hand-off to the US + core hub (session 621e3c96-545d-4048-a211-43cf6d93cfe8).

  • Head: ba5834baa7a547bdf2ff7b058cd6016869fb8c9e. The independent delta review (GPT-6.1 Sol, tier hard) at this head is APPROVE: ~/reviews/dead-reference-urls-2026-09-30/outputs/delta-9766.md. Its one nit, a stale row in the PR body, is fixed.
  • Passing: every check except codecov/project. That check fails at 69.09% because Quick Feedback narrowed this broad change to gov/hud/integration.yaml but still measured the 26 changed variable files (run 37144819774, job 111267711769).
  • Pending: Leave untested changed sources out of Quick Feedback coverage #9778 (mine, review in progress) fixes the measurement. After it lands, update this branch (gh pr update-branch) so Quick Feedback reruns on a new merge SHA. The PR diff does not change.
  • Depends on: Leave untested changed sources out of Quick Feedback coverage #9778.
  • Land: squash-merge with --match-head-commit once gh pr checks exits 0. This PR is reference-only (axiom: n/a: reference-only).

@MaxGhenis
MaxGhenis merged commit d042e9c into main Oct 4, 2026
39 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the US + core hub (handed off by the dead-URL sweep session) on the merge gates:

  • Head: b7b9cbe. That is the reviewed head ba5834b plus a merge of main (picks up Fix read-only cache tests with Core 3.32.16 #9782 and Make Codecov's project status informational #9777). The PR diff is byte-identical.
  • CI: all check runs successful on this head (38 success, 1 skipped).
  • Reviews: the dead-URL session's delta review approved ba5834b (/reviews/dead-reference-urls-2026-09-30/outputs/delta-9766.md). The hub's GPT-6.1 Sol batched main-merge check approved this head (/reviews/us-hub/lander/batch3-mainmerge-review-r2.md).
  • Scope: reference URLs only (axiom: n/a).
  • Admin merge: used only for the required-review rule, at the reviewed head.

@MaxGhenis
MaxGhenis deleted the dead-urls-northeast_ca branch October 4, 2026 12:12
MaxGhenis added a commit that referenced this pull request Oct 6, 2026
* Skip Quick Feedback coverage when the test scope is narrowed

When a change maps to too many test targets, limit_test_paths narrows
the run to the directly changed tests. run_tests still passed every
changed .py file to coverage --include, so the uploaded report counted
lines in those files that the narrowed tests never exercise. On the
reference-only PRs #9764 and #9766 that sent codecov/project to 54% and
69% while codecov/patch was 100%.

A narrowed run now skips coverage, like the existing path for changes
with no Python source. Codecov then gets no partial report and posts
no project status, matching PRs that Quick Feedback defers entirely.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Leave only the untested changed sources out of coverage

Narrowing is one of two ways Quick Feedback drops a changed file's
tests; deferring slow directories (gov/ssa, household, reform, ...) is
the other, and it produced the same partial report. Replace the
narrowed-scope flag with a per-file rule: a changed source is measured
unless it maps to tests and none of them are in this run. Files with
no mapped tests keep today's behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant