Keep the ZCTA geography current, and let it reach installed clients

[john-zither]

Your checklist for this pull request

• Pulling a feature branch (feature/geography-build) against power-source.
• Tests pass with coverage up: 456 passed (15 new, covering the Gazetteer build, the subdivision assignment, the plausibility floor, and geography's place in the update set), ruff check clean. The 10 failures are pre-existing on power-source (pandas 2.x with numpy 2.5) and are fixed by Stop constructing Timedeltas the way numpy 2.5 deprecates #106; they are unrelated to this change.
• CHANGELOG.md updated under the "Development" section.
• Code style: ruff.
• New functions carry numpy-style docstrings; CONTRIBUTING.md documents the build step.
• All commits carry the DCO Signed-off-by trailer.
• Parts of this change were drafted with an AI assistant; every commit records it in a Co-Authored-By trailer. The tool's terms impose no conditions inconsistent with the project's licence or the Open Source Definition, and the output does not incorporate third-party code.

Description

The self-updating registry works as designed. This changes one premise it was scoped on — registry/update.py's module docstring:

Static content (geography packs, archive station lists) never changes and is not part of the update.

Geography does change. Census redrew the ZCTA boundaries for 2020 and republishes the Gazetteer annually. The packaged place table is still the 2010 definition, built from cb_2016_us_zcta510_500k.zip — zcta510 is the vintage; GENZ2016 is only the release year — and measured against the current Gazetteer it cannot place 851 ZCTAs that exist today while still carrying 204 that have been retired.

Two consequences followed from the premise rather than from any defect. Geography was excluded from UPDATABLE, so the refresh mechanism never reached it however often it ran. And the download scripts were deleted in 2548283, so there was no way to build a replacement pack even if something had been willing to install one — _migrate_places copies rows forward indefinitely. geography_us.db has no meta table either, so the staleness was not observable.

This reuses the existing update path unchanged — no second maybe_update, no parallel CDN channel, no additional commit marker or staleness clock. The update-side diff is eleven lines: add the geography packs to UPDATABLE so the mechanism carries a fourth file, mark them OPTIONAL_IN_PACK, and correct that docstring sentence.

The substantive half is eeweather/build/geography.py, which rebuilds place from the annual Census Gazetteer ZCTA file plus cb_YYYY_us_state_500k.zip, stamped with a vintage and wired into the existing refresh workflow. The two halves are one change because neither is useful alone: adding geography to UPDATABLE installs nothing without a build step, and a build step produces a pack nothing installs.

Smaller than it sounds, for two reasons: eeweather stores no ZCTA geometry — a ZCTA is a point, a state, and four zone assignments — so the rebuild does not need the cartographic boundary files, which Census stopped publishing after GENZ2020 anyway. The one genuinely new piece is deriving subdivision by point-in-polygon against the state layer, since the Gazetteer has no state column.

Details worth review:

• OPTIONAL_IN_PACK exists because the two cadences differ: geography moves once a year while the station registry moves continuously, so a weekly published pack should not have to re-upload an unchanged 8 MB of geometry, and a pack without one should refresh the rest rather than abort.
• The point-in-polygon assignment carries a tolerance because the 500k boundaries are generalized — an island ZCTA's internal point can sit just off its own coastline (04108 Peaks Island, 11957 Orient). Roughly 5 km at these latitudes, far too small to reach another state.
• A plausibility floor rejects a rebuild that loses a large share of the packaged place count, on the grounds that it is a bad download rather than a real change. The expected move is 33,144 → 33,791.
• Provenance records registry_vintage, the refreshed_at stamp of the registry a request resolved against. It belongs here rather than on its own: putting geography in the updatable set is what makes that vintage vary, and with geography in the set it is stamped through the same key as the station registry rather than under a name only this module knows.

[travis-recurve]

Good work, and the docstring evidence is exactly right. Three things before merge.

Vintage retention (blocking). build_places deletes the ZCTA rows and place is keyed
(kind, code), so the pack holds one vintage. Those rows are the only copy of the geography
past results were computed against, and they came from a cartographic boundary file rather
than the Gazetteer, so a rebuild will not reproduce them. This PR also adds geography to the
auto-update set, so the vintage now rolls on installed clients and a 2015 analysis silently
returns different station matches, which undercuts the registry_vintage field added here.

We also cannot assume a caller's ZIP codes are current or historical, and one file can mix
both, so neither period inference nor a caller declaration is reliable alone. Retaining both
vintages lets the library resolve a code in each and report whether the choice matters:
silent where the points agree too closely to change the ranked station, warned where they
diverge, error where a code exists in only one. Please key place and place_zone on
vintage and drop the delete. The selection rule can follow.

pyshp. The shapefile is read only to resolve each ZCTA's state. Census publishes
ZCTA-to-county relationship files as plain text, and county FIPS carries the state FIPS in
its first two digits, which drops the dependency and is more accurate at state borders.
Those are per decennial census, though ZCTA definitions only change decennially anyway. If
you already rejected them, say why and I will take the dependency.

Workflow. --geography ${{ inputs.year }} interpolates a dispatch input into run: in
a job holding contents: write, and should go through env:. The obvious fix breaks the
scheduled run, where the input is empty:

--geography      -> geography=True  -> build_places
--geography ""   -> geography=''    -> refresh()

Unquoted hits the const=True default. Quoted is falsy, so main() falls through and
rebuilds the station registry instead. Fix both together, with the branch made explicit.

Minor: GEOGRAPHY_FILENAMES globs at import time, so an empty registry directory drops
geography from the update set. without_subdivision is counted but never gated, unlike the
ZCTA count above it.

The branch was rebased from our side after #105 merged and briefly closed when the base
branch was deleted. Your commits are unchanged, but you will need
git fetch && git reset --hard origin/feature/geography-build.

[john-zither]

Rebased and pushed. All three items plus the two minors:

Vintage retention. place and place_zone now have a vintage column and are keyed on it. Both deletes in build_places are gone, so a rebuild adds a vintage next to the old one instead of replacing it. The migration stamps the existing rows as vintage 2016, since that's the GENZ2016 release the docstring traces them to, and it gives newest-resolution a number to sort on. get_place reads the newest vintage per code and filters the place_zone sub-select to that vintage — otherwise dict(fetchall()) would collapse system keys across vintages. get_zcta_ids dedupes to the newest codes.

The selection rule (quiet when the points agree, warn when they diverge, error when a code only exists in one vintage) comes next, on top of this. It's all in get_place, which is the only reader on the code→station path.

pyshp — dropped, switched to the relationship file. I hadn't rejected it; the shapefile was just the only source with a state column. Now I pull the ZCTA-to-county relationship file instead: state FIPS is the first two digits of the county GEOID, and a small static table maps FIPS to the USPS abbreviation so subdivision stays two letters. Where a ZCTA spans counties, I take the dominant one by the file's overlap field. Dropping the shapefile let me delete _import_shapefile, STATE_URL, the shapely STRtree/prep code, the offshore NEAREST_TOLERANCE fallback, and the [build] pyshp extra; resolve_vintage no longer checks for the state layer. Per-decennial is fine since ZCTA definitions only change decennially.

Workflow. The input goes through env: YEAR: now, and the run is python -m eeweather.build --geography ${YEAR:+$YEAR}. Empty gives the bare flag (the scheduled rebuild-newest case), set gives --geography 2023. No more interpolating the input straight into run:, and both cases do the right thing.

Minors. GEOGRAPHY_FILENAMES comes from a fixed pack name instead of globbing at import, so an empty registry dir doesn't drop geography from the update set. And without_subdivision is gated to raise past a small floor — with the relationship file it should be ~0.