[rusackas]

SUMMARY

Country Map GeoJSON Generator.ipynb special-cased ~26 countries in one growing notebook, mixing shared helper functions, per-country correction logic, and inline .plot() previews. #45602 fixed the worst symptom (the whole file diffing as binary), but a 30-section notebook was still awkward to review a single-country change in, and painful to resolve real conflicts in.

This replaces it entirely with per-country Python scripts, a few shared modules, and an orchestrator CLI (build_maps.py). The notebook is deleted - npm run update-maps and the "Adding a New Country" docs now point at the script pipeline instead.

Layout

superset-frontend/plugins/plugin-chart-country-map/scripts/
  build_maps.py                 # orchestrator CLI
  maps/
    catalog.py                   # the ~214-country plain catalog + aliases
    assemble.py                   # combines catalog + module overrides, purge logic
    context.py, data.py            # Natural Earth loading
    helpers.py                      # shared geometry helpers
    output.py                        # simplify + save_geojson
    ts_writer.py                      # writes countries.ts + docs/data/countries.json
    preview.py                         # --preview: thumbnails + contact sheet
    countries/                          # one module per special-cased country (23 total)

Validation

This is the part that matters most, given it touches shipped map data. Two independent checks:

1. Against master. Ran the full pipeline against live Natural Earth + every pinned/live community source. 197 of 201 src/countries/*.geojson files (one, madagascar.geojson, is a separate story below) matched what's currently committed byte-for-byte.
2. Against the real notebook. Patched a copy of the actual notebook - only to skip interactive plotting and reuse the same cached data, no logic changes - and ran it end to end with nbconvert. Diffed its real output directly against this script pipeline's output: all 201 comparable GeoJSON files, countries.ts, and docs/data/countries.json came out byte-for-byte identical.

That second check is the important one: it proves the 4 files that don't match committed master aren't bugs in this conversion - they're places where upstream Natural Earth data or the notebook's own current logic had already drifted from what's committed, before this PR existed. Running the unmodified notebook today reproduces the exact same output this script pipeline does.

What actually changes in this PR (map data)

• russia.geojson: Karelia and Murmansk now use their 1:10m boundaries instead of 1:50m, per the existing NE#929 workaround - only those two regions' geometry changes (confirmed feature-by-feature).
• south_africa.geojson: Western Cape's geometry shifts by a handful of vertices - upstream Natural Earth coastline refinement. The Prince Edward Islands reposition still applies correctly; the other 8 provinces are byte-identical.
• philippines.geojson (fixed, see below): now 83 real provinces instead of raw Natural Earth's 118 mixed provinces/highly-urbanized-cities.
• madagascar.geojson (re-sourced, see below): same 23 regions, now from WHO's current endpoint instead of a dead one.

(Two earlier revisions of this PR also touched iran.geojson and uk.geojson - both have since been corrected back to match what's currently shipped; see their commit messages for why.)

I spot-checked all of the above visually (before/after renders) - no missing or distorted regions. Please still look at the rendered diffs before merging, especially south_africa.geojson since it reflects upstream data this codebase doesn't control.

Two bugs found along the way - both now fixed

• countries.ts/docs/data/countries.json: the notebook's current countryOptions cell was missing a case for italy_regions_and_autonomous_provinces, even though the nicer "Italy (regions and autonomous provinces)" label is what's shipping today. Restored in ts_writer.py (see its own commit) since it's a pure regression with no upside.
• philippines.geojson: the notebook fetched a community source with the Philippines' 83 provinces already corrected (Maguindanao split into del Norte/del Sur, the Davao split, Compostela Valley renamed to Davao de Oro) and even renamed its columns to match - but never put the result anywhere the final output dict could pick it up. The shipped file was still raw, outdated Natural Earth ADM2 data, complete with stray highly-urbanized-city polygons (Manila, Quezon City, Caloocan, etc. as their own "provinces"). philippines.py now returns that corrected source as the philippines entry.
• madagascar.geojson: this one wasn't a bug in the notebook, but it surfaced while regenerating - the WHO AFRO GDB service this module queried (Hosted/AFROGDB_PUBLIC_DATA) has been retired on WHO's end and now 404s, independent of anything in this repo (the unmodified notebook hits the exact same error). Repointed at WHO's current public replacement (Hosted/AFROGDB_published_boundary, layer latest_boundary_adm1), which exposes the same fields and returns the same 23 regions - confirmed name-for-name against what was previously committed. Regenerated; boundaries are visually unchanged.

Deliberate behavior changes from the notebook (all documented in-code)

• latvia.py, india.py, kenya.py: external-source fetch failures now raise instead of silently falling back to outdated Natural Earth boundaries (kenya.py's notebook cell already did this; latvia/india's didn't - applied for consistency).

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A (no UI surface; see Validation above for the before/after render check on the 4 changed countries)

TESTING INSTRUCTIONS

cd superset-frontend/plugins/plugin-chart-country-map/scripts
pip install geopandas shapely matplotlib requests
python3 build_maps.py --preview --out-dir /tmp/maps-out
# compare /tmp/maps-out/*.geojson against src/countries/*.geojson

Or just npm run update-maps for the real thing.

ADDITIONAL INFORMATION

• Has associated issue:
• Required feature flags:
• Changes UI
• Includes DB Migration (follow approval process in SIP-59)
  • Migration is atomic, supports rollback & is backwards-compatible
  • Confirm DB migration upgrade and downgrade tested
  • Runtime estimates and downtime expectations provided
• Introduces new feature or API
• Removes existing feature or API

🤖 Generated with Claude Code

[netlify]

✅ Deploy Preview for superset-docs-preview ready!

Name	Link
🔨 Latest commit	a0947c4
🔍 Latest deploy log	https://app.netlify.com/projects/superset-docs-preview/deploys/6ae526db38428cc0f06639a8
😎 Deploy Preview	https://deploy-preview-45607--superset-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.74%. Comparing base (7f2b04f) to head (dd2e9d5).
⚠️ Report is 17 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #45607   +/-   ##
=======================================
  Coverage   84.73%   84.74%           
=======================================
  Files        3069     3069           
  Lines      205819   205896   +77     
  Branches    48408    48429   +21     
=======================================
+ Hits       174403   174483   +80     
+ Misses      28122    28115    -7     
- Partials     3294     3298    +4     

Flag	Coverage Δ
hive	34.49% <ø> (-0.02%)	⬇️
javascript	81.00% <ø> (+<0.01%)	⬆️
mysql	52.60% <ø> (?)
postgres	52.61% <ø> (+0.01%)	⬆️
presto	36.22% <ø> (-0.02%)	⬇️
python	87.76% <ø> (+<0.01%)	⬆️
sqlite	52.37% <ø> (-0.01%)	⬇️
unit	82.53% <ø> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[codeant-ai-for-open-source]

CodeAnt PR Risk: Low Risk

• The PR appears safe to merge; subset builds now write only selected modules, preventing unselected maps from being overwritten.

Assessed commit: 9d7f199b4876

[sadpandajoe]

Two regressions fixed during this review, the --countries subset rewriting corrected maps from raw Natural Earth data and the bare-filename makedirs crash, have no test, and nothing in CI runs these scripts: the existing Jest tests only read the committed artifacts. Reverting include_plain=selected is None or dropping an or "." fallback would pass every check. Could a small pytest run main() against fixtures, once with --countries france over an existing corrected usa.geojson (asserting it, countries.ts and the docs JSON are untouched), and once in a temp cwd with --countries-ts-path countries.ts, and be wired into an existing Python CI job?