Skip to content

Add aquifers to the explorer - #26

Open
prayaslashkari wants to merge 10 commits into
feat/material-dropdown-groupingfrom
exp/aquifers-schema
Open

Add aquifers to the explorer#26
prayaslashkari wants to merge 10 commits into
feat/material-dropdown-groupingfrom
exp/aquifers-schema

Conversation

@prayaslashkari

@prayaslashkari prayaslashkari commented Aug 17, 2026

Copy link
Copy Markdown
Collaborator

1. What?

Adds aquifers to the explorer. Aquifers show up in two places:

  • As a new entity Type in the analysis-question builder, used as a spatial filter. You can ask things like "wells near sand and gravel aquifers" or "samples near aquifers". The aquifer is the spatial container; the matched features (wells, samples, facilities) are what render.
  • As a toggleable map overlay ("Aquifers (extent)") that draws real aquifer boundaries, colored by kind (surficial vs bedrock), independent of any query.

Data coverage is Maine and Illinois only (that is all the KG has loaded): 8,441 aquifers. Well-based and aquifer-based questions are rich in both states. Contamination-sample questions are rich in Maine and thin in Illinois, because Illinois has few samples in the KG (about 66 samples across 33 sample points). That is a data-ingestion gap, not an app issue.

2. Involved SPARQL queries

Key facts about the data model that shaped the approach:

  • The aquifer class is gwml2:GW_Aquifer where gwml2: is http://gwml2.org/def/gwml2# (not the OGC opengis.net namespace the UML diagrams suggest).
  • Aquifers have no name and no geometry in the KG. They carry an id (il-isgs:ilSawAqId), a type (saw_water:aquiferType), and links to S2 level-13 cells.
  • There is no direct well-to-aquifer or sample-to-aquifer link. The only association is spatial, through shared S2 cells, which is exactly how the existing query engine already relates entities.

Filter binding (added to the fused query engine): an aquifer is bound inside an S2 cell like wells are, plus an optional type filter.

?s2 spatial:connectedTo ?aquifer .
?aquifer rdf:type gwml2:GW_Aquifer .
# optional, expanded from the UI's Sand & gravel / Bedrock choice:
?aquifer saw_water:aquiferType ?aqType .
VALUES ?aqType { "sand and gravel" "coarse-grain_materials" "sand_gravel" }

Hydrate (aquifer detail for the results list): id and type only, no S2 cell join since one aquifer can touch tens of thousands of cells.

SELECT DISTINCT ?aquifer ?aquiferType WHERE {
  VALUES ?aquifer { ... }
  ?aquifer rdf:type gwml2:GW_Aquifer .
  OPTIONAL { ?aquifer saw_water:aquiferType ?aquiferType . }
}

Both verified live. The controlled vocabulary is inconsistent across states (Maine "sand and gravel" vs Illinois "coarse-grain_materials"); the UI collapses these into two choices and the query builder expands them back. More verified queries are in docs/aquifer-queries.md.

Endpoints: the filter/match runs on federation (it joins samples in sawgraph, aquifers in hydrologykg, facilities in fiokg); aquifer detail uses hydrologykg.

3. How are we rendering aquifers?

Aquifers have no geometry in the KG, so the map overlay is a precomputed static GeoJSON built by scripts/precompute_aquifers.py, pulling real polygons per state from each state's geological survey ArcGIS service:

  • Maine (MGS): authoritative 1:24,000 aquifer polygons from the Maine Geological Survey, joined to the KG by AQUIFERID (MGS-Aquifer.{id:05d}), verified 100% coverage of the 4,972 KG Maine aquifers. Carries yield (SYMBOLOGY), shown in the popup.
  • Illinois (ISGS): the ILWATER/Aquifers service, whose per-layer counts match the KG 1:1 (layer 1 = 3,367 coarse-grained, layer 0 = 88 sand and gravel, layer 3 = 14 bedrock). The KG's CM/SG/BR id numbers are stale shapefile indices that no longer match the service OBJECTIDs, so we do not join by id. The overlay is spatial context and query matching still uses the KG's S2 cells, so id linkage is not needed.

Both states now render real, smooth boundaries. Output is 8,441 features (4,972 MGS + 3,469 ISGS), about 3.8MB gzipped. An earlier S2-cell fallback was used before these real sources were found; it has been removed (dropped the s2sphere dependency).

The rendering layer is a self-loading Leaflet overlay: lazy-fetched on first toggle, canvas renderer for the polygon count, colored by kind, click for type, yield, and id. It sits under the sample and facility layers and appears in the layer panel even with no query loaded. The GeoJSON artifact is gitignored and regenerable with python3 scripts/precompute_aquifers.py.

Data sources

Verification

  • tsc clean, npm run build green.
  • Filter binding and hydrate verified live; Maine join is 100%; Illinois per-layer counts match the KG exactly.
  • Precompute runs end to end; 8,441 features, all real geometry.
  • Dev server serves the app and the GeoJSON asset.
  • Not visually confirmed in a real browser as part of this PR beyond the screenshots shared during review.

Notes and limitations

  • Data is Maine and Illinois only.

  • Aquifers work best on the "C" side of a query (the container you relate to); on the "A" side they have no geometry to plot.

  • Matching uses S2 cells (about 1km), while the overlay draws precise polygons, so a "near, 0 miles" match can sit up to about 1km outside the drawn boundary. The match is correct; the gap is the S2 cell size.

image
  • File size is about 3.8MB gzipped; the SIMPLIFY tolerance in the script can be raised to shrink it.
  • 6 pre-existing react-hooks/refs lint errors in LayerPanel.tsx are unrelated to this change and were left as is.

The GET_FLOWLINE_GEOMETRIES step re-derived its own anchor set and applied
whichever region it happened to be handed, which is a guess the other two
layers never make. Both guesses produce a visibly wrong map:

  - anchor region (previous behaviour): the downstream prebuilts and the
    editor both put the region on the *target* block, so the facility side
    came through unfiltered. The trace started from every matching facility
    in the country and the stream layer covered the USA — blue flowlines
    across the Ohio basin for a Maine question.

  - target region (first attempt at a fix): drops anchors sitting outside
    the region that genuinely drain into it. For "samples in Maine
    downstream of Solid Waste Landfills", 16 of the 39 contributing
    landfills are in New Hampshire, so 41% of the traces vanished and Maine
    sample points along the Androscoggin were left with no stream beneath
    them.

FIND_ANCHOR_IRIS has already resolved the correct anchors by this point —
they passed both the anchor-side and target-side filters — so bind those
IRIs directly and drop the region clause. The layer is now consistent with
the facility layer by construction.

Verified against the live federation endpoint for the question above:
1,446 flowlines, bbox lon -71.30..-67.72 / lat 42.97..46.88, zero vertices
west of -80, Rumford and Jay coverage restored (249 and 203 vertices, both
previously empty), 0.84s. Reusing the resolved IRIs is faster than the
re-derivation it replaces. Empty anchor sets emit `VALUES ?x { }`, which the
endpoint accepts and returns no rows for.
fix(map): trace stream layer from resolved anchors, not a guessed region
Aquifers are now a first-class entity in the analysis-question builder and a
toggleable map layer, for Maine and Illinois (8,441 aquifers; ME 4,972, IL 3,469).

- Filter: aquifers join the fused S2-cell query engine as an entity Type (the
  spatial container you relate samples/wells/facilities to), with a
  Sand & gravel / Bedrock type filter. Endpoint routing, hydrate, and
  natural-language phrasing wired through.
- Overlay: real aquifer boundaries as a static, toggleable layer. Geometry is
  precomputed by scripts/precompute_aquifers.py from each state's authoritative
  ArcGIS source (Maine Geological Survey, Illinois State Geological Survey) and
  shipped as a content-hashed Vite asset so it is cache-safe. Colored by kind
  with a WCAG 3:1 stroke; popup shows type and yield.
- Docs: gwml2 namespace + counts in SCHEMA.md, verified queries in
  docs/aquifer-queries.md, aquifer colors added to the contrast audit.

Verified live against the FRINK federation/hydrologykg endpoints and both
ArcGIS services.
Per tester feedback: the extent overlay drew every aquifer regardless of the
query. Now when a query matches aquifers, the overlay shows only those (the
aquifers near the sample points), matching how the streams layer behaves; with
no aquifer query it still shows all.

- useMapLayers collects matched aquifer IRIs (already in result.data, just
  discarded before) onto MapLayerData
- AquiferBoundaryLayer filters features to the matched set, remounting the
  GeoJSON on change
- Maine features (KG-IRI ids) scope exactly; Illinois features (synthetic isgs
  ids) can't be matched to KG IRIs yet, so IL stays unscoped (follow-up)
@prayaslashkari

Copy link
Copy Markdown
Collaborator Author

FRINK retired https://frink.apps.renci.org/<kg>/sparql; all five knowledge
graphs now return 503 there and are served from https://apps.okn.us/<kg>/sparql
instead. Reported by David Kedrowski, who hit the same break in his notebooks.

Verified all five endpoints live on the new host (200, CORS *), and replayed
the real pipeline through planPipeline/executePipeline against them: 7 of 8
prebuilt queries succeed and all discovery queries return live data rather
than falling back to hardcoded constants.

The Indiana downstream prebuilt still fails, but with a QLever memory-limit
error ("Tried to allocate 819.2 MB, but only 743.3 MB were available"), not a
routing failure. Tracked separately.
fix(endpoints): move SPARQL endpoints from FRINK to apps.okn.us
The app deploys on Railway from main, but the repo still carried a Render
blueprint from the publish workflow and no mention of Railway anywhere. The
stale file was actively misleading about where production runs.

Removes render.yaml, documents the real deploy target in the README, and adds
the W35 changelog covering this and the FRINK endpoint migration.
chore: drop stale render.yaml, document Railway as the deploy target
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant