feat: fort map-data consumer — gyms, stations & pokestops - with DNF filtering - #1228
feat: fort map-data consumer — gyms, stations & pokestops - with DNF filtering#1228jfberry wants to merge 62 commits into
Conversation
…wMap#1225) * fix(scanArea): prevent crash when area feature has no name/key Guard the scan area search filter against features missing a properties.key (which happens when a scan area polygon has no name set), instead of throwing TypeError: Cannot read properties of undefined (reading 'toLowerCase'). Also fixes a longstanding typo (geoJsonFilName / geoJsonFilname -> geoJsonFileName) in the multi-domain example config and docs. * fix: copilot comments --------- Co-authored-by: Mygod <contact-git@mygod.be>
Validating in a test environmentScope: this branch routes only gyms through Golbat ( 1. Golbat prereqs — deploy #385, curl -H "X-Golbat-Secret: $S" "$G/api/gym/available"
curl -H "X-Golbat-Secret: $S" -H 'Content-Type: application/json' -XPOST "$G/api/gym/scan" \
-d '{"min":{"latitude":LAT1,"longitude":LON1},"max":{"latitude":LAT2,"longitude":LON2},"limit":1000,"filters":[]}'
curl -H "X-Golbat-Secret: $S" "$G/api/gym/id/<gym-id>"2. ReactMap — 3. Config — add 4. Confirm the endpoint path (logs):
5. Functional — gym + raid markers render; a popup shows team/slots/defenders and raid boss + moves + CP; the filter drawer populates teams, tiers ( 6. Golden parity (endpoint vs SQL): 7. Fallback — stop Golbat or set 8. Area restrictions — as an area-restricted user, confirm gyms appear only within permitted areas (exercises the new client-side |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…st-layer availability fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dadba73 to
af62c30
Compare
… badge poison Review findings: - tier-override mode matches raid_level alone (curated boss/egg keys under-returned tier raids on endpoint sources) - b<display_type> keys move to the onlyEventStops gate (secondaryFilter's events branch consumes them, not the invasions branch) - endpoint rows resolve the quest layer as dual-capable (pure-endpoint ctx flags made effectiveQuestLayer resolve to 'both') - poison on onlyGymBadges only: onlyBadge defaults to 'all' for every user with the gymBadges perm and was silently disabling gym DNF entirely Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the all-gyms/ex/in-battle poison with real narrowing: every gym-layer display requires the team/slot match (hasGym = enabler && (team || slot)), so team_id/available_slots clauses mirroring finalTeams/finalSlots are a tight superset for all four enablers — ex/ar/in-battle narrowing stays residual. The standalone is_ar_scan_eligible clause is subsumed (ar-shown gyms also need the team match). Badge viewing still poisons. Power-up narrowing is gone for good — power-ups are no longer in the game. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
This is now in my production and I have verified the DNFs through manual testing |
|
Six fixed, one is a repeat that's already handled. Fixed (
|
|
The endpoint station path bypasses the SQL field projection, while successful empty availability snapshots leave stale filters indefinitely. The new refresh TTL is also unavailable through the repository's environment-variable interface. Full review comments:
|
…env TTL) Three ReactMap-side fixes. - [P1] Station endpoint no longer leaks total_stationed_pokemon / total_stationed_gmax to non-dynamax callers. The endpoint spreads the raw Golbat record, so those totals rode along regardless of permission; the SQL path selects them only under includeBattleData. finalizeEndpointStation now strips them when battle data is not included, so a GraphQL selection cannot read stationed/gmax counts without the dynamax perm. - [P2] Distinguish a total-source-failure availability refresh from a genuine empty snapshot. getAvailable returns null when every source failed (caller retains last-good); a successful empty result returns [] and clears the category, so a drawer whose last option disappears no longer shows stale filters forever. EventManager retains only on null; the /api/v1/available route guards its sorts with || []. - [P2] Map api.availableRefreshSeconds to API_AVAILABLE_REFRESH_SECONDS in custom-environment-variables.json so container deployments can override the refresh throttle, matching every other scalar api default. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
All three fixed in
Lint / |
|
Endpoint-backed scans can omit valid markers because they cap Golbat results before ReactMap's local gates, and explicit availability refreshes can reuse stale snapshots. The exported availability contract also diverges from the consumed Golbat payload. Full review comments:
|
Two ReactMap-side fixes (the third review item, scan-limit-before-residual, is the recurring point already handled by Golbat max_fort_results). - [P2] Forced availability refreshes now start a fresh combined-snapshot generation. force=true previously bypassed only EventManager's TTL, not the fortAvailable combined cache, so a config reload or PUT /api/v1/available within the window reused a stale snapshot (or cached failure). loadLocalContexts and the PUT route now call bustCombinedFortCache() before the forced batch; the concurrent fort refreshes still coalesce via the repopulated window. The combined-cache window now tracks api.availableRefreshSeconds instead of a fixed 30s, so sub-30s API_AVAILABLE_REFRESH_SECONDS values take effect for forts. - [P3] Align the exported availability types (server.d.ts) and the mapper JSDoc with the Golbat payload: AvailablePokestopInvasion gains slot2/slot3 fields and drops count; AvailablePokestopLure / AvailablePokestopShowcase drop count (quest keeps count, which Golbat does return). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The fort scan requests sent to Golbat passed limit: queryLimits.<type>, which Golbat applies as a hard traversal cap BEFORE ReactMap's local gates (filterRTree, freshness, secondaryFilter) run. A viewport with more forts than the display limit could therefore drop valid forts: rejected forts earlier in the scan consumed the cap, leaving valid ones past it unseen. Send limit: 0 instead, so Golbat traverses up to its own server-side max_fort_results backstop, and apply queryLimits.<type> as a display cap AFTER the local gates: - pokestop: already capped via secondaryFilter's resultLimit - gym: final.slice(0, queryLimits.gyms) - station: stations.slice(0, queryLimits.stations) This mirrors the SQL path, where .limit(queryLimits.<type>) runs after the WHERE clause, and relies on Golbat's max_fort_results as the true traversal bound. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
All three fixed. [P2] Apply the scan limit after local gates (
This mirrors the SQL path, where [P2] Bypass the snapshot cache on forced refreshes ( [P3] Align the availability types with Golbat's response ( |
|
Forced availability refreshes can retain data from a replaced source, and endpoint-backed station data loses or invents user-visible battle metadata. These are functional regressions despite the successful lint and build checks. Full review comments:
|
…attles)
Three ReactMap-side fixes from the latest review; none touch Golbat or
broaden its queries.
- Forced availability refreshes no longer reuse superseded in-flight work.
setAvailable's single-flight guard now applies only to non-forced calls; a
forced refresh (hot reload / PUT /api/v1/available / scheduled interval)
always starts a fresh refresh against the current Db, and a monotonic
per-category generation token gates the commit so an older in-flight refresh
can't commit or TTL-stamp its (old-manager) result after a reload. Non-forced
session-init stampedes still coalesce on the shared pending promise.
- Endpoint-backed station battles now carry an `updated` timestamp. Golbat's
scan battle has no per-battle `updated`, so it is populated from the station's
own `updated` (b.updated ?? apiStation.updated ?? null), mirroring the SQL
getFallbackStationBattle path so StationBattleTimer has a last_seen to render.
- Station availability no longer advertises a phantom boss. A battle whose boss
isn't known yet arrives with a null (Golbat) or 0 pokemon_id; mapStationAvailable
now guards the boss key with Number(pokemon_id) > 0 so it doesn't inject a
null-null/0-0 Pokémon into the filter catalog that no marker can match. The
tier key (j{level}) is still published while the boss is unknown.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
All three fixed in [P2] Forced availability refreshes no longer reuse superseded work ( [P2] Endpoint station battles carry [P2] Skip boss keys for unknown bosses ( |
|
Thanks. The unknown-boss guard is functionally correct, but I don’t think the other two findings are fully resolved. For The generation token fixes the immediate stale The |
|
Thanks for your observation about leaking sentinals, that's a poor api - I'll fix that in golbat and push the reactmap changes later; other comments are bang on, they'll be fixed at the same time |
|
Hello fellow human |
…aware refresh
Companion to Golbat's availability null contract and per-battle `updated`,
plus the availability-refresh lifecycle fixes from the review.
A2 (null availability): the availability mappers already withhold a boss key
via `> 0` guards, so a null pokemon_id flows through identically to the old 0.
JSDoc/types updated to number|null (station battle, raid, showcase focus +
type_id, invasion slots), and the incorrect station comment corrected — Golbat
now sends null, and Number(null) is 0 (not NaN), so `> 0` still withholds the
key while shipping the tier key. Quests unchanged.
B1 (per-battle updated): Golbat now exposes the real per-battle `updated` on
ApiStationBattleResult, so the station endpoint passes `b.updated` through
unchanged instead of borrowing the station-wide timestamp (which fabricated
last_seen for multi-battle cards).
C1 (intervals): a database-only reload now restarts the availability intervals,
rebinding their captured Db to the freshly installed manager (previously only
an events reload restarted them, leaving timers querying the replaced manager).
C2 (atomic gated refresh): DbManager.getAvailable is now pure, returning
{ available, conditions?, rarity? }; EventManager commits the drawer AND the
manager metadata (questConditions/rarity, via applyAvailableMetadata) together
under the generation gate, so a superseded refresh can overwrite neither.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
getAvailable now resolves { available, conditions?, rarity? } | null, not the
old Promise<T[]>; correct the @ts-check annotation (drop the stale @template T)
and add a JSDoc for applyAvailableMetadata so callers' `.available` access
type-checks.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Thanks — all addressed, and the
Availability Generation token — extended to close both gaps you flagged:
|
|
These versions deployed on my prod - no raids atm, but everything working as expected otherwise |
|
The endpoint-backed paths introduce request amplification on failed searches, persist expected by-ID misses as errors, and disable healthy endpoints when a dual source's DB probe fails. These are operational and fallback correctness regressions. Full review comments:
|
Unfortunately, the systems you touched have a fuckton of features + edge cases etc. |
Yes, and you didn't want me to build a test suite lol |
|
Raids availability working correctly also. So lmk if any other review angles pick up anything |
- Stop the radius-widening search when every source rejects. runScannerSources
returns [] only when all sources rejected (a genuine-empty source is fulfilled
with [], giving [[]]); an all-pure-endpoint fort search (no endpoint branch on
search) or a DB outage otherwise spun ~557 failing queries + warnings until the
2s cap. Break on data.length === 0.
- Treat fort-by-id 404s as normal misses. fetchJson's miss exemption only covered
/api/pokemon/id, so a getOne / manual / deep-link miss on gym/pokestop/station
logged an error and dumped the request under logs/. Broaden to the by-id
endpoints via /\/api\/(pokemon|gym|pokestop|station)\/id\//.
- Keep endpoint context when the DB probe fails. getDbContext ran schemaCheck
inside the try and overlaid mem/secret/httpAuth after it, so a schemaCheck
rejection (transient scanner-DB outage) skipped the overlay and left a dual
source without mem — its fort queries then bypassed the healthy endpoint and
hit the down DB. schemaCheck now has its own try/catch (degrading SQL flags to
{}), and the endpoint overlay + assignment always run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
All three fixed in [P1] Stop retrying after all search sources reject ( [P2] Exclude fort-by-id 404s from failed-request logging ( [P2] Initialize endpoint context before probing the fallback DB ( |
|
Endpoint-backed fort scans can silently truncate valid markers, and the endpoint gym path breaks the no-badge filter. These are functional regressions in the newly added endpoint flows. Full review comments:
|
|
For the first point, the maximum limit is aligned with the model for pokemon settings as well - and is a very high limit for forts. The AI suggestion is stupid. If individual users actually have a real problem with this (ha!) they can increase the limit. |
The endpoint candidate set isn't pre-filtered by badge (per-user ReactMap data Golbat can't know), so the shared secondaryFilter's badge clause — `(actualBadge === 'none' && onlyGymBadges)` — matched every candidate, letting previously-badged gyms leak into the `none` view that the SQL path removes via `whereNotIn(userBadges)`. Replace the always-true clause with one that mirrors the SQL whereIn/whereNotIn: `onlyGymBadges && (actualBadge === 'none' ? !newGym.badge : !!newGym.badge)` — the `none` view keeps only gyms with no user badge, a specific/all badge view keeps only those with one. It's a no-op for the SQL path (results are already badge-filtered) and closes the endpoint leak. Verified equivalent across all badge views (none/tier/all/off) for both paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Fixed in [P2] Exclude badged gyms from the endpoint's Replaced the always-true clause with one that verifies badge membership per view, mirroring the SQL onlyGymBadges && (actualBadge === 'none' ? !newGym.badge : !!newGym.badge)So the |
Deploying this (end users)
Two small config changes — one on Golbat, one on ReactMap. Once both are set, fort map-data (gyms, pokestops, stations) is served from Golbat's in-memory fort API instead of SQL, with automatic SQL fallback if Golbat is ever unreachable — so it's safe to enable on a dual (endpoint + DB) source.
1 · Golbat — run Golbat with UnownHash/Golbat#385 (it already includes the reward type-20 quest decode from #382) and turn on the in-memory fort index.
fort_in_memoryis a top-level option and defaults tofalse:Make sure your Golbat API
secretis set. Sanity-check the endpoint is live:2 · ReactMap — add
endpoint+secretto the scanner DB source whoseuseForincludes your fort types (gym/pokestop/station). This is the same Golbat endpoint most instances already point their pokémon source at; the DB source simply becomes dual (endpoint + DB):Restart ReactMap — this is a server-only change (no client rebuild). Gyms/pokestops/stations now fetch from Golbat; markers, popups, filters and deep-links keep working; and if Golbat is down or
fort_in_memoryis off, each fort type transparently falls back to SQL on the same DB.Confirm it's live (logs): each ~15-min availability refresh logs a line like
[GYM] loaded available from http://<golbat>:<port>/api/fort/available — …, and map pans stop issuing fort SQL on endpoint-backed sources. A Golbat outage instead logs a fallback warn and reverts to SQL.What this does
Routes fort map data — gyms, pokestops, and stations (
getAll,getOne,getAvailable) — through Golbat's in-memory fort API (UnownHash/Golbat#385) when a scanner source has anendpoint, with transparent SQL fallback. Map pans stop issuing fort SQL entirely on endpoint-backed sources, and Golbat's DNF filtering returns only the forts that will render.Architecture
memset on the source):POST /api/<type>/scan→ per-record pure mapper → the existing, unchangedsecondaryFilter. On any failure (503 / network / bad shape) the model logs and falls through to SQL — a dual source degrades gracefully.secondaryFilterguarantees exactness. Pure backends (server/src/filters/fort/*) translate the map filters into Golbat DNF clauses (OR across clauses, AND within). The one hard invariant is never under-return; anything inexpressible (quest title/target conditions, gender, gym badges, time-window cutoffs) poisons to match-all or stays a client-side residual.secondaryFilter's per-perm field gating — andgetOnereturns only{lat, lon}like SQL.DNF(n): X matched -> Y after secondaryFilter (-residual) | <clause shapes>), so any translation gap is visible in production logs.onlyManualIdfort is fetched via the by-id endpoint and joins the candidate set, mirroring SQL's(bbox) OR id = ?.Source shapes
A scanner source can be pure-DB (unchanged), dual (endpoint + DB — fort reads use Golbat, everything else and the fallback use the bound DB), or pure-endpoint (endpoint only, no DB — degrades gracefully where a query would need a DB). Enabling the endpoint on an existing DB source (the common case above) makes it dual.
Verified live (dual source, endpoint vs SQL)
station_active, gmax)Requirements
fort_in_memory = true), deployed before this — Golbat's scan decoder ignores unknown request fields, so an out-of-date Golbat degrades to match-all/SQL rather than erroring.endpoint+secret(and optionalhttpAuth) as for pokémon; DB config unchanged (used for fallback + un-migrated queries).No test framework exists in this repo; verification was throwaway node goldens per pure module (mappers, DNF backends),
eslint/prettier, a full correctness/security/concurrency review of the branch, and the live parity runs above.🤖 Generated with Claude Code