Repository navigation
Get the backend test suite green and run it in CI - #171
Merged
Merged
Conversation
- Retire the dead /load/census route, query/timeseries_db.py, its tests and golden CSVs. The frontend uses /load/acs5-db/timeseries/*; the only caller (Filter_Call.tsx, also removed) was never imported and hit a path the route did not match, and the acs5_* tables it read are gone from the warehouse. - Point test_sql_render at the current templates (build/sql moved to data_cleaning/sql) and fix the stale compiled-CTE expectation. Fix the SQL lint debt that test now catches; rendered output is unchanged. - Update test_clean_acs5 for the DOUBLE value column and add a fixture test for the "(X)" and sentinel-code nulling. - conftest: give import-time get_db() an empty in-memory DuckDB when no warehouse is built, and skip `requires_warehouse` tests instead. - Rename the DuckLake diagnostic test_lake.py to inspect_lake.py so pytest no longer opens the lake on collection. - Add a Backend tests workflow running the whole suite. - Fix stale build/sql paths in the justfile, CLAUDE.md and .sqlfluff.
iansargent
reviewed
Oct 31, 2026
iansargent
reviewed
Oct 31, 2026
iansargent
reviewed
Oct 31, 2026
iansargent
reviewed
Oct 31, 2026
iansargent
left a comment
iansargent
left a comment
Collaborator
There was a problem hiding this comment.
Looking through the code, most of this stuff looks to be reformatting SQL queries, updating documentation references to the old build/ logic, deleting stale files (timeseries_db.py, Filter_Call.tsx, etc.), and updating the testing files.
Everything looked good besides that one SQL fix with the county column.
iansargent
approved these changes
Oct 31, 2026
The compiled WHERE clause names columns unqualified, and the county lines share county, county_fips and geoid with PLACES. Excluding county from the join only covered that one column; a county_fips filter still failed with an ambiguous reference. Filtering PLACES in a CTE first leaves nothing to clash.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CI only ran the MCP tests, so the rest of the backend suite had decayed: 38 failures and 2 collection errors on main. This fixes or retires each failing test and adds a Backend tests workflow that runs the whole suite without a warehouse. Tests that need real warehouse data are marked
requires_warehouseand skip there. The result is 444 passed, 53 skipped and 0 failed, both locally and in CI./load/census/{category}/{subcategory}route,query/timeseries_db.py, its tests and golden CSVs, and the unusedFilter_Call.tsx. The frontend reads ACS series from/load/acs5-db/timeseries/*.test_sql_render.pyat the 40 templates that exist and fixes the lint violations it caught. The three structural SQL rewrites return identical results against the warehouse.test_clean_acs5.pyfor the DOUBLE value cast and adds a fixture test for Census unavailable codes.test_lake.pytoinspect_lake.pyso pytest doesn't collect it.conftest.pygives modules that open it at import an empty in-memory database.tests/test_base.pyuntil the margin-of-error work ports it todata_collection.base.zoning/footprints.sqlcallsST_AsGeoJSONon a BLOB, so the building-footprints endpoint fails before and after this change.zoning/geo_query.sqlrenders invalid SQL when there are no filter sources. The repo-rootData/lakeis a stale committed DuckLake catalog.