Repository navigation
Increment 26a: MapBiomas catalogue entries and per-year land-cover class windows - #232
Merged
Merged
Conversation
…lass strips (@tester) Failing tests for increment 26a, "MapBiomas in the catalogue, and class windows" (docs/increments/26-land-cover.md, section 9 row 26a and section 10 "26a (Python)"), written before any production code. 48 tests, all red: .venv/bin/python -m pytest tests/python/test_sources_mapbiomas.py \ tests/python/test_io_class_window.py tests/python/test_fetch_per_year.py \ tests/python/test_cli_mapbiomas.py -o addopts="" --no-cov -> 35 failed, 13 errors What each file pins and what it fails on today: - tests/python/class_fixtures.py: a 40 x 48 uint8 class raster in EPSG:4326, PixelIsArea, 16 x 16 tiles, cell step 1/128 degree (so every cell corner is exact in binary and a box edge "on a cell edge" is on it), one grid per year, a 10 x 10 no-data (0) corner, empty tiles by cog_fixtures.with_zero_byte_count; the oracle `expected` zeroes the empty tiles' rectangles by hand from tifffile's page attributes. A per-year catalogue entry for the local range server. The pins are listed in its docstring. - test_sources_mapbiomas.py (16): SOURCES["mapbiomas-c11"] (KeyError today): kind one-cog-per-year, values "classes", years 1985..2025, EPSG:4326, nodata 0, the design's URL for 2025 and 1985 through url_for, the default year the last, a year outside the range refused naming 1985 and 2025, the credit with "Collection 11", {date} and {url}, CC BY-SA 4.0, the Souza et al. citation; credit_for renders the date as ISO; anadem-v1 and glo30 hold heights with no years; check_values refuses a class source as --dem and a heights source as --land-cover, naming source, values and option. - test_io_class_window.py (20): ClassWindow (AttributeError today): read-only uint8, corner(row, col), anything but 2-D uint8 refused; decode_class_window over five windows equals the written grid, empty tiles read as zeros, the PixelIsArea corner is the cell's corner (the DEM reader's meta is half a cell in), LZW with the horizontal predictor through imagecodecs (needs_codecs), one thread and eight the same bytes, a missing block NotCached before any read; CacheRepository.load_class_window and class_strips: strips of at most `rows` rows top to bottom covering the cells whose interior meets the box (open cells), a box on cell edges, a box past the raster clipped, a box meeting no cell a plain ValueError, a missing block NotCached. - test_fetch_per_year.py (9): RemoteSource refuses the per-year kind today. Default year names brazil_coverage-col11_2025; each year its own object in one manifest; a year outside the range or a year given for a one-cog source is a FetchError before any request; empty tiles never requested and written as empty files (report.empty 2); the cached class window reads them as zeros; CachedObject.fetched is the day the header was first fetched and is kept on a later run; NOTICE.txt carries each object's credit with its date and URL. Note: FetchRequest(year=...) is silently ignored by Pydantic today, so the one-cog refusal test is red as "did not raise" until `year` exists. - test_cli_mapbiomas.py (5): `fetch --year` ("No such option" today), the object named in the output, a year outside the range a usage error naming it and the range with no request made, --year on a source without years a usage error, and `mesh --dem mapbiomas-c11` refused naming the key, "classes" and --dem with the network closed (today the unknown key is read as a path). - test_cli_fetch.py: catalogue_with split into install(sources, monkeypatch, entry), so the new CLI suite installs a per-year entry the same way; the existing 15 tests pass unchanged. No throwaway implementation: section 10 says "no throwaway implementations", so the red reasons were checked per test (--tb=line) instead. No mutation targets: section 9 names only the ledger and the reduction suites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d class strips (@developer) The green step of increment 26a (docs/increments/26-land-cover.md, section 9 row 26a; section 3.1 "The catalogue entry"; Ola's rulings of 2026-10-04: Collection 11, year 2025 by default, CC BY-SA 4.0). The 48 tests of cb5f254 pass; no test file is touched. Python only, no new dependency; the C++ core sees no file and no CRS. sources.py: `kind` gains "one-cog-per-year" (`url_template` with `{year}`, validated), `years` (ascending, the default is the last) and `values` ("heights" | "classes"). `url_for(year=None)` picks the year's file and refuses a year outside the range naming the first and last, or any year for a source without years. `credit_for(url=, date=)` fills `{date}` (ISO) and `{url}` into the credit; a credit without them is unchanged. `check_values` is the one refusal by `values`, named by the option. The entry `mapbiomas-c11` carries the design's URL template, 1985..2025, EPSG:4326, nodata 0, MapBiomas's credit format, the CC BY-SA 4.0 note and the Souza et al. (2020) citation. `notice(source, credits=())` renders the given credits instead of the entry's, so `NOTICE.txt` carries each object's fetch date and URL; with none given it is unchanged (test_fetch_run pins the old text). The date type is a Protocol (`Dated`: `isoformat()`), because test_sources_imports_pydantic_only pins this module to Pydantic alone. fetch/plan.py: `FetchRequest.year: int | None = None`. fetch/run.py: every non-tiled source takes its file from `url_for(year)`, the object id from the file name as before (`brazil_coverage-col11_<year>`); a `ValueError` there is a `FetchError` before any request, as is `--year` on a tiled source. A new object records `fetched=date.today()`; known objects keep theirs. The notice is built from every object's rendered credit, deduplicated. io/models.py: `ClassWindow(x_min, y_max, delta_x, delta_y, array)`, frozen, a read-only uint8 copy (anything else refused naming uint8), `rows`, `cols`, `corner(row, col)`. io/cog.py: `decode_class_window(source, meta, window, *, threads=None)`: the same window checks and NotCached-before-any-read as `decode_window`, the page's dtype must be uint8, empty blocks (byte count 0) stay 0 in a zeroed canvas, the corner is the node moved out half a cell (PixelIsArea). The block decoding loop is now `_decode_into`, shared with `decode_window`, whose behaviour is unchanged. io/repository.py: `CachedObject.fetched: date | None = None`; `CacheRepository.load_class_window(name, window)` and `class_strips(name, box, *, rows)`: the cells whose open interior meets the box (floor of the lower index, ceil of the upper, so a box edge on a cell edge takes neither neighbour), clipped to the raster, a box meeting none a plain ValueError, top to bottom in strips of at most `rows` rows. cli.py: `fetch --year`, refused as a usage error before any request when the source has no years or lacks the year; `mesh --dem <class source>` refused by `check_values` in `_dem_sources`, before the cache is read. Checks run: the four 26a suites (48 passed), test_fetch_run, test_cli_fetch, test_cli_mesh_cache, test_refine_golden (.vtk unchanged) and test_layering (no new rows); mypy strict, ruff check and format; check_prohibited_deps, check_detria_boundary, check_citations. Whole Python suite: 7191 passed, 17 skipped, 1 failed on the datetime import, fixed as above and re-run. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…P row, by the main session Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…romised, by the main session Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…thin band; ACCEPTED (@Perf) tools/bench_quick.py run at f51f0a1 (worktree-mapbiomas-26a, base master de9b6c0) against docs/benchmarks/quick/baseline-ac.json (357808b, AC, numpy 2.5.4, saved by #229). Verdict NO CHANGE, exit 0: no MESH CHANGED, BROKEN, SLOWER or FASTER line on any of the seven cases, so every mesh SHA-256 equals the baseline's, every total is within its band, and decode (judged on every case, 42 % of geilo-al-ramp) is within its band. 26a touches no refine or mesh code; the one speed-relevant change is the block-decoding loop moved into _decode_into in io/cog.py (8eec354). Hotspot lines: the same eight as the baseline run, six at the same share, tile refine 46 -> 44 % and quarter refine 48 -> 47 %. Lagan features clip (ruled 40.7 %) did not print. AC power, battery 100 %, caffeinate on, nothing else running. Evidence: docs/benchmarks/2026-10-11/mapbiomas-26a/ (README.md, run.log, power-state.txt). The run's per-case medians are not recorded: the tool saves a record only as the baseline, and a second run with --save-baseline (to copy the record here and restore the baseline) was refused by the session's permission classifier. baseline-ac.json is untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ow, by the main session Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ady for @tester' phrases fixed, by the main session Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…OADMAP row, by the main session Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
skavhaug
enabled auto-merge
October 12, 2026 04:44
github-merge-queue
Bot
removed this pull request from the merge queue due to a conflict with the base branch
Oct 12, 2026
…26a (@developer) PR #232 (increment 26a, land-cover class raster in the cache) went DIRTY in the merge queue after master moved. origin/master at a1aec50 brings two merged PRs: #231 (increment 36, GeoPackage output, merge commit fa6e485) and #230 (release hygiene, merge commit a1aec50). Merge commit, no rebase: the branch is pushed and the PR is open. One conflict, src_python/tin_engine/io/repository.py, in its module docstring only. Master wrote the paragraph "Increment 36 adds :func:`create_sqlite` ..." directly under the "Audit PR C" line; this branch wrote the paragraph "Increment 26a adds the class raster's reads ..." after a blank line. Resolved by keeping both, 36's first (it merged first), then 26a's, each as its side wrote it. The rest of the file auto-merged, and both sides' code is present: CachedObject.fetched, CacheRepository .load_class_window and .class_strips (26a), create_sqlite (36). ROADMAP.md, cli.py and io/models.py auto-merged. ROADMAP rows 26 and 36 are byte-identical to their own branch's text. In cli.py, 26a's --year option and check_values on --dem and 36's .gpkg output path are disjoint additions; in io/models.py, 26a's ClassWindow and 36's SrsEntry likewise. Gates, in this worktree's venv, after the merge: ruff check . all checks passed ruff format --check . 250 files already formatted mypy no issues in 68 source files python3 tools/check_prohibited_deps.py OK, 399 files clean python3 tools/check_citations.py exit 0; 60 at-risk lines listed, all of the form "this branch edits <file>" from the merged edits pytest tests/python/ 7236 passed, 17 skipped (environment skips only), coverage 99% (floor 85) No test broke. No C++ change on either side; _core not rebuilt. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Increment 26a, the first part of
docs/increments/26-land-cover.md(section 13 is the as-built record). 163 net production lines (python3 tools/count_loc.py de9b6c03 HEAD).RemoteSourcegainsyears,values(heightsorclasses),url_for(year=None),credit_for(url=, date=)andcheck_values;FetchRequest.year,CachedObject.fetched.ClassWindow(origin, pixel size, integer class array),io.cog.decode_class_window,CacheRepository.load_class_windowandclass_stripsshare one_decode_intopath with the height decoder.Evidence: @Perf quick check NO CHANGE (
docs/benchmarks/2026-10-11/mapbiomas-26a/). No mutation round, as section 9 of the design rules for 26a. Code review rounds 1 and 2 recorded in the increment file, round 2 APPROVED. 26b (exact class fractions) onward is designed, ruled and not built.🤖 Generated with Claude Code