diff --git a/ROADMAP.md b/ROADMAP.md index 7b4bfd4f..869065fd 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -50,13 +50,13 @@ increment that most needs a picture to check against | 27 | The sampler reads a vertex on a DEM node from that node: `_core.sample` refused a cell with any NoData corner even at zero weight, so on the path without `--tolerance` a valid node next to a NoData node lost its height (increment 12's accepted one-cell trim, `12-dem-to-mesh.md` R2). Found again by increment 25's green step. A point that is a node bit for bit (`RasterGeometry::node_at`) now reads `value_at`; every other point keeps the four-corner rule; the `--tolerance` path is unchanged | **merged (#160, 2026-10-04)** (designed and approved 2026-10-03; Ola ruled nodes only, the rule in the C++ sampler; red `6605dfe`, pins `80990e0`, green `6fd4076`/`a0817ba`; 13 net production lines; no `@perf` run needed) | `docs/increments/27-node-sampling.md` | | 23 | Basin scale: the domain cut into pieces along lattice lines, Nx × Ny ≈ `--pieces` from the domain's bounding box, cut only when the estimated memory at the tolerance exceeds `--memory-budget` (16 GB; BHO dropped as geometry; sub-catchments later from the DEM, never seams), each seam given its heights by a one-dimensional seam pass and then frozen, so pieces refine independently and in parallel; windowed DEM reads with no dense canvas; `rasputin fetch`, a resumable block cache of remote COGs (ANADEM, GLO-30), so meshing never touches the network; pieces plus an index, stitched into one file by default, with the seams removed and locally thinned. Absorbs 15d's window decoding; supersedes Q9's dense canvas at basin scale | **designed and ruled** (`@architect`, 2026-10-01; Ola ruled B1-B14 the same day, B15 (the refusals at half of physical memory: delete both) and B16 (licence notes in the mesh file) on 2026-10-02; 23a-1 merged as #136, 23a-2 merged as #138; 23b (frozen edges and the seam pass) merged as #162 (merge commit 4f56551), green 3c464ec with N18 in, 249 net lines, `@perf`'s acceptance recorded at 91c7cb5 and again at the merged head 4541e38, ACCEPTED; 23c split in two, 23c-1 (the partition rule, the piece vocabulary, the index model, a start mesh from arrays) built at 156 lines and merged as #164 (merge commit e56a2bf), 23c-2 (pieces, the run, the CLI) in progress on `worktree-23c`, not pushed, its state in that branch's copy of `23-basin-scale.md`): eight PRs, 23a-1, 23a-2, 23b, 23c-1, 23c-2, 23d, 23f, 23g (about 225, 373, 285, 215, 265, 235, 300 and 345 lines estimated); measured for it: the whole basin is 3,061 ANADEM blocks, 1.71 GiB, cut into 23 pieces at 1 m at the default budget | `docs/increments/23-basin-scale.md` | | 24 | Release hardening, switched on: the bounds-checking standard library (libc++ FAST, libstdc++ assertions) on by default in every build, with a build option to turn it off for heavy runs (Ola, 2026-10-03: "So normal release, turn it on. For real heavy lifting, option to turn it off."); the loaded mode reported by `_core.hardening`, `rasputin version` and `--stats`; a guard test that fails when a toolchain ignores the macro; `bench.py --hardening on|off`, with hardened the baseline from here on | **designed and ruled** (`@architect`, 2026-10-03; Ola ruled Q1-Q4 the same day): the CMake option `RASPUTIN_HARDENING`, default ON, off by rebuilding into a second venv; an Ubuntu CI leg builds it OFF; one PR; **implemented on `worktree-agent-a08216a68d5534b18`** (red `21d5168`, green `23d4dad`, 53 net production lines; T6 reworked `57e1264`); `@reviewer` APPROVED (round 3); **`@perf` acceptance ACCEPTED** (2026-10-03, AC: built OFF it matches master `6cdc8cc`, meshes byte-identical; hardening costs refine +4.2 to +5.9 % at 1 thread and +3.2 to +11.1 % over 2 to 20 threads; first hardened baseline `docs/benchmarks/2026-10-03/24-on-r2/`, evidence `docs/benchmarks/2026-10-03/24-acceptance.md`). Two extensions in one wheel (option (b)) rejected. Measurement: `docs/benchmarks/2026-10-03/release-hardening/README.md` (#149) | `docs/increments/24-release-hardening.md` | -| 26 | Land cover for the São Francisco basin, MapBiomas (the land-cover half of "the basin's own inputs"): water bodies of at least a minimum area traced on cell edges and reduced with their area kept, as constraints (`rasputin water`); every other class as fractions per triangle from exact cell areas, small entries (under 5 % of the triangle and under 1 ha) dropped and their area carried along a Hilbert curve to the next triangles holding the class (Ola's "local out-of-balance ledger"); `rasputin fetch mapbiomas-c11 --year` | **26a built** (branch `worktree-mapbiomas-26a`: red `cb5f254b`, green `8eec354b`, 163 net production lines; `@perf`'s quick check `38f86930` NO CHANGE, every mesh byte-identical; code review APPROVED in two rounds; next the push on Ola's yes); 26b onward designed and ruled (`@architect`, 2026-10-03; Ola ruled all 11 questions on 2026-10-04: Collection 11, year 2025 by default; CC BY-SA 4.0 on the outputs; triangles inside a lake carry the raster's own mix, nothing forced to water; 26a-26c now alongside the basin work, the water PRs after the basin PRs, 26e with 23g). Measured for it on the Rio Corrente meshes at 20 to 2 m: renormalising inside triangles loses 8 to 45 % of the water, the ledger keeps every class within 0.05 %. Six PRs, 26a to 26e (about 170, 420, 300, 425, 355 and 210 lines) | `docs/increments/26-land-cover.md` | +| 26 | Land cover for the São Francisco basin, MapBiomas (the land-cover half of "the basin's own inputs"): water bodies of at least a minimum area traced on cell edges and reduced with their area kept, as constraints (`rasputin water`); every other class as fractions per triangle from exact cell areas, small entries (under 5 % of the triangle and under 1 ha) dropped and their area carried along a Hilbert curve to the next triangles holding the class (Ola's "local out-of-balance ledger"); `rasputin fetch mapbiomas-c11 --year` | **26a merged (#232)** (branch `worktree-mapbiomas-26a`: red `cb5f254b`, green `8eec354b`, 163 net production lines; `@perf`'s quick check `38f86930` NO CHANGE, every mesh byte-identical; code review APPROVED in two rounds; master merged in as `5f28292c` for a docstring conflict with #231; merged 2026-10-12); 26b onward designed and ruled (`@architect`, 2026-10-03; Ola ruled all 11 questions on 2026-10-04: Collection 11, year 2025 by default; CC BY-SA 4.0 on the outputs; triangles inside a lake carry the raster's own mix, nothing forced to water; 26a-26c now alongside the basin work, the water PRs after the basin PRs, 26e with 23g). Measured for it on the Rio Corrente meshes at 20 to 2 m: renormalising inside triangles loses 8 to 45 % of the water, the ledger keeps every class within 0.05 %. Six PRs, 26a to 26e (about 170, 420, 300, 425, 355 and 210 lines) | `docs/increments/26-land-cover.md` | | 29 | NVE reference catchments: a catchment from the DEM for each of the 140 gauging stations in NVE's reference list of active, unregulated stations (Ola, 2026-10-04: "NVE should have a list of unregulated catchments, where they also record the discharge"), each compared with NVE's own catchment polygon (share of NVE's polygon in ours and of ours in NVE's, area ratio, mean distance between the divides). Adds flow accumulation from increment 22's flood and places each gauge where it physically is: on NVE's mapped river line, then on the DEM's flow path along it (burnt where they disagree), never by an area or flow-count objective (Ola's direction, 2026-10-04); a per-station sensitivity of the area along the river marks ill-posed stations `uncertain`. `rasputin fetch-stations nve-hrd` fetches the stations, rivers and NVE's polygons (NLOD), `rasputin station-catchments` runs them all. Residual inflow between gauges is later; the interfaces leave room for it. Discharge out of scope; station and series numbers kept for a later join. 138 of the 140 polygons lie inside DTM10's data; two cross into Finland and are expected to be refused | **PR 1, 2, 3 and 4 merged (#173, #179, #178, #182); the acceptance run over all 140 stations is in, at `docs/benchmarks/2026-10-05/nve-hrd/` (as fixed in `cd8b5be`): 74 match, 5 close, 6 miss, 39 uncertain, 16 refused (14 of them expected), 33 minutes and at most 7.7 GB; evidence review round 2 approved, the push waits for Ola; PR 5 (the nearest-stream fallback) and acceptance step 5 (the comparison with nearest stream) not started; questions 6 and 10 go back to Ola with the run's figures** | `docs/increments/29-nve-reference-catchments.md` | | 30 | The bottlenecks of `rasputin mesh` on real catchments, in three pull requests, with every output byte-identical: 30a the `land cover` phase (point-in-polygon against prepared polygons, one sort fewer, no sort to pick each area's point), 30b the CORINE clip (point tests before the segment test, a feature whose lines miss the search region skipped whole, no clip of a line lying in the domain's interior), 30c reading the DEM (decode threads from the machine's cores instead of a fixed 4, the seam report's test against the domain's region run only on the nodes it can count). Measured by `@perf` on Numedalslågen and Skiensvassdraget (14.4 s and 21.2 s on battery; `docs/benchmarks/2026-10-06/bottlenecks/README.md`), projected to about 7.8 s and 11.7 s with all three. Approved by Ola 2026-10-06, in that order ("yes, go ahead in that order") | **30a merged (#199)** (`worktree-landcover-speed`: red `6cde0eb`, green `3066d60`, +12 net production lines; `@perf` ACCEPTED on AC power, median of 3: land cover 2.85 → 0.67 s on Numedalslågen and 6.54 → 1.74 s on Skiensvassdraget, whole run 13.66 → 11.41 s and 20.46 → 15.68 s, every `.vtk` byte-identical to the base; `docs/benchmarks/2026-10-06/30a-landcover/README.md`); **30b merged (#200)** (`worktree-clip-speed`: red `08269e77`, green `c5340459`, +23 net production lines; `@perf` ACCEPTED on AC power, median of 3: `features clip` 5.83 → 1.83 s on Numedalslågen and 5.66 → 1.35 s on Skiensvassdraget, whole run 11.39 → 7.52 s and 15.69 → 11.41 s, every `.vtk` byte-identical to the base and every base line of the probe unchanged; `docs/benchmarks/2026-10-06/30b-clip/README.md`); **30c approved by code review round 2; waits for Ola's yes to push** (`worktree-dem-read-speed`, 2026-10-06; Ola said "yes, build 30c"; red `bdf7b57d`, green `0663efc9`, +3 net production lines; `@perf` ACCEPTED on AC power, median of 3: `decode` 1.788 → 1.051 s on Numedalslågen and 1.323 → 0.842 s on Skiensvassdraget, whole run 7.48 → 6.72 s and 11.46 → 10.97 s, every `.vtk` byte-identical to the base and every base line of the probe unchanged; `docs/benchmarks/2026-10-06/30c-dem-read/README.md`; master merged in as `26a5d839`, where every base line of the probe is unchanged and the 16 new lines are all from the red suite) | `docs/increments/30a-landcover-speed.md`, `docs/increments/30b-clip-speed.md`, `docs/increments/30c-dem-read-speed.md` | | 30d | Growing the catchment outline, made faster, with every mesh byte-identical: the same mitred buffer of the domain computed once instead of twice on the reprojected path; the CORINE search region grown from the domain's convex hull instead of taking the hull of the grown domain; and a long staircase outline (SMHI's raster-traced ones) grown in pieces of 1,000 edges and united, behind a gate that keeps smooth outlines on GEOS's single buffer. Measured by `@perf` on Lagan with the European CORINE GeoPackage: 103.4 s on master, about 17 s with all three (`docs/benchmarks/2026-10-07/30d-buffer/README.md`); from the performance review's finding F1 (PR #208). Ola, 2026-10-07: grow the outline in pieces; the gate is an open question, default yes | **merged (#215, 2026-10-07)** (branch `worktree-buffer-speed-2`; 29 net production lines; Python only, no refine or mesh code; Lagan 103.38 s → 16.97 s, meshes byte-identical; push waits on Ola) | `docs/increments/30d-outline-buffer-speed.md` | | 31 | Binary output by default: `rasputin mesh` writes binary `.vtk` (and `.ply`) unless `--ascii` is given, because the text writer formats one number at a time and costs minutes at basin scale (the performance review's finding F6 and question 3, PR #208). Ola, 2026-10-07: "change 3 to default binary". The flag pair `--binary/--ascii` stays; the help says binary is the default; README updated. Python only; no `@perf` run (not refine or mesh code) | **merged (#212, 2026-10-07)** (designed 2026-10-07; red `e3fe391a`/`9008811c`, green `29bc5ad3`; 4 net production lines against about 1, the overrun is `ruff format` wrapping the option); Ola ruled `.ply` binary too, as built; next the push | `docs/increments/31-binary-default.md` | | 32 | Land-cover borders simplified within a band, each class keeping its area exactly (Ola, 2026-10-08: "we need to implement is a CORINE simplification, similar to what we do on the outline of the auto-catchments"): the land-cover coverage cut into borders between junctions, each border reduced by the area-preserving segment collapse of the catchment outline (`reduce_ring`), with junctions and the domain outline fixed, a crossing test and a side test across all borders, and a band no border moves past (two-sided, proved by anchors on the source border). `--features-tolerance` becomes that band, default 50 m (half CORINE's positional accuracy), replacing GEOS's coverage simplifier; start angle 15° when land cover is simplified (band above 0), 25° otherwise. Prototype on the German fused catchment: 5.7 times fewer triangles at 50 m vertical tolerance | **merged (#222, 2026-10-11)** (green `82c0a7d4`, 519 net production lines, one PR; mutation round `30f939b6`, one mutant recorded as not killed); Ola ruled all four questions (2026-10-08); code review round 1 asked for changes (comments, docstrings and `project_structure.md` the change made false), fixed by `4d4f0411` and the docs commit after it; code review rounds 1-3 done (round 3 approved); `@perf`'s short timing check found the runs about 70 % slower (the outline rule on borders clipped to the catchment) and the worst angle far lower (the simplifier placing points millimetres from junctions); fix built (section 15: outline rule first, a 1 m clearance in the simplifier; green `05c8f73a`, 573 counted lines in all; mutation round M7 to M13 killed); its quick check `b192a2c6`: Numedalslagen −11 %, Lagan −1.3 %, outline rule and moved area as before, triangles −42 %; questions 5 and 6 decided on default while Ola was away, reversible; fix code review round 1 asked for prose fixes and the test file's red-step staging removed; fix code review round 3 approved; next the push on Ola's yes | `docs/increments/32-landcover-simplify.md` | -| 36 | GeoPackage output, so QGIS opens a mesh as it opens any vector layer: `rasputin mesh --out X.gpkg`, chosen by the suffix of `--out` as `.vtk` and `.ply` are (Ola, 2026-10-05: "Is the vtk_to_phkg.py the way to go, or as a direct output from rasputin? I think the latter, and it should go into the roadmap and eventually into the codebase."). Proposed content: a `triangles` layer, one PolygonZ per triangle in the run's CRS (z at each of the three corners, as the stopgap writes), with the attributes the run already knows: land-cover code and class name (the code system's names, as `rasputin palette` has them), mean z, plan area; a layer of the constraint polygons (the `--features` polygons clipped to the domain, with their code); optionally a QGIS style for land cover stored in the file (the `layer_styles` table), so the triangles open coloured. Land-cover colour: when the mesh carries CORINE classes, each triangle and each constraint polygon also gets its class's colour as a `#rrggbb` text column, so a reader can colour by the column without the stored style (in QGIS, a data-defined fill colour). The Graz file of 2026-10-05 carries `clc_color` on its `triangles` and `corine` layers; its stored styles categorise on `clc_code` and do not read the column. The colours are those of `rasputin palette` (`src_python/tin_engine/palettes.py`), the same 26 the Graz file carries: natural colours, as Ola chose over the official CLC legend in 16c's ruling 2 (that legend paints peat bogs blue and the sea almost white). **One attribute is not in the mesh yet**: the ID of the source polygon each triangle lies in (a sub-catchment cell, a parcel). Today only a class code labels triangles, one code system per mesh (16c, 16e's D1); the stopgap finds the cell by a centroid lookup outside rasputin. It needs a per-triangle array from a polygon source's ID property: its own small step, done just before this writer, because the `.vtk` and `.ply` outputs gain the attribute too (Ola, 2026-10-05, accepting the default: "defaults on the cell ID"). Written with the standard library's `sqlite3` beside the existing reader `src_python/tin_engine/io/geopackage.py`, which can read it back in the tests; no GDAL, OGR or Fiona (CLAUDE.md §2). Python only: the C++ core is unchanged and never sees the file (the I/O boundary). Prior art and working spec: Ola's stopgap `vtk_to_gpkg.py`, outside the repository (`rasputin_data/lands/scripts/`), which converts a mesh `.vtk` with `sqlite3`, NumPy, Shapely and PyProj, no GDAL, and its first use, the Mur above Graz (2 451 734 triangles, 10 m DEM, tolerance 10 m, 2026-10-05): **the GeoPackage was 677.9 MB against 110.6 MB for the binary `.vtk`**; the whole stopgap run took 39 s and 4.1 GB peak, and that includes the centroid cell lookup and the CORINE clipping, so it is not the cost of the GeoPackage encoding alone: the design measures the writer on its own. A GeoPackage row holds the triangle's whole ring (four corners, the first repeated) and a bounding box, and the file adds a spatial index, where `.vtk` stores each vertex once. Questions for the design: file size (keep the spatial index or not; at most an opt-in variant of 2D polygons with mean z as an attribute, proposed default: full PolygonZ; writing in chunks to bound memory); the CRS entry for a CRS with no EPSG code; where the run record (increment 25) goes; whether the constraint lines get a layer of their own | **built** (branch `worktree-gpkg-out`: red `87f41c3d`, green `c59246e5`, fix-up `7085dddf`, 377 net production lines, mutation round `dddce8e8` ten of ten killed; `@perf`'s acceptance `57c25074` ACCEPTED: Numedalslagen 3.93 s / 203 MB, Romsdalen 19.4 s / 810 MB, every mesh byte-identical; code review round 1's one blocker, an empty land-cover layer crashing the write, fixed by `55cc3c3e` + `18442b27`, 378 net lines; code review round 2 APPROVED; next the push on Ola's yes). Designed by `@architect`, 2026-10-11, on `81a551b3`; proposed by Ola 2026-10-05, "yes, roadmap row now, build after PR D"; PR D merged as #205, so the writer joins the encoders under `io/`). Measured on this design's probe: the quick check's Numedalslagen mesh (653,039 triangles) written in 4.0 s to 203 MB against 0.28 s and 29 MB for the binary `.vtk`, the stopgap 5.1 s; the Romsdalen slope mesh (3.1 million) in 21.5 s to 922 MB at 2.1 GB peak, the stopgap 28.7 s at 4.1 GB; the R-tree index is 60 % of the time; the same run writes the same bytes. Layers `triangles` (PolygonZ with code, name, colour, z_min, z_max, z_mean, area_m2, slope_deg), `lines`, `land_cover`, `land_cover_classes`, a `run` table with the run record, and the stored QGIS style; about 305 net lines, one PR; questions 1-5 answered with their defaults while Ola is away | `docs/increments/36-geopackage-output.md` | +| 36 | GeoPackage output, so QGIS opens a mesh as it opens any vector layer: `rasputin mesh --out X.gpkg`, chosen by the suffix of `--out` as `.vtk` and `.ply` are (Ola, 2026-10-05: "Is the vtk_to_phkg.py the way to go, or as a direct output from rasputin? I think the latter, and it should go into the roadmap and eventually into the codebase."). Proposed content: a `triangles` layer, one PolygonZ per triangle in the run's CRS (z at each of the three corners, as the stopgap writes), with the attributes the run already knows: land-cover code and class name (the code system's names, as `rasputin palette` has them), mean z, plan area; a layer of the constraint polygons (the `--features` polygons clipped to the domain, with their code); optionally a QGIS style for land cover stored in the file (the `layer_styles` table), so the triangles open coloured. Land-cover colour: when the mesh carries CORINE classes, each triangle and each constraint polygon also gets its class's colour as a `#rrggbb` text column, so a reader can colour by the column without the stored style (in QGIS, a data-defined fill colour). The Graz file of 2026-10-05 carries `clc_color` on its `triangles` and `corine` layers; its stored styles categorise on `clc_code` and do not read the column. The colours are those of `rasputin palette` (`src_python/tin_engine/palettes.py`), the same 26 the Graz file carries: natural colours, as Ola chose over the official CLC legend in 16c's ruling 2 (that legend paints peat bogs blue and the sea almost white). **One attribute is not in the mesh yet**: the ID of the source polygon each triangle lies in (a sub-catchment cell, a parcel). Today only a class code labels triangles, one code system per mesh (16c, 16e's D1); the stopgap finds the cell by a centroid lookup outside rasputin. It needs a per-triangle array from a polygon source's ID property: its own small step, done just before this writer, because the `.vtk` and `.ply` outputs gain the attribute too (Ola, 2026-10-05, accepting the default: "defaults on the cell ID"). Written with the standard library's `sqlite3` beside the existing reader `src_python/tin_engine/io/geopackage.py`, which can read it back in the tests; no GDAL, OGR or Fiona (CLAUDE.md §2). Python only: the C++ core is unchanged and never sees the file (the I/O boundary). Prior art and working spec: Ola's stopgap `vtk_to_gpkg.py`, outside the repository (`rasputin_data/lands/scripts/`), which converts a mesh `.vtk` with `sqlite3`, NumPy, Shapely and PyProj, no GDAL, and its first use, the Mur above Graz (2 451 734 triangles, 10 m DEM, tolerance 10 m, 2026-10-05): **the GeoPackage was 677.9 MB against 110.6 MB for the binary `.vtk`**; the whole stopgap run took 39 s and 4.1 GB peak, and that includes the centroid cell lookup and the CORINE clipping, so it is not the cost of the GeoPackage encoding alone: the design measures the writer on its own. A GeoPackage row holds the triangle's whole ring (four corners, the first repeated) and a bounding box, and the file adds a spatial index, where `.vtk` stores each vertex once. Questions for the design: file size (keep the spatial index or not; at most an opt-in variant of 2D polygons with mean z as an attribute, proposed default: full PolygonZ; writing in chunks to bound memory); the CRS entry for a CRS with no EPSG code; where the run record (increment 25) goes; whether the constraint lines get a layer of their own | **merged (#231)** (branch `worktree-gpkg-out`: red `87f41c3d`, green `c59246e5`, fix-up `7085dddf`, 377 net production lines, mutation round `dddce8e8` ten of ten killed; `@perf`'s acceptance `57c25074` ACCEPTED: Numedalslagen 3.93 s / 203 MB, Romsdalen 19.4 s / 810 MB, every mesh byte-identical; code review round 1's one blocker, an empty land-cover layer crashing the write, fixed by `55cc3c3e` + `18442b27`, 378 net lines; code review round 2 APPROVED; merged 2026-10-12). Designed by `@architect`, 2026-10-11, on `81a551b3`; proposed by Ola 2026-10-05, "yes, roadmap row now, build after PR D"; PR D merged as #205, so the writer joins the encoders under `io/`). Measured on this design's probe: the quick check's Numedalslagen mesh (653,039 triangles) written in 4.0 s to 203 MB against 0.28 s and 29 MB for the binary `.vtk`, the stopgap 5.1 s; the Romsdalen slope mesh (3.1 million) in 21.5 s to 922 MB at 2.1 GB peak, the stopgap 28.7 s at 4.1 GB; the R-tree index is 60 % of the time; the same run writes the same bytes. Layers `triangles` (PolygonZ with code, name, colour, z_min, z_max, z_mean, area_m2, slope_deg), `lines`, `land_cover`, `land_cover_classes`, a `run` table with the run record, and the stored QGIS style; about 305 net lines, one PR; questions 1-5 answered with their defaults while Ola is away | `docs/increments/36-geopackage-output.md` | | — | A missing station number refused in plain words. In a reference-polygon file (`rasputin station-catchments --reference`), a feature whose `station` is null becomes the text "None" (`src_python/tin_engine/io/station_set.py@ed12512:89`). No station can match it, since a station number must look like `2.11.0` (`src_python/tin_engine/hydrography.py@ed12512:40`), so the polygon is dropped without a word and the station it belonged to is written with no comparison (`src_python/tin_engine/catchment_batch.py@ed12512:220`); two such polygons are refused as "station None appears twice". In a stations file a null number is refused, but in Pydantic's validation text rather than a sentence naming the file (`src_python/tin_engine/io/station_set.py@ed12512:79`). NVE's files have a number on every station, so only hand-made or third-party files are hit. The fix: both readers refuse a null `station`, naming the file; one or two lines. Plainer GeoJSON error messages in general are not planned until other people feed rasputin their own files | approved by Ola 2026-10-06; not designed (a small fix) | - | | — | A GeoTIFF whose coordinate-system GeoKeys hold garbage refused in plain words. The reader turns each code key into a whole number with `int()`, and a key stored as a double or as text can hold NaN, infinity or letters. Probed 2026-10-06 on hand-made files: NaN or text in `ProjectedCSTypeGeoKey` (3072) stops `rasputin mesh --dem` with Python's conversion message (for NaN, "Invalid value for --dem: cannot convert float NaN to integer"), which names neither the key nor the file, and infinity ends in a Python traceback (`OverflowError`) (`src_python/tin_engine/io/geotiff.py@ed12512:171`). The same holds for every other code key the reader reads: 1025, 4099, 3076, 3074, 3075, 2048 and 2054 (`src_python/tin_engine/io/geotiff.py@ed12512:310`, `src_python/tin_engine/io/geotiff.py@ed12512:176`, `src_python/tin_engine/io/geotiff.py@ed12512:339`, `src_python/tin_engine/io/geotiff.py@ed12512:352`, `src_python/tin_engine/io/geotiff.py@ed12512:362`, `src_python/tin_engine/io/geotiff.py@ed12512:499`, `src_python/tin_engine/io/geotiff.py@ed12512:423`). Worse, a fractional value is cut to a whole number first: 2.5 in 4099 is refused as 2, and 32633.5 in 3072 is read as EPSG:32633 without a word (`src_python/tin_engine/io/geotiff.py@ed12512:321`); 1.5 in 1025, 9001.7 in 4099 and 9001.5 in 3076 are likewise accepted as 1 and 9001 (probed by code review round 1 on this row's branch, exit 0). The parameter keys are already checked (`_one_number`, `src_python/tin_engine/io/geotiff.py@ed12512:454`). The fix: one check for every code key, a whole number or a refusal naming the key and its value. No real file is known to do this | approved by Ola 2026-10-06; not designed (a small fix) | - | | — | tifffile's warning about the NoData tag silenced. On a float32 DEM whose `GDAL_NODATA` (42113) is the lowest float32 written in 17 digits (`-3.4028234663852886e+38`), tifffile prints "parsing GDAL_NODATA tag raised ValueError: ... is not castable to float32" to stderr when the file is opened (`src_python/tin_engine/io/geotiff.py@ed12512:146`). It is noise: rasputin reads the tag itself and accepts the value, since it is that float32 exactly (`src_python/tin_engine/io/geotiff.py@ed12512:535`). The fix: silence that one tifffile message while the reader opens the file, leaving every other tifffile warning visible | approved by Ola 2026-10-06; not designed (a small PR) | - | @@ -66,7 +66,7 @@ increment that most needs a picture to check against | 16d | A record of the input polylines in the mesh. Today a constraint edge keeps only its type bits; the source feature, its attributes (a river's name or order, a road's class) and which edges belong to one polyline are lost, and where edges from two features merge only the union of their bits survives. Proposed: a feature table (one row per input feature: source ID and attributes) and, per constraint edge, the list of features it came from (a list, since a merged edge belongs to several). The noder already tracks each piece's input chain; the open part is the output: a per-edge list in `.vtk`/`.ply`, or a side file | proposed by Ola 2026-09-28; to be designed by `@architect` once Ola has said what it is for; after 16b | - | | 16e | Several `--features` sources in one mesh, so CORINE land cover and the hunting-parcel constraints go in together: `--features`, `--features-map`, `--features-crs` and `--features-layer` become repeatable, paired by position. Motivated by the Holleia hunting terrain, where the parcel boundary had to be breaklines alongside CORINE. The internal `FeatureRequest.sources` was already a tuple looped by `open_features`; this is the CLI that builds several sources | **shipped (#112)** (2026-09-29): repeatable options paired by position (R1 left-align, Ola's ruling — the Nth option applies to the Nth `--features`, a single option to source 0, a shorter list defaults: map `property`, crs/layer `None`); per-source degeneracy `BadParameter`s naming the 1-based index and path; the same file twice allowed (two layers/maps of one GeoPackage); cross-source coincident edges merge as within-source (R4); `FeatureSet.counts` per source; D1 refuses two different non-empty code systems in one mesh (defensive — every coded map today is CORINE, so it never fires). Net +41 production lines, one PR, no `@perf` | `docs/increments/16e-multi-features.md`, `docs/increments/16b-terrain-polygons.md` (Q4) | | 20c | Soft quality criterion: a penalty that each Steiner node or constraint split must pay for in angle gained, instead of 20's hard 25°; applied at the start and during DEM refinement; may split constraint segments when that improves the mesh. Ola's rulings on 20's C1-C3 | **designed** (`@architect`, 2026-10-06) in three PRs. Measured on Lagan and Numedalslagen: most slivers come from the quality start and the final check, which lack 20b's foot rule, not from refinement. **20c-1**: the foot rule on all three insertion paths, 20b's search widened to the neighbouring triangles; slivers (smallest angle under 1°) 1 598 to 679 on Lagan and 1 644 to 433 on Numedalslagen for +0.3 to +0.4 % triangles, worst angle 0.000412° to 0.0024° and 0.00040° to 0.00068°; **built and approved in code review round 2** (Ola, 2026-10-07: build, no push), 163 net lines, at those figures as measured by `@perf` on the built code; the whole run within 0.3 % of master's time, the 1 m benchmark 0.3 to 2 % faster after `@developer`'s fix `69f37d1c` (foot work only where a constraint line is within reach); refinement's one-point-at-a-time step is 7 % slower with feet on than off on both catchments but 3 to 8 % faster than master, judged under a revised limit, and Ola accepted that time (2026-10-07: the foot search stays in the one-point-at-a-time step); code review round 1 asked for the mutation record, which `@tester`'s mutation round now gives (4 of 16 planted faults survived, each ruled, and `@tester`'s added tests then killed them: all 16 killed, tests only); code review round 2 approved; **merged** as PR #207. **20c-2**: a quality point is skipped if it would not raise its neighbourhood's worst angle, and one that lies just beyond a constraint line splits the line instead (Ola said yes to the split, question 4, 2026-10-06); **built** (branch `worktree-soft-quality-2`), 157 net lines; measured by `@perf` on the built code: slivers 437 and 296, Numedalslagen's worst angle 0.0086°, 7.5 and 11.4 % fewer triangles than master, every gate passed; after `@developer`'s speed fix (same meshes) the whole run within 0.1 % of 20c-1's time and the 1 m benchmark accepted; refinement's one-point-at-a-time step takes 16 % and 4 % longer because it adds 17 % and 7 % more points, but each point costs no more, and Ola ruled it is judged per point (2026-10-07); the quality start is 23 % and 20 % slower (0.08 and 0.11 s), reported. Code review round 2 approved; **merged** as PR #214. **20c-3**: land-cover borders between polygons of one class dropped (on with a class map) and outlines simplified within a tolerance (a flag, off), as Ola ruled 2026-10-06 ("defaults on the 20c questions"); Ola also ruled the soft criterion on at gain 0 and quality during refinement out of 20c. 20c-3 simplifies with `coverage_simplify` alone (round 3: the 1 cm snap first broke Numedalslagen's land cover, 0.000006°); at 2 m it leaves 151 slivers on Lagan and 32 on Numedalslagen. The worst angle stays set by short input edges (on Lagan 0.0067°, on the 1 cm edge that closes a 1 cm wide gap between two CORINE polygons in the source data, whose nearest vertex the simplification drops; the gap itself caps any mesh of the input as given at 0.0071°). Ola ruled 2026-10-07 that such a gap is an input error to repair ("The edges should have been shared"), reversing a default the main session had taken: 20c-3 first joins borders of one land-cover file that come within 1 m of each other and fills narrower gaps (shapely 2.2's `coverage_clean`, on by default, `--features-repair`; 5 cm at first, 1 m by Ola's ruling 2026-10-08). Probed on the land cover: the slit is the only gap on either catchment and closes; land-cover edges under 10 cm fall from 356 to 5 and 464 to 1; about 1.1 s per catchment, and the coverage check the old design had (15.8 s) is dropped. Ola's outline rule (land-cover borders within 5 m of the catchment outline moved onto it, thin pieces there handed to the neighbour), measured on 20c-3's input: slivers 151 to 61 and 32 to 4, Numedalslagen's worst angle 0.012° to 0.83°, under 0.25 % more triangles, about 0.7 ha of land cover per catchment changing class; 20c-3 builds it on by default at 5 m (Ola, 2026-10-07; `--features-outline-snap 0` turns it off). Design review rounds 1 to 5: changes requested, answered; round 6 approved. 20c-3's design brought up to date 2026-10-08 (the repair; 30d's read region; a speed check before the push). 20c-3 **built** (branch `worktree-soft-quality-3`): shapely raised to 2.2, red tests then green `7334b82b`, 331 net production lines against about 150 (the outline rule alone 188 against about 70; under 700, one PR); `@architect` ruled the green step 2026-10-08: two tests to fix (one passed `--stats` twice, one checks labels against the unmoved input, so it runs with the outline rule off), the merge switch recorded as the text on/off like the other switches, and the rule's cuts placed along each edge's line so the read region does not move them; those done, and the mutation round on the outline rule; `@perf`'s timed check (2026-10-08): the run +30 % on Lagan and +107 % on Numedalslagen, the land-cover step 8.0 s and 8.8 s against the 1.1 s estimated, and Numedalslagen's outline-rule gate missed (15 slivers against at most 10). `@architect` ruled: the miss is a defect in how the rule cuts border edges (a cut can fall centimetres from an edge's own vertex), fixed in code; the step rebuilds and measures only the land cover that moved, with the same output. Both built (355 net production lines) and re-timed by `@perf`: Numedalslagen's outline-rule gate met at the designed figures (4 slivers, worst 0.83°), the mesh unchanged by the speed change, the land-cover step 5.92 s and 5.48 s, the whole run +17 % on Lagan and +54 % on Numedalslagen against master; with the defaults, slivers under 1° 437 to 180 and 296 to 158 against master, worst angle 0.0024° to 0.19° and 0.0086° to 0.63°. Ola accepted that time for 20c-3 (question 10, 2026-10-08); speeding the step up is its own row, below. The reported area of land cover moved was wrong (1 km² for 20 m² in a test case) when no vertex of a polygon's border stays put; fixed (356 net production lines). Code review round 1 asked for changes, ruled by `@architect` 2026-10-08: a lake inside a thin ring of another class could take the ring's class when the outline rule put the lake's border onto the outline (fixed in code, one word; no mesh change expected); the run record's line for that area reworded; and the gate's remaining checks (the repair together with the outline rule on the fixed code, shared borders still matching, no land-cover line left on the two same-class cuts, the slit's far corner) run by `@perf`. Those two code changes built (360 net production lines) and `@perf`'s checks run: the default meshes unchanged; the repair with the outline rule met on both catchments (59 and 4 slivers, none near the outline); shared borders all matched; the slit's far corner clean; and no land-cover line left on the two same-class cuts (26 m of border remains within 1 m of them, all between different classes in the source data, so the check's wording was corrected, not the code). Code review round 2 done. The repair's distance then probed on the built code at 5 cm, 25 cm and 1 m: 25 cm changes nothing, 1 m takes the slivers under 1° with the defaults to 26 on Lagan and 2 on Numedalslagen (from 180 and 158 at 5 cm), the worst angle to 0.73° and 0.83°, moving land-cover borders by under 1 m along about 45 km and 123 km of border; Ola set the default to 1 m (2026-10-08), built (three lines changed, 360 net production lines). Not re-timed by `@perf`: single runs give the land-cover step 5.98 s and 5.46 s and the whole run 21.62 s and 10.82 s. `@tester` brought one test docstring line to the 1 m default. Next `@reviewer`'s code review round 3 | `docs/increments/20c-soft-quality.md`; `docs/increments/20-start-quality.md` (Ola's rulings) | -| 35 | Speed up the land-cover clean-up step: 20c-3's clip, repair, merge by class and outline rule, the clean-up time `--stats` reports. `@perf`'s re-time (`41db7333`, repair then at 5 cm): the step 5.92 s on Lagan and 5.48 s on Numedalslagen, the whole run 17.64 → 20.59 s (+17 %) and 6.81 → 10.51 s (+54 %) against master; with the repair's default now 1 m (20c's M8, single runs, not timed by `@perf`): the step 5.98 s and 5.46 s, the whole run 21.62 s and 10.82 s. Where the time goes on Numedalslagen (20c's T3, `@perf`'s profile `4d52dec0` less the T2 fix): the outline rule about 2.4 s (its Python loop over ring edges, and `make_valid` and union of the big rebuilt parts), `coverage_clean` 1.2 s, the clip of the lines to the catchment 1.0 s, the clip to the read region 0.75 s. Leads from T3, not designed or measured: a vectorised ring loop, no whole-part repair and union for the big parts, no catchment clip for chains the rule already knows are inside (perhaps 1.5 to 2 s). An option, Ola's (2026-10-08: "fix the CORINE data itself, as a preprocessing step, for areas of interest"): repair the land cover once per area of interest and cache the repaired copy, so a run does not repeat the repair; its entry is in Ola's private ideas file, outside git ("Repair CORINE once per area of interest") | **built** (branch `worktree-cleanup-speed`; design `7da1d663` approved in one review round; red `c4130194`, green `e805d37f`, 23 net production lines, meshes byte-identical; mutation round `7621a99e` M1-M4 killed; `@perf`'s quick check `2cf224a6` ACCEPTED: Lagan features clip 11.16 -> 6.34 s, run 19.3 -> 14.5 s, all meshes byte-identical; next code review). Designed by `@architect`, 2026-10-11, on `@perf`'s profile `c7dba32e`: Numedalslagen's step 6.08 s of 9.06 s, Lagan's 7.17 s of 19.61 s plus 4.08 s reprojecting 15.9 million CORINE vertices before it): four exact changes, about 20 net lines, Python only, every output byte-identical: the cover polygons reprojected in one call split over up to 8 threads; `_untouched` widened to lines whose ends touch the outline (DE-9IM `T**F**FF*`); `covers` skipped where it holds; one outline tree per clean-up. Measured on the built patches: Lagan `features clip` 11.74 to 6.74 s, run 20.64 to 16.28 s; Numedalslagen 6.29 to 6.00 s; meshes byte-identical on both. Left out, with the reason measured: skipping the region and domain clips (GEOS rewrites ring order), the outline rule's overlays, `coverage_clean` (Ola's cached-repair idea). Two questions for Ola decided on their defaults while he was away (reprojection threads from the core count; ship with Numedalslagen's step near 6 s); code review round 1 APPROVED; next the push on Ola's yes | `docs/increments/35-cleanup-speed.md`, `docs/increments/20c-soft-quality.md` (T3, ruling 10) | +| 35 | Speed up the land-cover clean-up step: 20c-3's clip, repair, merge by class and outline rule, the clean-up time `--stats` reports. `@perf`'s re-time (`41db7333`, repair then at 5 cm): the step 5.92 s on Lagan and 5.48 s on Numedalslagen, the whole run 17.64 → 20.59 s (+17 %) and 6.81 → 10.51 s (+54 %) against master; with the repair's default now 1 m (20c's M8, single runs, not timed by `@perf`): the step 5.98 s and 5.46 s, the whole run 21.62 s and 10.82 s. Where the time goes on Numedalslagen (20c's T3, `@perf`'s profile `4d52dec0` less the T2 fix): the outline rule about 2.4 s (its Python loop over ring edges, and `make_valid` and union of the big rebuilt parts), `coverage_clean` 1.2 s, the clip of the lines to the catchment 1.0 s, the clip to the read region 0.75 s. Leads from T3, not designed or measured: a vectorised ring loop, no whole-part repair and union for the big parts, no catchment clip for chains the rule already knows are inside (perhaps 1.5 to 2 s). An option, Ola's (2026-10-08: "fix the CORINE data itself, as a preprocessing step, for areas of interest"): repair the land cover once per area of interest and cache the repaired copy, so a run does not repeat the repair; its entry is in Ola's private ideas file, outside git ("Repair CORINE once per area of interest") | **merged (#229)** (branch `worktree-cleanup-speed`; design `7da1d663` approved in one review round; red `c4130194`, green `e805d37f`, 23 net production lines, meshes byte-identical; mutation round `7621a99e` M1-M4 killed; `@perf`'s quick check `2cf224a6` ACCEPTED: Lagan features clip 11.16 -> 6.34 s, run 19.3 -> 14.5 s, all meshes byte-identical; design review round 1 and code review round 1 APPROVED). Designed by `@architect`, 2026-10-11, on `@perf`'s profile `c7dba32e`: Numedalslagen's step 6.08 s of 9.06 s, Lagan's 7.17 s of 19.61 s plus 4.08 s reprojecting 15.9 million CORINE vertices before it): four exact changes, about 20 net lines, Python only, every output byte-identical: the cover polygons reprojected in one call split over up to 8 threads; `_untouched` widened to lines whose ends touch the outline (DE-9IM `T**F**FF*`); `covers` skipped where it holds; one outline tree per clean-up. Measured on the built patches: Lagan `features clip` 11.74 to 6.74 s, run 20.64 to 16.28 s; Numedalslagen 6.29 to 6.00 s; meshes byte-identical on both. Left out, with the reason measured: skipping the region and domain clips (GEOS rewrites ring order), the outline rule's overlays, `coverage_clean` (Ola's cached-repair idea). Two questions for Ola decided on their defaults while he was away (reprojection threads from the core count; ship with Numedalslagen's step near 6 s); Numedalslagen's `features clip` at 68 % of its run ruled expected by Ola 2026-10-12 ("Rule it expected at 6 s, cached extract is the fix"; `docs/benchmarks/quick/hotspots.toml`): the cached repaired extract is the fix, when picked | `docs/increments/35-cleanup-speed.md`, `docs/increments/20c-soft-quality.md` (T3, ruling 10) | | — | Noder failure at a coarse snap spacing: `rasputin mesh --dem glo30 --cache ../rasputin_data/sweden_glo30_cache --out-crs EPSG:3006 --domain ../rasputin_data/sweden_smhi_svar/lagan_mouth_svar2022_3006.geojson --features ../rasputin_data/sweden_corine/lagan_clc2018_3035.geojson --features-crs EPSG:3035 --features-map corine --tolerance 10 --snap-spacing 0.1 --out lagan.vtk` (from the repository root, master `ed125121`) stops with "MalformedOutput. the noder produced output violating its own guarantees; this is our bug chain 319 does not tile the buffer", raised at `include/terrain/noding/noded_pslg_builder.hpp@ed125121:304`; `--snap-spacing 1` stops the same way, and the default (1 mm) runs. Found while measuring 20c's input coarsening, which does not need it (20c-3 simplifies in Python, before the noder); outside 20c | approved by Ola 2026-10-06 ("add the snap-spacing row"), not designed | `docs/increments/20c-soft-quality.md` (M4) | | — | Auto-catchment: the watershed upstream of a coordinate, computed from the DEM and handed to `--domain`, so a catchment no longer has to be supplied as a file (Ola, 2026-09-27: "not far into the future"). The textbook route is depression handling (Priority-Flood, Barnes, Lehman and Mulla 2014), D8 flow directions (O'Callaghan and Mark 1984) and accumulation, the pour point snapped to the strongest flow nearby, the upstream cells traced and their outline turned into a polygon; the literature check is `@architect`'s. Open for its design: whether it runs in the C++ core (a 10 m tile is 25 M cells); how a stair-stepped cell outline becomes a domain polygon, which meets input coarsening; **a requirement, not an option (Ola, 2026-09-29): "we will get an extreme amount of points in the catchment polygon. This must be taken into account, or the number of triangles will be overwhelming."** A cell outline has a vertex at every cell step (tens of thousands for a mid-sized catchment at 10 m, far more for Glomma), and every boundary edge is a constraint the mesh must honour; 16b measured what dense constraints cost (CORINE borders make the 10 m mesh 3.6x larger). The design must state how the outline is reduced, to what horizontal tolerance, and what it guarantees (a simple polygon, the pour point inside); the fine outline is defined on the DEM's node lattice (Ola, 2026-09-29: "Isn't the DEM points, really?"), but the reduced polygon's vertices need not be DEM nodes: off-node domain vertices are supported since 16 (Ola: "We have aleady established that the polygons and DEM don't need to match"), so vertices may be placed to keep the area; and the reduced polygon keeps approximately the fine one's area (Ola, 2026-09-29: "the resulting polygon has approximately the same area as the fine one"), since catchment area drives runoff volume. Literature to check: area-preserving polyline simplification (e.g. Bose et al. 2006; Kronenfeld et al. 2020, segment collapse); and that a real catchment crosses tile edges, so it needs gap 6 (a DEM in several tiles) first. Legacy has nothing on it (`grep -rliE "watershed|flow.?acc|flow.?dir|pour.?point|catchment" legacy` returns no files) | **shipped as increment 22** (#110, #111; 2026-09-29), in two PRs: PR 1, the fine catchment, net 619 lines; PR 2, the outline reduction, net 406 lines. Bygdin: 304.91 km² against NVE's 305.54 km² (−0.21 %), 99.1/99.3 % node overlap with NVE's polygon, 17 812 → 740 outline vertices at 20 m with the area kept, catchment in 6.5 s (`docs/benchmarks/2026-09-29/bygdin.md`). The reduction bounds the distance to the fine outline at its vertices only (measured on Bygdin: 19.92 m at 20 m). Parked for later (Ola, 2026-09-29): "It could be useful to have the option of some tolerance, doing more adaptive coarsening, but write it down and leave it for now": an enforced whole-outline tolerance as an option, which also belongs with 20c's input coarsening. Designed by `@architect`, 2026-09-29, Bygdin first, moved ahead of 20c for Ola's night run: a lake polygon (CORINE) as the seed, a Priority-Flood labelling the lake's catchment, a marching-squares outline, area-preserving segment collapse to `--outline-tolerance` (default twice the cell). Two PRs on one branch; defaults for Ola to confirm are marked in the file | `docs/increments/22-auto-catchment.md` | | — | Release hygiene: README and INSTALL rewritten for a new user; relicensed from GPL v3 to MIT, `Copyright (c) Expert Analytics AS` (Ola, 2026-09-29: "permissive, like MIT or similar"); `legacy/` removed from the tree and kept under the tag `legacy-archive`, with `tools/check_legacy_imports.py` and `@migration-expert`; `check_citations.py` resolves `legacy/` citations through the tag; `NOTICE.md` credits detria, pybind11, Catch2, Kartverket DTM10 and CORINE | **shipped (#114)** (2026-09-29); the wheel ships `LICENSE` (MIT) and `NOTICE.md`, checked by `@reviewer` | `docs/increments/release-hygiene.md` | diff --git a/docs/benchmarks/quick/hotspots.toml b/docs/benchmarks/quick/hotspots.toml index 152368ae..59185424 100644 --- a/docs/benchmarks/quick/hotspots.toml +++ b/docs/benchmarks/quick/hotspots.toml @@ -10,3 +10,17 @@ phase = "features clip" share = 40.7 date = "2026-10-07" ruling = """Asked: "The clip hotspot: features clip is now 41 % of Lagan's run. Record it as expected for now." Ola: "4: yes\"""" + +# Measured by @perf: features clip 6.000 s of 8.829 s (head, medians of 3, AC), +# docs/benchmarks/2026-10-11/cleanup-speed/README.md; the profile behind it is +# docs/benchmarks/2026-10-11/cleanup-profile/README.md (its three largest +# sub-steps are 80 % of the step: the outline rule 41 %, mostly GEOS union_all +# and make_valid; the two clips 21 %; coverage_clean 18 %). +# The fix Ola named is the cached repaired extract per area of interest (his +# private ideas file, "Repair CORINE once per area of interest"). +[[hotspot]] +case = "numedalslagen" +phase = "features clip" +share = 68.0 +date = "2026-10-12" +ruling = """Asked: "For the hotspots file, my proposal is to rule it expected at the current 6 s and let the cached-extract idea be the fix when you pick it." Ola: "Rule it expected at 6 s, cached extract is the fix\"""" diff --git a/docs/increments/26-land-cover.md b/docs/increments/26-land-cover.md index 4a102c5c..fae9bb10 100644 --- a/docs/increments/26-land-cover.md +++ b/docs/increments/26-land-cover.md @@ -1,6 +1,6 @@ # Increment 26: land cover for the São Francisco basin — water bodies as constraints, class fractions per triangle -Status: **26a built** on branch `worktree-mapbiomas-26a` (red `cb5f254b`, `@tester`, 48 tests; green `8eec354b`, `@developer`, 163 net production lines by `python3 tools/count_loc.py de9b6c03 HEAD` against the estimate of 170; no mutation round, section 9; section 13 is the as-built record; `@perf`'s quick check `38f86930` ACCEPTED; code review rounds 1 and 2, APPROVED; next the push on Ola's yes). 26b onward not built. The design: **by `@architect`, 2026-10-03, at Ola's request; ruled by +Status: **26a merged (#232, 2026-10-12)** from branch `worktree-mapbiomas-26a` (red `cb5f254b`, `@tester`, 48 tests; green `8eec354b`, `@developer`, 163 net production lines by `python3 tools/count_loc.py de9b6c03 HEAD` against the estimate of 170; no mutation round, section 9; section 13 is the as-built record; `@perf`'s quick check `38f86930` ACCEPTED; code review rounds 1 and 2, APPROVED; master merged in as `5f28292c` for a docstring conflict with #231; 26b red in progress on `worktree-mapbiomas-26b`). 26b onward not built. The design: **by `@architect`, 2026-10-03, at Ola's request; ruled by Ola, 2026-10-04** (all eleven questions; "Ruled by Ola, 2026-10-04", below, and each question marked). Q8 was ruled against the recommendation: triangles inside a lake carry the raster's own mix, nothing is forced to diff --git a/docs/increments/35-cleanup-speed.md b/docs/increments/35-cleanup-speed.md index 0427acc3..117f6b8b 100644 --- a/docs/increments/35-cleanup-speed.md +++ b/docs/increments/35-cleanup-speed.md @@ -1,11 +1,11 @@ # Increment 35 — the land-cover clean-up step, made faster -Status: **built** on branch `worktree-cleanup-speed` (design `7da1d663`, review +Status: **merged (#229, 2026-10-11)** from branch `worktree-cleanup-speed` (design `7da1d663`, review round 1 approved `7218dfb7`; red `c4130194` (`@tester`), green `e805d37f` (`@developer`, 23 net production lines by `python3 tools/count_loc.py c7dba32e HEAD`: `crs.py` +14, `feature_input.py` +9), mutation round `7621a99e` (M1-M4 all killed, one count-based test added for M3); section 11 is the as-built -record; `@perf`'s quick check `2cf224a6` ACCEPTED, section 11; code review round 1 APPROVED, Review section). Next: the push on Ola's yes. Master +record; `@perf`'s quick check `2cf224a6` ACCEPTED, section 11; code review round 1 APPROVED, Review section; Numedalslagen's 68 % hotspot ruled expected by Ola 2026-10-12, `docs/benchmarks/quick/hotspots.toml`). Master `c7dba32e` is `@perf`'s profile commit (`docs/benchmarks/2026-10-11/cleanup-profile/README.md`). Every `file:line` below is pinned to `c7dba32e`. Every figure was measured on this worktree's @@ -368,6 +368,10 @@ venv's numpy 2.5.4 differed from its 2.5.3). Hotspots: Lagan features clip Numedalslagen features clip 68 %, unruled, GEOS overlay time (section 4). Decided on defaults while Ola was away: the 68 % is recorded as expected here; the `hotspots.toml` entry and the Lagan 40.7 % ruling wait for Ola's word. +Ruled by Ola 2026-10-12 ("Rule it expected at 6 s, cached extract is the fix"): +the entry is in `docs/benchmarks/quick/hotspots.toml` at 68.0 %; the fix is the +cached repaired extract per area of interest, when picked. Lagan's 43.6 % is +within 10 points of its 40.7 % ruling, so the tool is quiet; no new entry. Lesson: a stored quick-check baseline goes stale when a merge changes outputs without a refresh (32 merged without one). @@ -376,3 +380,7 @@ without a refresh (32 merged without one). Design review round 1, 2026-10-11, @reviewer (7da1d663, prose only, 0 counted lines): APPROVED. Every figure in sections 2, 4 and 7 matches /Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/docs/increments/35-probes/README.md@7da1d663 and the profile README; every feature_input.py and crs.py citation quotes the pinned code; both mesh digests recorded; check_citations clean. The relate pattern `T**F**FF*` read in words: the line's interior meets the domain's interior and neither its boundary nor its exterior, and no end lies outside; a line touching the outline only at an interior vertex is excluded, which is necessary since GEOS nodes it there. One correction applied in this recording commit: /Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/docs/increments/35-cleanup-speed.md@7da1d663:234 claimed `along-the-outline` kills M1; it does not (measured, GEOS 3.14.1), M1 is killed by `along-then-inside`, `domain-vertex-on-the-line` and `touching-a-hole`. Nit applied: the comprehension cited at `:450` is at `:448`. Ideas for @tester: add `end-on-a-hole` to test 3 (new rule True, GEOS returns the line unchanged, measured); M4 cannot be killed on a one-core runner, so the mutation round stays local. Lesson: a claimed mutant kill is checked by running the fixture table against the mutant. Code review round 1, 2026-10-11, @reviewer (3e6f87eb..4de504af, 23 net counted lines by count_loc.py from both 3e6f87eb and c7dba32e, estimate about 20; not pushed, no CI yet): APPROVED. The green diff touches only the two production files and does exactly the four section-3 changes plus the three section-11 pins; the pattern is `relate_pattern(domain, line, "T**F**FF*")` with the domain first (/Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/src_python/tin_engine/feature_input.py@4de504af:255); chunking cannot change bits (same Transformer per thread, `np.array_split` keeps order, PROJ maps point by point, arrays under 500k take the old single call: /Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/src_python/tin_engine/crs.py@4de504af:57-72); one threading.local per closure, the pool opened and closed inside `apply`; every mutant M1-M4 names an existing test and fixture row (/Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/tests/python/test_feature_input.py@4de504af:1274-1295, 1680-1717; /Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/tests/python/test_crs.py@4de504af:219-237); no red-step scaffolding; the acceptance README's figures equal the raw medians in its base and head records; check_citations clean; the two test files 291 passed. Correction applied in this recording commit: the ROADMAP row's stale "next @tester" phrase. Non-blocking note: `_finite` on cover polygons runs after the loop (/Users/skavhaug/projects/rasputin/.claude/worktrees/cleanup-speed/src_python/tin_engine/feature_input.py@4de504af:381-386), so when a cover polygon and a later line feature are both unmappable, the fid named is the line's. + +Ruling-record review round 1, 2026-10-12, @reviewer (`origin/master..bc7639fb`, 0 counted lines): CHANGES REQUESTED, two prose claims, applied in this recording commit: /Users/skavhaug/projects/rasputin/.claude/worktrees/hotspot-ruling/ROADMAP.md@bc7639fb:69 said "code review rounds 1 and 2 APPROVED" for 35, which had one design round and one code round; /Users/skavhaug/projects/rasputin/.claude/worktrees/hotspot-ruling/docs/benchmarks/quick/hotspots.toml@bc7639fb:16 said "about 85 % GEOS calls", the profile README gives the three largest sub-steps 80 % of the step with about 60 % in GEOS proper. Verified: PR merges and dates, every hash, 6.000 s of 8.829 s = 68.0 %, Lagan 43.6 under 40.7 + 10 (`tools/bench_quick.py:166`), TOML loads and the shipped-file tests pass, check_citations exit 0. Suggestion applied: increment 36's Status line said "built ... Next: the push", now merged (#231). Lesson: a percentage in a comment names the README line it sums from. + +Ruling-record review round 2, 2026-10-12, @reviewer (`origin/master..0dfca860`, 0 counted lines): CHANGES REQUESTED, one blocker, the reviewer's exact wording applied in this recording commit: /Users/skavhaug/projects/rasputin/.claude/worktrees/hotspot-ruling/docs/increments/36-geopackage-output.md@0dfca860:10 said #231 merged "on top of #229 and #230"; by the parents (`git log -1 --format=%P fa6e4855`) it sits on #229's merge and #230's `a1aec506` sits on it, so "on top of #229, ahead of #230". Round 1's wordings verified against their sources; 41/21/18 % = 80 % matches the profile README; #231 merged 2026-10-12T05:01Z; TOML parses, 11 hotspot tests pass; check_citations exit 0. Lesson: a merge-order claim is checked with the commit's parents, not the merge timestamps. Pushed on the reviewer's wording without a third round, as the recording-commit practice allows (retrospective 2026-10-12, P1). diff --git a/docs/increments/36-geopackage-output.md b/docs/increments/36-geopackage-output.md index a88e1127..427bf30a 100644 --- a/docs/increments/36-geopackage-output.md +++ b/docs/increments/36-geopackage-output.md @@ -1,13 +1,13 @@ # Increment 36: GeoPackage output -**Status: **built** on branch `worktree-gpkg-out` (design `01118660`, fixes +**Status: **merged (#231, 2026-10-12)** from branch `worktree-gpkg-out` (design `01118660`, fixes `0233d1e3`, design review rounds 1 and 2 recorded; red `87f41c3d` (`@tester`, 42 tests), green `c59246e5` + `0943a7a5` (`@developer`), layering rows `4e7901ee`, mutation round `dddce8e8` (ten mutants, all killed, record in its commit message), fix-up `7085dddf` (the `.part` housekeeping moved to the CLI); 377 net production lines by `python3 tools/count_loc.py 81a551b3 HEAD` (the merge base; `3e6f87eb` gives the same count) against the design's about 305; section 11 is the as-built record; `@perf`'s -acceptance `57c25074` ACCEPTED). Code review round 1 CHANGES REQUESTED (one blocker, Review section), fixed: red test `55cc3c3e` (`@tester`), guard `18442b27` (`@developer`), 378 net lines; code review round 2 APPROVED. Next: the push on Ola's yes, then a merge of master (#229, increment 35) before the queue. +acceptance `57c25074` ACCEPTED). Code review round 1 CHANGES REQUESTED (one blocker, Review section), fixed: red test `55cc3c3e` (`@tester`), guard `18442b27` (`@developer`), 378 net lines; code review round 2 APPROVED. Merged by the queue on top of #229, ahead of #230, 2026-10-12. ## 1. What Ola asked