Increment 38: --threads and --memory-budget as settings, no core count hardcoded

INSTALL.md

@@ -194,6 +194,13 @@ rasputin mesh --dem glo30 --domain catchment.geojson --out-crs EPSG:32723 \
     --tolerance 5 --out catchment.vtk             # offline from here
 ```
 
+One more environment variable: `RASPUTIN_THREADS` is the default thread
+count for `mesh`, `catchment` and `station-catchments` (0 or unset means every
+core; `--threads N` overrides it), and inside a container the core count and
+the memory follow the container's own limits (`cpu.max`, `memory.max`).
+`--memory-budget 8G` caps the DEM window one run assembles; the default is
+half the machine's memory.
+
 `glo30` is the Copernicus GLO-30 (30 m, worldwide) and `anadem-v1` the ANADEM
 terrain model of South America. Both are in geographic coordinates, so `mesh`
 needs `--out-crs`; its refusal suggests one. The source's cache directory

README.md

@@ -50,8 +50,9 @@ its CORINE land-cover class. Elevation © Kartverket (CC BY 4.0); land cover
 Install first ([INSTALL.md](INSTALL.md)); the example needs the `codecs`
 extra, because the committed DEM tile is LZW-compressed. The install is
 bounds-checked by default; INSTALL.md's "Bounds checks" section says how to
-build an unchecked copy for heavy runs. From the repository
-root:
+build an unchecked copy for heavy runs. Every run uses every core unless
+`--threads N` or the environment variable `RASPUTIN_THREADS` says otherwise
+(0 means every core). From the repository root:
 
 ```sh
 # A 50 km DTM10 tile, meshed inside a quarter disc of radius 30 km,

ROADMAP.md

@@ -58,6 +58,7 @@ increment that most needs a picture to check against
 | 32 | Land-cover borders simplified within a band, each class keeping its area exactly (Ola, 2026-10-08: "we need to implement is a CORINE simplification, similar to what we do on the outline of the auto-catchments"): the land-cover coverage cut into borders between junctions, each border reduced by the area-preserving segment collapse of the catchment outline (`reduce_ring`), with junctions and the domain outline fixed, a crossing test and a side test across all borders, and a band no border moves past (two-sided, proved by anchors on the source border). `--features-tolerance` becomes that band, default 50 m (half CORINE's positional accuracy), replacing GEOS's coverage simplifier; start angle 15° when land cover is simplified (band above 0), 25° otherwise. Prototype on the German fused catchment: 5.7 times fewer triangles at 50 m vertical tolerance | **merged (#222, 2026-10-11)** (green `82c0a7d4`, 519 net production lines, one PR; mutation round `30f939b6`, one mutant recorded as not killed); Ola ruled all four questions (2026-10-08); code review round 1 asked for changes (comments, docstrings and `project_structure.md` the change made false), fixed by `4d4f0411` and the docs commit after it; code review rounds 1-3 done (round 3 approved); `@perf`'s short timing check found the runs about 70 % slower (the outline rule on borders clipped to the catchment) and the worst angle far lower (the simplifier placing points millimetres from junctions); fix built (section 15: outline rule first, a 1 m clearance in the simplifier; green `05c8f73a`, 573 counted lines in all; mutation round M7 to M13 killed); its quick check `b192a2c6`: Numedalslagen −11 %, Lagan −1.3 %, outline rule and moved area as before, triangles −42 %; questions 5 and 6 decided on default while Ola was away, reversible; fix code review round 1 asked for prose fixes and the test file's red-step staging removed; fix code review round 3 approved; next the push on Ola's yes | `docs/increments/32-landcover-simplify.md` |
 | 36 | GeoPackage output, so QGIS opens a mesh as it opens any vector layer: `rasputin mesh --out X.gpkg`, chosen by the suffix of `--out` as `.vtk` and `.ply` are (Ola, 2026-10-05: "Is the vtk_to_phkg.py the way to go, or as a direct output from rasputin? I think the latter, and it should go into the roadmap and eventually into the codebase."). Proposed content: a `triangles` layer, one PolygonZ per triangle in the run's CRS (z at each of the three corners, as the stopgap writes), with the attributes the run already knows: land-cover code and class name (the code system's names, as `rasputin palette` has them), mean z, plan area; a layer of the constraint polygons (the `--features` polygons clipped to the domain, with their code); optionally a QGIS style for land cover stored in the file (the `layer_styles` table), so the triangles open coloured. Land-cover colour: when the mesh carries CORINE classes, each triangle and each constraint polygon also gets its class's colour as a `#rrggbb` text column, so a reader can colour by the column without the stored style (in QGIS, a data-defined fill colour). The Graz file of 2026-10-05 carries `clc_color` on its `triangles` and `corine` layers; its stored styles categorise on `clc_code` and do not read the column. The colours are those of `rasputin palette` (`src_python/tin_engine/palettes.py`), the same 26 the Graz file carries: natural colours, as Ola chose over the official CLC legend in 16c's ruling 2 (that legend paints peat bogs blue and the sea almost white). **One attribute is not in the mesh yet**: the ID of the source polygon each triangle lies in (a sub-catchment cell, a parcel). Today only a class code labels triangles, one code system per mesh (16c, 16e's D1); the stopgap finds the cell by a centroid lookup outside rasputin. It needs a per-triangle array from a polygon source's ID property: its own small step, done just before this writer, because the `.vtk` and `.ply` outputs gain the attribute too (Ola, 2026-10-05, accepting the default: "defaults on the cell ID"). Written with the standard library's `sqlite3` beside the existing reader `src_python/tin_engine/io/geopackage.py`, which can read it back in the tests; no GDAL, OGR or Fiona (CLAUDE.md §2). Python only: the C++ core is unchanged and never sees the file (the I/O boundary). Prior art and working spec: Ola's stopgap `vtk_to_gpkg.py`, outside the repository (`rasputin_data/lands/scripts/`), which converts a mesh `.vtk` with `sqlite3`, NumPy, Shapely and PyProj, no GDAL, and its first use, the Mur above Graz (2 451 734 triangles, 10 m DEM, tolerance 10 m, 2026-10-05): **the GeoPackage was 677.9 MB against 110.6 MB for the binary `.vtk`**; the whole stopgap run took 39 s and 4.1 GB peak, and that includes the centroid cell lookup and the CORINE clipping, so it is not the cost of the GeoPackage encoding alone: the design measures the writer on its own. A GeoPackage row holds the triangle's whole ring (four corners, the first repeated) and a bounding box, and the file adds a spatial index, where `.vtk` stores each vertex once. Questions for the design: file size (keep the spatial index or not; at most an opt-in variant of 2D polygons with mean z as an attribute, proposed default: full PolygonZ; writing in chunks to bound memory); the CRS entry for a CRS with no EPSG code; where the run record (increment 25) goes; whether the constraint lines get a layer of their own | **merged (#231)** (branch `worktree-gpkg-out`: red `87f41c3d`, green `c59246e5`, fix-up `7085dddf`, 377 net production lines, mutation round `dddce8e8` ten of ten killed; `@perf`'s acceptance `57c25074` ACCEPTED: Numedalslagen 3.93 s / 203 MB, Romsdalen 19.4 s / 810 MB, every mesh byte-identical; code review round 1's one blocker, an empty land-cover layer crashing the write, fixed by `55cc3c3e` + `18442b27`, 378 net lines; code review round 2 APPROVED; merged 2026-10-12). Designed by `@architect`, 2026-10-11, on `81a551b3`; proposed by Ola 2026-10-05, "yes, roadmap row now, build after PR D"; PR D merged as #205, so the writer joins the encoders under `io/`). Measured on this design's probe: the quick check's Numedalslagen mesh (653,039 triangles) written in 4.0 s to 203 MB against 0.28 s and 29 MB for the binary `.vtk`, the stopgap 5.1 s; the Romsdalen slope mesh (3.1 million) in 21.5 s to 922 MB at 2.1 GB peak, the stopgap 28.7 s at 4.1 GB; the R-tree index is 60 % of the time; the same run writes the same bytes. Layers `triangles` (PolygonZ with code, name, colour, z_min, z_max, z_mean, area_m2, slope_deg), `lines`, `land_cover`, `land_cover_classes`, a `run` table with the run record, and the stored QGIS style; about 305 net lines, one PR; questions 1-5 answered with their defaults while Ola is away | `docs/increments/36-geopackage-output.md` |
 | 37 | An MCP (Model Context Protocol) server over the rasputin commands, so an external agent (Codex, Claude Desktop, ChatGPT, Mistral) discovers and calls rasputin as tools instead of reading its help text: `rasputin mcp` serves over stdio, one tool per command with a JSON schema derived from the Typer command's own parameters (one source of truth, so schema and CLI cannot drift), each call run as a child `python -m tin_engine <command>` so a cancel kills it and a crash ends only the child, with the exit code, output paths, run record and `--stats` returned as data and a refusal returned as a structured error carrying the CLI's plain words and exit code (Ola, 2026-10-11: "my working name for what's we're doing is intent-driven triangulation. This means accessing the tools using agents and intent"; "Add an MCP server to the ideas file after GeoPackage"; 2026-10-12: "Rule the mcp SDK: yes, use it"). Built on the official `mcp` Python SDK 2.3 as an optional extra `rasputin[mcp]`; stdio first, Streamable HTTP later or never; no Tasks extension (the SDK has none server-side); a jobs cap, default 1; the GeoPackage output (36) is the mesh an agent reads | **built and review-approved** (branch `worktree-mcp-server`: design `b990e38c` approved in three review rounds, red `ee5b5d52` 62 tests, green `851592ba`, three code review rounds with a README section, an `error`-field fix and a help-text escape, master merged in as `030fd32b`, 328 net production lines against 260; `dev` pulls the `mcp` extra; next the push on Ola's yes). Designed (`@architect`, 2026-10-12, on `b4a81c0d`): about 250 counted lines, one PR; nine decisions with defaults for Ola (section 6); measured by the design's probe: a tool call costs 0.25 s over the command itself (the child's import of `tin_engine.cli`), the 1 m benchmark mesh 0.72 s through the tool, a cancelled child dead within 0.5 s over stdio and in process, unknown and mistyped arguments refused before any child runs; no refine or mesh code, so no `@perf` run | `docs/increments/37-mcp-server.md` |
+| 38 | Threads and memory as settings, the first fix from the generality audit (Ola, 2026-10-12: "the number of cores should of course not be hardcoded"): `--threads N` on `mesh`, `catchment` and `station-catchments` (0 = every core; `RASPUTIN_THREADS` is the default when the option is absent; neither set = every core, a container's CPU quota counting), carried as one `Resources` model built once in `cli.py` to every `_core` call that takes `threads` (five: `refine`, `refine_strip`, `refine_points`, `SlopeTolerance`, `class_areas`) and every Python pool (three: the reprojector, whose fixed cap of 8 goes, the target grid's resample and check points, the block decoder); the stats file and run record print the count used; `--memory-budget 8G` caps the mosaic and flood windows, default half the machine's memory with a container's cgroup limit (`memory.max`, v1 `memory.limit_in_bytes`) counting as the machine's; the unused `decompose.py` deleted with its test. Python only, no C++ change, about 60 net lines | **built** (branch `worktree-threads`: design `9e0d60c2` approved in two review rounds; red `4b9c0dfe` 110 tests, green `b252a70d`, test fixes `9de3212c`, 96 net production lines against 60; quick check NO CHANGE; code review two rounds, APPROVED; next the push on Ola's yes). Designed by `@architect`, 2026-10-12, as the generality audit's first fix | `docs/increments/38-threads-and-memory.md` |
 | 39 | Six Swedish river systems meshed sub-catchment by sub-catchment in one night (Ola, 2026-10-12: "Yes, do the swedish catchments, high-pri."; re-scoped the same day to "have each (sub)catchment TINed by tomorrow morning"): every SMHI SVAR 2022 sub-catchment polygon of the Ljungan, Indalsälven, Ångermanälven (with Faxälven, Fjällsjöälven and Åseleälven) and Umeälven — 5,338 polygons, 98,095 km² — meshed by `rasputin mesh` from the polygon as the outline, on GLO-30 with CORINE land cover at 10 m, as the Lagan quick-check case does; the 31 coarser BARO areas first, then the finest level, resumable, largest first; a table and a map per system under `docs/benchmarks/2026-10-13/se2-catchments/`. No production code: a runner script under `docs/benchmarks/`. Measured: 3.1 s per median unit (12 km²), 7.4 s per 1,413 km² BARO area; projected 4.6 h serial for the finest level, under 20 min for BARO | **built, the batch done** (branch `worktree-se2-catchments`: runner `d06a601a` by `@perf`; 5,338 finest SVAR units, 31 BARO areas and 4 whole systems meshed on GLO-30 with CORINE at 10 m, 0 refusals, finished 2026-10-12 18:05 UTC; record `51303179`; two review rounds; next the push on Ola's yes). Ola ruled the five questions "Yes to all" on 2026-10-12; the SVAR data (169 MB) and the GLO-30 blocks for all four systems fetched | `docs/increments/39-se2-catchments.md` |
 | 41 | Land-cover class maps and palettes as data files (generality audit rows B1, B2, B3, B6, B8; Ola, 2026-10-12: "4: yes" and "On 3: Probably, but we should have the option of override"): one TOML file per map (attribute, classes with code, label, colour and water flag, edges, notice, licence, text-to-code table, extra edge names, positional accuracy), read with `tomllib`; `--features-map` and `rasputin palette` take a shipped name or a path; CORINE's three maps and a MapBiomas Collection 11 map shipped in the package; the clean-up defaults derived as repair s/100, simplification s/2, snap min(s/20, 50) with s = max(accuracy, DEM cell), giving CORINE's 1, 50 and 5 m, the options kept as overrides; every CORINE mesh byte-identical | **built** (branch `worktree-class-maps`: design approved in two review rounds; red `19b9b62a` 119 tests, green `59779b35`, layering `34cfe2bc`, 114 net production lines against about 100; the quick check's meshes unchanged on every case; code review two rounds, APPROVED; next the push on Ola's yes) | `docs/increments/41-class-maps-as-data.md` |
 | — | A missing station number refused in plain words. In a reference-polygon file (`rasputin station-catchments --reference`), a feature whose `station` is null becomes the text "None" (`src_python/tin_engine/io/station_set.py@ed12512:89`). No station can match it, since a station number must look like `2.11.0` (`src_python/tin_engine/hydrography.py@ed12512:40`), so the polygon is dropped without a word and the station it belonged to is written with no comparison (`src_python/tin_engine/catchment_batch.py@ed12512:220`); two such polygons are refused as "station None appears twice". In a stations file a null number is refused, but in Pydantic's validation text rather than a sentence naming the file (`src_python/tin_engine/io/station_set.py@ed12512:79`). NVE's files have a number on every station, so only hand-made or third-party files are hit. The fix: both readers refuse a null `station`, naming the file; one or two lines. Plainer GeoJSON error messages in general are not planned until other people feed rasputin their own files | approved by Ola 2026-10-06; not designed (a small fix) | - |

docs/increments/23-basin-scale.md

@@ -3270,7 +3270,7 @@ LOC was 598 added and 6 removed, 592 net, against an estimate of 373. That is +6
 
 **23b, code review, round 8, 2026-10-04.** Range `4541e38..0a3187b` plus `4541e38` (4541e38 round 7 recorded and `20-start-quality.md` states the 23b rule; 0a3187b @perf's acceptance at the merged head). Verdict: CHANGES REQUESTED, closed in the commit that records this round. LOC: 0 this round (no production file in the range), 249 net for the PR against about 285. Round 7's blocker is closed: `20-start-quality.md:608-610` now states the rule, which matches `include/terrain/mesh/quality.hpp@ed125121:64-65,143,182` and `include/terrain/refinement/refine.hpp@ed125121:321-322`. @perf's verdict at `4541e38` against master `d20126b` is ACCEPTED (`docs/benchmarks/2026-10-04/23b-merged-acceptance.md`). The pooled medians, run ranges and changes were recomputed from the eight `raw.tsv` files and match: +0.8 % tile and +0.2 % quarter at 1 thread, -1.7 to +1.7 % over 2 to 20 threads. Each run's commit, module hash, `bench.py` version and AC power state match the doc, and the mesh hashes and quality are identical in all eight runs. `check_citations.py` passes. Blocking: the status line (`23-basin-scale.md:9-10`) and ROADMAP row 23 still said the merged-head acceptance was "owed"; both now say it is recorded. Merge-ready waits on CI: #162's remote head is still `4cd28dc`, which has never had a CI run; not pushed.
 
-**23c-1, code review, round 1, 2026-10-04.** Range `aacf37c..1426245` (red 58fb1f1, 6dd4ffe, 714b0b3; rulings 6ca9a8c; green 3ffae13; rulings 6ce8e57; red b9f154a; green 1426245), stacked on 23b. Verdict: CHANGES REQUESTED. LOC: 154 net (`bindings/core.cpp` 30, `_core.pyi` 3, `decompose.py` 53, `features.py` 3, `io/mesh_index.py` 65) against about 215. No @perf run needed: nothing under `include/` changed, the binding only copies. The loop-bound comment's argument is sound; the binding's GIL and refusals match `refine_points`'s (stricter on negative indices); `project_structure.md` and docstrings match the code. Blocking: (1) "HOW THIS FILE GOES RED" paragraphs in `test_core_mesh_arrays.py:22-23`, `test_decompose_partition.py:27-28`, `test_mesh_index_conformity.py:21-22`; (2) ROADMAP row 23 (still "23c … 430", "23b … in review") and this file's status paragraph ("23b onwards not implemented"). Suggestions: say "integer indices" in `indexed_mesh`'s docstring; a ValueError for `rows == 0` in `partition()`; plain imports for the `dec`/`mi` fixtures. Whichever of 15f-3 and 23c-1 merges second renumbers `tests/python/test_features.py@1426245:583`'s citation. Not pushed; no CI.
+**23c-1, code review, round 1, 2026-10-04.** Range `aacf37c..1426245` (red 58fb1f1, 6dd4ffe, 714b0b3; rulings 6ca9a8c; green 3ffae13; rulings 6ce8e57; red b9f154a; green 1426245), stacked on 23b. Verdict: CHANGES REQUESTED. LOC: 154 net (`bindings/core.cpp` 30, `_core.pyi` 3, `decompose.py` 53, `features.py` 3, `io/mesh_index.py` 65) against about 215. No @perf run needed: nothing under `include/` changed, the binding only copies. The loop-bound comment's argument is sound; the binding's GIL and refusals match `refine_points`'s (stricter on negative indices); `project_structure.md` and docstrings match the code. Blocking: (1) "HOW THIS FILE GOES RED" paragraphs in `test_core_mesh_arrays.py:22-23`, `tests/python/test_decompose_partition.py@5d2e706d:27-28`, `test_mesh_index_conformity.py:21-22`; (2) ROADMAP row 23 (still "23c … 430", "23b … in review") and this file's status paragraph ("23b onwards not implemented"). Suggestions: say "integer indices" in `indexed_mesh`'s docstring; a ValueError for `rows == 0` in `partition()`; plain imports for the `dec`/`mi` fixtures. Whichever of 15f-3 and 23c-1 merges second renumbers `tests/python/test_features.py@1426245:583`'s citation. Not pushed; no CI.
 
 **23c-1, code review, round 2, 2026-10-04.** Range `5216cb3..0679950` (c4ec906 test headers, 9633282 status paragraph, ROADMAP row 23 and point 4, 7e5fe0a and e5881a5 the empty-window tests, 0679950 the refusal); whole PR `aacf37c..0679950`. Verdict: APPROVED. LOC: 156 net against about 215. Both blockers closed; the empty-window suggestion taken (point 4, ten cases); citations hold; no @perf run needed. Nit: the status paragraph and ROADMAP say 154 lines built; update to 156 at merge. Not pushed; no CI.