Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
6c46773
Teach the UK geography identity kernel the residential split branch
juaristi22 Oct 4, 2026
323003b
Leave the nation-native alias codes out of the scratch dataset the me…
juaristi22 Oct 4, 2026
0fb4680
Collect garbage after every per-clone engine block in the measures loop
juaristi22 Oct 5, 2026
c957f89
Compile the dense role's national problem per engine block, never on …
juaristi22 Oct 5, 2026
0af677d
Guard a compiled row's entity axis with one shared array, not a per-r…
juaristi22 Oct 5, 2026
01b8640
Refuse the compiled-row axis test's foreign frame on a valid axis
juaristi22 Oct 5, 2026
4f20ca9
Hand the terminal gates' diagnostics the holdout in the schema's shape
juaristi22 Oct 6, 2026
959e698
Let the diagnostics schema admit the graph's holdout artifact fields
juaristi22 Oct 6, 2026
ab6bee6
Scale each engine block's weights to the pool so weight-share formula…
juaristi22 Oct 6, 2026
061ac3d
Evaluate the register gates in the dense battery as the certifier doe…
juaristi22 Oct 6, 2026
488f4a5
Write the three area codes under the consumers' names at the export b…
juaristi22 Oct 7, 2026
486ece1
Warm-start the size selection's L0 budget search at a known penalty
juaristi22 Oct 7, 2026
d2f8b19
Fix issues from review round 1: exact blocks verify their allocation …
juaristi22 Oct 7, 2026
736719a
Re-cut the pins the solver edit moved; verify the allocation keys aga…
juaristi22 Oct 7, 2026
6c27a68
Re-cut the US spec digest the pool tool's run config pins
juaristi22 Oct 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK geography identity kernel now lists `cgt_residential_split` among its structural branches, between the CGT incidence clone and the geographic pool expansion. The population node had keyed residential arms (microcosm#1063 item 7) on that branch since the stage landed, but the kernel's roster never learned it, so `--release-role dense` refused every spine carrying a residential arm with "unknown structural branch" (the main-head spine of 2026-10-04 carried 2,735). Arms now key their declared path; every other household's key is unchanged.
1 change: 1 addition & 0 deletions changelog.d/1115-dense-battery-register-gates.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The dense build's terminal gate battery now evaluates the register gates the way the release certifier does. The spine checkpoint publishes its build state (household weight kind, period, mass log) as a `spine_build_state` artifact and the terminal gate hands it to `uk_release_input_coverage` as the build-state frame, so the stage families are checked against the spine's importance weights rather than the calibrated release frame (every required family had failed on weight kind alone). `uk_degenerate_release_surface` skips the columns the release boundary drops before writing (`UK_RELEASE_EXPORT_DROPPED_COLUMNS`) and records them, instead of reporting a column the exported H5 never carries. The dense gate context computes the CGT projection artifact the `uk_cgt_projection_entrants` fence needs, as the national calibration kernel does.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Per-clone engine resolution (`--engine-blocks K`) now scales every block's engine household weights by pool mass / block mass before the policyengine-uk engine loads it, so formulas that allocate a national total by weighted share (corporate land value, shareholding, consumption share) see the pool's denominator instead of handing each block the whole aggregate (the K-times land-value artefact behind the #736 erratum). The measures receipt records the representation (`engine_population_representation`: the factor per block and whether the blocks are identical copies of one another, which makes the scaling exact), and the block-sensitivity caveat is dropped when it is. `release_verdict` gains `engine_population_exact`; a multi-block run with an exact representation is releasable, and the dense release-candidate posture no longer refuses `--engine-blocks` greater than one. The resolver's own frame keeps the block's true mass; the scaling lives on the engine scratch frame's mass log.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK export-surface allow-list (`uk/gates.json`, `uk_export_surface`) names the graph export's columns: `household.household_clone_index`, `benunit.benunit_clone_index` and `person.person_clone_index` replace the rowwise tool's `household.clone_index`; `household.constituency_code`, `household.local_authority_code` and `household.region_code` replace the `_oa` aliases; the ITL1–3 and ward codes the atomic geography derives are allowed. The gate-battery vintage pins move with the spec.
1 change: 1 addition & 0 deletions changelog.d/1115-export-area-code-names.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The UK single-year export writes the three area codes under the names every consumer reads (microcosm#1114): `constituency_code_oa`, `la_code_oa` and `region_code_oa` replace `constituency_code`, `local_authority_code` and `region_code` at the export boundary (`graph_terminal._tables`, `uk_release_export_frame`), with no alias kept. The graph keeps the ladder names in memory; `uk_export_candidate_columns` translates them so the export-surface gate compares the artifact's names, the allow-list names the `_oa` columns again, the export read-back refuses a ladder name or an empty consumer column, and the export descriptor and the rowwise candidate manifest record `area_codes` (the columns and the code frames behind them, read from the committed support provenance). The gate-battery vintage pins move with the spec.
1 change: 1 addition & 0 deletions changelog.d/1115-review-round-1.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Review round 1 of microcosm#1115: a per-block engine resolution is `exact` only when the blocks are identical copies with a source identity and the allocation keys of policyengine-uk's weight-share formulas (`UK_WEIGHT_SHARE_FORMULA_INPUTS`: corporate land value, shareholding, consumption share) carry the same `sum(x * w)` in every block, recorded per block in the measures receipt; a weights-only comparison is never exact. The `release_input_coverage` binding requires the spine build state (evidence absent without it) and refuses to judge the calibrated frame. The diagnostics schema stays strict: the terminal node's converter hands the holdout in the schema's shape and the loosening that admitted the graph artifact's fields is withdrawn.
1 change: 1 addition & 0 deletions changelog.d/1115-warm-start-size-search.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
`--selection-initial-lambda` warm-starts the UK size selection's L0 budget search at a known penalty (`select_uk_dataset_size(initial_lambda=...)`, the size-search node's `initial_lambda` parameter, recorded in the run parameters and the search receipt). The shared search probes the given penalty first and stops there when the draw is feasible and within tolerance; a miss is followed by one probe half a decade away on the side it steered to, then interpolation between the measured ends (regula falsi with a progress safeguard) instead of re-bisecting the whole bracket. A stale hint costs probes, never feasibility; a warm and a cold search may settle on different penalties inside the budget window. The cold path (no warm start) is unchanged.
1 change: 1 addition & 0 deletions changelog.d/932-collect-each-engine-block.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The per-clone measure resolution (`--engine-blocks` greater than one) now collects garbage after every engine block. A policyengine simulation is a large cyclic object graph that `del` alone does not reclaim, so the first K=25 dense build (2026-10-05) retained all 25 per-clone engines, reached a 140 GB memory footprint on a 24 GiB machine and was killed before the solve.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The scratch dataset a UK measure engine loads now leaves the nation-native alias codes of the derived geography layers (`output_area_code`, `data_zone_code`, `intermediate_zone_code`, `super_data_zone_code`, `district_electoral_area_code`) behind, exactly as the release export does. They are NA outside their own nation by design, and policyengine-uk refuses a single-year dataset with any NaN column, so every `--release-role dense` build under the default atomic assignment failed at `uk.full.measures` with "Column 'data_zone_code' contains NaN values" (the 28 Sept dense rungs ran the legacy assignment and never met it). One helper, `without_uk_native_alias_columns`, now serves both boundaries; the resolver receipt lists the columns it dropped.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The dense role's measures node now compiles the national constraint problem per engine block and stitches the per-block matrices into the pool's household order (`resolve_uk_full_national_problem`), instead of materializing every national target as a column on the whole K-clone pool and copying the prepared pool twice more to compile and check it. On a 1.6 million-household pool that step needed tens of gigabytes: the first K=25 dense build (2026-10-05) reached a 140 GB footprint on a 24 GiB machine and was killed before the solve. Materialization is row-wise (band edges come from the compiled register), so the stitched problem is the pool's problem; the measures receipt records `national_materialization: per_engine_block`. `resolve_uk_full_measures` keeps its contract for the national seam and the evaluation tools.
1 change: 1 addition & 0 deletions changelog.d/932-terminal-gates-skipped-holdout.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The dense role's terminal gate kernel now hands the calibration diagnostics the rotated holdout in the schema's shape: a skipped holdout as the bare `{"skipped": true}` marker and a measured one without the graph's own `graph_binding` provenance, which the diagnostics records (no undeclared fields) refuse. The `uk.full.holdout` artifact keeps the full report. The first K=25 dense build (2026-10-06) reached its terminal gates after 28 hours with `--skip-holdout` and the battery refused the graph's marker with nineteen validation errors; the solve stayed in the content store and resumed.
1 change: 1 addition & 0 deletions changelog.d/calibrate-compiled-row-axis-guard.fixed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A decoded problem's compiled rows (`OrderedProblem.to_target_set`) now guard the entity axis with one shared array and a vectorised comparison instead of rebuilding the axis as a Python tuple from the frame on every row. On the first K=25 UK dense build (2026-10-05) that rebuild made compiling 22,053 local rows over a 1.6 million-household pool cost ~3.5e10 scalar conversions, roughly ten hours per compile; the guard's refusal of a different or reordered axis is unchanged.
16 changes: 8 additions & 8 deletions docs/evidence/spec-engine/us-f0-coverage.json
Original file line number Diff line number Diff line change
Expand Up @@ -1656,13 +1656,13 @@
"compiler_ir.node_slices"
],
"expected": {
"map_sha256": "0fa116d9f5dae4ba0378f190d6c822b92f3dcf1e3fe80c9880bcaf87431c283c",
"protocol_sha256": "b4afa376b62fbc073fd10d74e40921f5ab60c1efe366e2c8504872091ea69d08"
"map_sha256": "b4d6ec9c11ce323d83319a3d0289e0cf7216a7936d6c6f08cef0a463ad369641",
"protocol_sha256": "114217a2918329b0601d94deb8ed83f73919584b85ca8aabe45a40665afb6560"
},
"failures": [],
"observed": {
"map_sha256": "0fa116d9f5dae4ba0378f190d6c822b92f3dcf1e3fe80c9880bcaf87431c283c",
"protocol_sha256": "b4afa376b62fbc073fd10d74e40921f5ab60c1efe366e2c8504872091ea69d08"
"map_sha256": "b4d6ec9c11ce323d83319a3d0289e0cf7216a7936d6c6f08cef0a463ad369641",
"protocol_sha256": "114217a2918329b0601d94deb8ed83f73919584b85ca8aabe45a40665afb6560"
},
"status": "covered"
},
Expand All @@ -1677,7 +1677,7 @@
"compiler_ir.seed_stream_map"
],
"expected": {
"implementation_sha256": "b4afa376b62fbc073fd10d74e40921f5ab60c1efe366e2c8504872091ea69d08",
"implementation_sha256": "114217a2918329b0601d94deb8ed83f73919584b85ca8aabe45a40665afb6560",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand All @@ -1698,7 +1698,7 @@
},
"failures": [],
"observed": {
"implementation_sha256": "b4afa376b62fbc073fd10d74e40921f5ab60c1efe366e2c8504872091ea69d08",
"implementation_sha256": "114217a2918329b0601d94deb8ed83f73919584b85ca8aabe45a40665afb6560",
"protocol": "legacy-v1",
"streams": [
"build_model",
Expand Down Expand Up @@ -2599,7 +2599,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "d1df6b316e4937e809f5f020ea2d3a7bd38bccc1845f3ae6e104f7eed6a73323"
"spec_sha256": "f76f921d4f74392b106b8c0729ce250494e0a55455eaba7b910b19cdf7825178"
}
},
"report_schema_version": 3,
Expand All @@ -2609,7 +2609,7 @@
"country": "us",
"schema_id": "country_spec",
"schema_version": 1,
"spec_sha256": "d1df6b316e4937e809f5f020ea2d3a7bd38bccc1845f3ae6e104f7eed6a73323"
"spec_sha256": "f76f921d4f74392b106b8c0729ce250494e0a55455eaba7b910b19cdf7825178"
},
"status": "pass"
}
11 changes: 8 additions & 3 deletions docs/uk-dense-release-assembly-runbook-762.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,12 @@ solve defaults (K=15, seed 42, 1,500 epochs, learning rate 0.15,
`grain_equal`), names the outputs `microcosm_uk_2024_25_local.h5` and
`microcosm_uk_2024_25_local.local_gates.json`, records itself in the
manifest, and refuses the national role's flags. `--release-candidate` pins
the doctrine (bound 10, `grain_equal`, K=15, 1500 epochs), resolves the
engine in a single block, and runs the rotated holdout. Best-effort staging
the doctrine (bound 10, `grain_equal`, K=15, 1500 epochs) and runs the
rotated holdout. The engine may resolve in a single block or per clone
(`--engine-blocks K`): each block's engine weights are scaled to the pool so
weight-share formulas see the pool's denominator, and the measures receipt
records whether the blocks were identical copies (an exact representation);
a per-block run is releasable only when it was. Best-effort staging
telemetry uploads every 300 s by default on this driver (the Hub allows about
128 commits per hour per repository).
Expect about 3.5 hours and 10 GB at K=15 (measured on the rowwise tool; the
Expand All @@ -73,7 +77,8 @@ uv run --no-sync python tools/score_uk_local_candidate.py ... --output-json <can

The pre-flight checks the manifest and the signed gate report for everything
the contract will demand: release posture attested, shippable, every
release-blocking gate passed, single-block engine, the doctrine values, the
release-blocking gate passed, an exact engine population (single block, or
per-block with identical copies scaled to the pool), the doctrine values, the
A15 census household uprating and A17 tenure application, the measure
exclusions and their windows, the holdout, the
Logbook row, the artifact digest.
Expand Down
Loading
Loading