Accept a restated Ledger filter bound only where it agrees with the compiled constraint

[MaxGhenis]

Re-levelled on main after #955 (us-chronicle-feed-repin) merged at 6710d0f5f; base is main. This was first opened stacked on #955. origin/main was merged in at 6710d0f5f and again at 2b85b7b22 (#976) with no conflicts, and no packages/*/src file differs from main. The restated-filter counts below reproduce unchanged on the main-based tree, for both US Chronicle exports on this machine; they are now committed as experiments/us-labelled-filter-support/restated_filter_arms_receipt.json, written by experiments/us-labelled-filter-support/measure_restated_filter_arms.py (rule and rule-reverted arms; the reverted arm is main's guard behaviour). REPORT.md section 8 cites it.

2026-09-22: Max's rulings on the four open questions are applied on this head — see Rulings below. The one code change they required: a restated EITC qualifying-child upper bound is now refused outright.

What this does

A labelled Chronicle feed can restate a fact's universe constraint in the Ledger's own concept vocabulary — ledger_filter_us:statutes/26/62#adjusted_gross_income_lower_bound — beside the compiled metadata the US fiscal materializer actually slices on (agi_lower_bound). Both keys restrict the microdata, so neither can join SUPPORTED_LEDGER_FILTER_METADATA_KEYS outright: a future feed whose labelled bound disagreed with the compiled one would then be silently ignored, materializing a wider population than the published cell covers. That is exactly the failure _assert_supported_ledger_filter_metadata exists to stop.

_restated_ledger_filter_refusal accepts such a key per spec, and only when the spec also carries the constraint the materializer applies and the two select the same population:

• AGI bounds — numeric equality against agi_lower_bound / agi_upper_bound, including the compiler's -inf / inf ends. Bound sides are judged independently; a suffix-free (exact-value) AGI restatement is refused, because the materializer applies a half-open band and no exact-value AGI filter.
• EITC qualifying children — mask equality against _soi_eitc_child_count_filter, the function the SOI loop calls, evaluated through _eitc_child_count_mask over the probed counts. A lower bound of 3 agrees with 3plus and only with it; a suffix-free key restates an exact count. An upper bound is refused outright, before any comparison, until the Ledger confirms whether it means < or <= (RESTATED_EITC_CHILD_COUNT_REFUSED_SIDES; Max's ruling, 2026-09-22). The two readings select different returns — < 1 agrees with a compiled 0, and so does <= 0 — so neither is guessed.

Anything else keeps the pre-existing refusal, extended to name the values that disagree:

irs_soi.…state_agi.ak.100k_to_200k.taxable_interest_amount:
  ledger_filter_us:statutes/26/62#adjusted_gross_income_lower_bound=50000
  disagrees with agi_lower_bound=100000.0

Unknown ledger_filter_* keys are untouched and still refuse under their bare name.

The same rule runs in _unsupported_soi_ledger_filters. That checker's non-empty result skips the spec from SOI materialization with no error at all, so accepting a restatement at the fatal guard while leaving it listed there would only have moved the spec from a refusal to a silent disappearance. A disagreeing restatement stays listed, and the fatal guard at the top of the same _materialize_target_frame refuses it before the loop runs, so the silent skip is unreachable for one.

The premise reproduces — and why the first measurement said it did not

An earlier revision of this body reported zero unsupported keys on every surface and concluded the rule was forward-compatibility scaffolding. That measurement was correct for the tree it ran in and is now superseded. Both branches have since been re-levelled on origin/main (18c6d39a0), and the refusal is real.

The missing piece is microcosm.build.ledger_targets._constraint_bound_filters, which postdates this stack's merge base 8c44daa52 — it arrived on main with the parameter_gated_threshold filter-aware work in the UK CGT lane (changelog 725-filter-aware-gated-provider.changed.md). It runs inside _ledger_metadata immediately after the dimension stamp:

for key, value in sorted(_constraint_bound_filters(fact, dimensions).items()):
    metadata.setdefault(f"ledger_filter_{key}", value)

and promotes every role: filter universe constraint with a >= > < <= operator, a finite numeric value, and a variable that is not already a dimension into ledger_filter_<variable>_lower_bound / _upper_bound. The earlier reading — "ledger_targets.py:3257 is the only producer of ledger_filter_* metadata and it stamps dimensions alone; nothing synthesises a *_lower_bound key from a constraint row" — was true of the pre-merge tree and is precisely what stopped being true. The feed never changed; it has always stated the AGI band and the qualifying-child floor as universe_constraints, and those rows are now filter keys.

Measured against the pinned feed, re-levelled

tools/build_us_acs_local_release.py::state_admin_specs(feed, ["snap","medicaid","soi"], soi_mode="full") over the feed #955 pins (Chronicle c5e5bf8, consumer_facts_us_c5e5bf8.jsonl, 39,158 facts), counted with the production guard functions. 31,066 state specs in every run.

Tree	Refused by _unsupported_ledger_filter_metadata	Silently skipped by _unsupported_soi_ledger_filters
#955 merged with origin/main, no rule	1,988	1,988
This head	0	0
This head, both call sites reverted (negative control)	1,988	1,988

The 1,988 refused specs carry 2,702 refusal entries:

Key	Entries
ledger_filter_us.tax.earned_income_credit_qualifying_children_lower_bound	1,076
ledger_filter_us:statutes/26/62#adjusted_gross_income_lower_bound	816
ledger_filter_us:statutes/26/62#adjusted_gross_income_upper_bound	810

— the three keys this rule was written for, in the vocabulary it was written for. So this is not a forward guard: it is what keeps the ACS local state surface compiling now that #955 is on main.

State-level target counts by family and target_role are unchanged from the earlier compile (irs_soi|soi_fiscal_distribution 30,306, irs_soi|aca_ptc_returns 547, irs_soi|aca_spending 60, usda_snap|snap_households 51, usda_snap|snap_total 51, cms_medicaid|medicaid_enrollment 51), as are the measure-matrix sizings in experiments/us-labelled-filter-support/REPORT.md §4. No selection or default changes in this PR.

Adjacent and not fixed here: 939 age-band refusals

_constraint_bound_filters also stamps the bare vocabulary. Over the whole compiled registry (32,866 targets, not just the ACS state surface), measured against origin/main 18c6d39a0 with none of this stack applied:

Tree	Compiled targets	Refused
origin/main 18c6d39a0 alone	32,866	3,208
This head	32,866	939

Main's 3,208 split irs_soi|soi_fiscal_distribution 2,269, census_population|population_age 936, ssa|ssa_ssi_age_band_recipients 3. This PR clears the 2,269. The remaining 939 are ledger_filter_age_lower_bound / ledger_filter_age_upper_bound restating the age_lower_bound / age_upper_bound that _population_age_household_values already slices on — the same pattern, one concept further out, and not a regression of this stack: it arrives with _constraint_bound_filters and refuses on main alone for any caller that materializes the age family. The ACS local state surface never selects census_population or ssa, which is why the state-surface table reads 0.

Adding age to RESTATED_LEDGER_FILTER_CONCEPTS under a third age_band rule would be a one-rule change of the same shape, but it widens what this PR accepts beyond the surface it was reviewed against. Per Max's ruling (2026-09-22) it gets its own follow-up PR; this PR adds no age concept.

Rulings (Max, 2026-09-22)

The four questions in REPORT.md section 6 are answered; the report records them in a dated block there.

1. The premise — merge. Evidenced on main (report section 7; section 8's committed receipt: 1,988 state-surface refusals with the rule reverted, which is main's behaviour, 0 with it).
2. agi_lower_usd / agi_upper_usd — do not add them to RESTATED_LEDGER_FILTER_CONCEPTS. The guard already refuses an unknown ledger_filter_* key loudly: _restated_ledger_filter_refusal returns the bare key for a concept it does not name (tools/build_us_fiscal_refresh_release.py:4634-4635) and _assert_supported_ledger_filter_metadata raises (:4675-4687). New test_legacy_agi_usd_dimensions_stay_refused_by_bare_key pins that, even where the legacy value equals the compiled edge.
3. Full mode — soi_mode="totals" becomes the ACS local default, with the AGI-band distribution opt-in. Done in a separate PR, not here; this PR still changes no default or selection.
4. EITC upper bounds — refused until the Ledger confirms its operator. Done here (above). Lower and exact restatements stay under the agreement rule; the comparison now raises if any other bound side reaches it, so removing "upper" from the refused set cannot silently fall back to an exact-count comparison. The receipt's counts do not change (deduced from the committed receipt, not re-measured): its reverted arm lists every otherwise-unsupported, non-noop ledger_filter_* key by bare name, and on every surface of both feeds the only qualifying-child key is …_lower_bound; no …_upper_bound or …_upper_bound_inclusive key appears, and the compile (packages/*/src) is untouched. An ad hoc scan of both feeds (not committed) agrees: their qualifying-child constraint rows use only == (3,534) and >= (1,180).
5. The 939 age-band refusals — own follow-up PR (above).

Tests

• test_restated_agi_bounds_pass_the_guard_only_where_they_agree — both bounds, lower only, upper only, and the -inf/inf ends.
• test_disagreeing_restated_agi_bounds_are_refused_by_value — disagreeing lower, disagreeing upper, a bound on a spec compiling no band, an exact-value restatement, and an unknown key as the untouched control; asserts the refusal text.
• test_restated_eitc_child_bounds_pass_the_guard_only_where_they_agree — 3 with 3plus, exact 2 with 2. (Its former upper 1 with 0 case now lives in the next test, as a refusal.)
• test_restated_eitc_child_upper_bounds_are_refused_outright (2026-09-22) — upper 1 with 0 (agrees if read <) and upper 0 with 0 (agrees if read <=) are both refused with the operator reason, as is an upper bound on a spec with no child-count filter; an exact restatement and an AGI upper bound on the same spec are still accepted; the fatal guard raises and the SOI skip keeps the key; _upper_bound_inclusive stays refused by bare key.
• test_restated_eitc_child_comparison_never_guesses_an_upper_reading (2026-09-22) — with the refused-sides set emptied, an upper bound raises ValueError rather than being compared as an exact count.
• test_legacy_agi_usd_dimensions_stay_refused_by_bare_key (2026-09-22) — ledger_filter_agi_{lower,upper}_usd equal to the compiled band are refused by bare name at both call sites, and the fatal guard raises.
• test_disagreeing_restated_eitc_child_bounds_are_refused_by_value — 3 with 2, 3 with no child-count filter, a non-count value.
• test_restated_filters_clear_the_soi_skip_only_where_they_agree — the SOI checker clears an agreeing restatement, keeps a disagreeing one, and the fatal guard refuses the disagreement first; unknown SOI filters unchanged.
• test_restated_concept_rules_all_have_a_comparison — every named concept routes to an implemented rule.
• test_pinned_chronicle_feed_compiles_no_unsupported_ledger_filters — the compiled-metadata fixture clears both guards, and every unclassified key in its registry-wide census is noop-valued. The fixture was captured before the re-level, so its census predates _constraint_bound_filters and lists none of the restated keys; it is kept as the pre-merge pin it is, and the fixture description and both tests reading it now say so.
• test_restated_bounds_track_each_compiled_band_in_the_fixture — replays the rule over real compiled bands, agreeing and perturbed.
• test_pinned_chronicle_feed_state_surface_compiles_no_unsupported_filters — this is now the arm that meets real restated keys. Before the re-level it passed over a feed compiling none; it now passes over 31,066 real specs of which 1,988 carry one, and reverting either call site fails it on exactly those 1,988.

CI

This PR now targets main, so both workflows (pull_request: branches: [main]) run on it. While it was based on us-chronicle-feed-repin they triggered nothing, so it was verified locally, at the re-levelled head before #955 merged:

MICROCOSM_US_CHRONICLE_FACTS=<pinned feed> \
  uv run --no-sync pytest packages/microcosm-build/tests/test_us_fiscal_refresh_builder.py
  233 passed in 68.61s                      exit 0   (the feed arm ran, not skipped)

uv run --no-sync python tools/ci_test_groups.py --verify    verification=ok    exit 0
uv run --no-sync python tools/spec_engine_coverage.py --check
  spec-engine coverage: 42162/42162 configuration fields; 41/41 inventory checks   exit 0
uv run --no-sync ruff check .               All checks passed!                     exit 0

Every test file either merge touched (45 files, both packages) also runs green on this head: 2,752 passed, 13 skipped, 0 failed across four batches, every batch exit 0. No attested spec-engine module was touched, so no identity pin moves.

At b6e0b04cd (after merging main 2b85b7b22), locally: the builder module against the pinned feed, 245 tests, no skips, exit 0; the 29 other modules importing tools/build_us_fiscal_refresh_release.py or ledger_targets, 1,405 passed, 13 skipped, and 11 failed plus 4 errors, all in test_us_multispine_pool_tool.py and all OSError: [Errno 28] No space left on device (the machine's disk filled mid-run); the seven affected test functions re-run with the disk clear, 22 passed, exit 0; ci_test_groups.py --verify ok; ruff check and format clean on the changed files. Details in REPORT.md section 8.

At 01dc806f4 (the rulings commit), locally: the builder module against the pinned feed, 248 passed in 80.37s, no skips, exit 0 (the feed arm ran); ci_test_groups.py --verify ok; ruff check . clean; ruff format --check clean on the two changed .py files. The other modules importing the builder were not re-run: the change is confined to _restated_eitc_child_count_refusal and its constants, reached only through the two guards. Details in REPORT.md section 6, "Rulings".

🤖 Generated with Claude Code