[FBumann]

Read the performance section before the rest. This PR opened as a simplification, the simplification was measured, and most of it was reversed. What is left is a smaller and different change than the title suggests, and it still costs emit time at the m rungs.

25 files, +1179/−1385.

What it actually changes

The engine reads the data instead of reasoning about the declarations. TermFragment used to carry keyed, label_dims and mapping — a log of every operator that had moved its labels — so _needs_aggregate and may_share_a_column could decide statically whether a constraint's terminal SUM(coeff) GROUP BY row, col had anything to collapse, down to "is there a line whose two ends are one bus?". That is gone. _matrix_share asks the assembled share two linear probes instead — is it ordered, does a cell repeat — and sorts or aggregates only on a yes. The objective asks n_unique() == height. Same answers, ~250 lines less of the subtlest reasoning in the tree, and nothing that has to be kept in step with how a fragment was reshaped.

Labeller becomes labels.frame, a function with no state, and the label order is verified rather than re-established: in_position_order is one linear is_sorted against a single witness column, and all three orderings in the lane go through it. The factored-prefix path survives as a guarded branch of that one function rather than a second method — a mask that cannot see the leading dims leaves the survivors a rectangle, and materialising the full product instead cost +177 MB on sector/m.

Masks restrict by semi-join. A joining predicate is evaluated over the product of only the dims it reads, and the full product is semi-joined against the truth set — so the mask's parameter columns never touch the full product, and left order survives, which is what keeps labelling's verify a verify. A pointwise predicate stays a plain filter.

Smaller, self-contained: _remap_fragment written once for sum(group_by=) and its adjoint at; the shift's presence rule named travelled_presence; ModelTables._spans so the budget=None rule is spelled once; Model._in_our_tree so the two validating doors cannot answer differently; data_validation asking the question it means instead of a cheap probe then a naming pass.

Three fixes, not refactors: plan.children never descended into At, so divisor_parameters could not name a divisor under one; Builtin.value_kwargs was a field no helper declared; relational/__init__ listed an engine.py that does not exist.

One dependency claim withdrawn: polars>=1.36 was raised for explode(empty_as_null=) in a rendering that got reverted, and is back at 1.30.

Performance — measured at both rungs, and it does not amortize

Do not merge this on the current numbers without deciding to accept them.

CodSpeed, against a base that now contains #511 and #568, test_emit (which times build and emit together):

rung	range	worst
m	−6.4% … −16.9%	dispatch-m-highs −16.9%
l	−6.8% … −13.0%	dispatch-l-highs −13.0%

plus two memory regressions at s: transport-s-lp −13.6%, sector-s-lp −8.2%.

The amortization argument is refuted. 0f3614c added the l rung on the claim that fixed per-declaration overhead "reads as −5..−10% at m and amortizes to within ±4% at l", measured locally. With a real l baseline in the series that is not what happens: l lands at −6.8% to −13%, and on sector and transport it is worse than m, not better. Whatever the cost is, it scales with the model rather than sitting on top of it.

That local measurement was mine and it was wrong — a single machine, one arm at a time, against a tree that has since taken three engine changes. The metered series is the instrument that settles this and it disagrees.

A third measurement agrees with CodSpeed rather than with the local one: profiled/l build alone is 1.037 s on this branch against 0.901 s on main, warm, median of five.

What that means for this PR

The change buys readability — the static avoid-work machinery is gone, replaced by probes that ask the data — and it costs roughly a tenth of the pipeline across the whole ladder, at every rung, on both sinks. That is the trade, stated without the amortization that was supposed to soften it.

The suspects, in the order I would look: _matrix_share's two linear probes per constraint share; labelling's _row_major plus the in_position_order verify per declaration; and the semi-join mask path. fleet carries 19 declarations per coordinate and is −10.5%, dispatch carries 3 and is −16.9%, so "fixed per declaration" does not fit the shape either — which is exactly why it needs profiling rather than another argument.

I have not found it yet. Until it is found, the honest options are to accept ~10%, or to hold the PR.

What this PR is no longer

The original description claimed the aggregate always runs, that three labelling paths collapsed to one, and that every mask name is left-joined. All three were true when written and none is true now — measurement reversed them, and the recovery reached the same capabilities by other means. The commit history keeps that arc; this description does not.

The style: inline comments become docstrings sweep that was the bulk of the churn is out, landed separately as #560.

Verification

938 passed, 5 skipped; ruff check, ruff format --check, pyrefly strict (0 errors), mkdocs build --strict. test_a_constraint_row_left_with_no_variables is no longer xfail — #561 resolved what the two lanes were disagreeing about.

🤖 Generated with Claude Code

[read-the-docs-community]

Documentation build overview

📚 lpspec | 🛠️ Build #34013917 | 📁 Comparing d81b5dc against latest (e3eff99)

  🔍 Preview build  

6 files changed · ± 6 modified

± Modified

• ARCHITECTURE/index.html
• SPEC/index.html
• api/index.html
• changelog/index.html
• design/ceiling/index.html
• design/linopy/index.html

[codspeed]

Merging this PR will degrade performance by 7.27%

❌ 17 regressed benchmarks
✅ 25 untouched benchmarks
⏩ 10 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	test_emit[profiled-l-lpspec-highs]	6.4 s	7.2 s	-11.17%
❌	WallTime	test_emit[nodal-m-lpspec-highs]	200.6 ms	223.8 ms	-10.35%
❌	WallTime	test_emit[nodal-l-lpspec-highs]	1.6 s	1.7 s	-9.32%
❌	WallTime	test_emit[fleet-m-lpspec-lp]	1.1 s	1.2 s	-8.47%
❌	WallTime	test_emit[transport-l-lpspec-highs]	5.1 s	5.5 s	-8.46%
❌	WallTime	test_emit[profiled-m-lpspec-highs]	866.6 ms	944.7 ms	-8.26%
❌	WallTime	test_emit[nodal-m-lpspec-lp]	289.5 ms	315.5 ms	-8.24%
❌	WallTime	test_emit[transport-m-lpspec-highs]	545.6 ms	594.4 ms	-8.2%
❌	WallTime	test_emit[fleet-m-lpspec-highs]	811.7 ms	869.3 ms	-6.63%
❌	WallTime	test_emit[storage-m-lpspec-highs]	606.2 ms	646.7 ms	-6.26%
❌	WallTime	test_emit[fleet-l-lpspec-lp]	8.7 s	9.3 s	-6.19%
❌	WallTime	test_emit[dispatch-m-lpspec-highs]	318.5 ms	337.6 ms	-5.67%
❌	WallTime	test_emit[profiled-m-lpspec-lp]	1.2 s	1.3 s	-5.49%
❌	WallTime	test_emit[nodal-l-lpspec-lp]	2.4 s	2.5 s	-5.2%
❌	WallTime	test_emit[fleet-l-lpspec-highs]	6.7 s	7.1 s	-5.16%
❌	WallTime	test_emit[storage-m-lpspec-lp]	858.8 ms	904.8 ms	-5.08%
❌	WallTime	test_emit[transport-l-lpspec-lp]	7.1 s	7.5 s	-5.06%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing refactor/simplify-relational-engine (d81b5dc) with main (4fc661a)}

Footnotes

1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩