[FBumann]

Prompt: 6. specsolve, in fluxopt/specsolve: attach reads sets and properties in both forms.

Note

The following content was generated by AI.

specsolve attaches sets: and properties: in place of relations:. A property arrives under subject.property, or as a column of the table under its subject's key, where a null means the row lacks it. Both lanes agree on every case.

This PR is the specsolve end of the mathspec stack, fluxopt/mathspec#1 to fluxopt/mathspec#5 (the last removes relations:). It cannot merge before mathspec releases that stack. The pin stays at the released 0.4.0, so CI here runs against a mathspec that does not know sets: and will be red.

The base is feat/mathspec-main. The first three commits are #2054's, cherry-picked, because the mathspec stack carries both that change and the conditions: rename. They drop out once #2054 merges.

What changed

The door (_sources.py)

• A set is a table with one column per member. A property is subject.property: the subject's members, then one column named after the property.
• A table under a subject's key that carries property columns is split into one table per property, keeping the rows where the column has a value. What stays under the key is the dimension's index or the set's table. A property given both ways is refused.
• A property of a set is checked against the set's rows: a row the set lacks is refused, and under missing: refused every row of the set needs one. A property of a dimension keeps the product check in attach, over the dimension's labels.
• Messages name set 'connection' and property 'bus0' of 'line', from one helper in _messages.py.

The engine

• A call that reads several properties of one subject (generator[bus, fuel]) names the subject. _engine/relations.py reads it as the inner join of the property tables on the subject's members. Conditions already name real tables, so predicates are unchanged.

The linopy lane (tests/linopy_lane)

• It reads the same inner join (bound_read).
• It now tests a set by its rows in a where. Before, it reduced over value columns, which a set does not have.

On disk

• The catalog records a set's file as set and a property's file as property, not relation. ANSWER_LAYOUT goes from 4 to 5.

The migration

• 96 spec files, the port JSON data, the PyPSA rung tables, differential/pypsa/prep.py, the projection, tools/ladder.py, the bench ports and the tests, through one rename table.
• A relation keyed by one dimension becomes properties of that dimension. One keyed by several dimensions becomes a set plus its properties. A bare relation becomes a set.
• The two ends of a line or a link are from_bus and to_bus everywhere.
• Each migrated spec file was checked to lower to the same constraint, expression and objective trees as before.

Docs

• reference/data.md states the contract. howto/data.md shows PyPSA's generator table passed whole as the wide form.
• warehouse.md, verification.md, the glossary and architecture.md follow.

Verification

The mathspec build is the stack's PR 5 tree, installed over the env with pixi run --no-install throughout.

• Suite: 2 failed, 5775 passed, 574 skipped, 4 xfailed.
  • The base is feat/mathspec-main plus feat!: specsolve requires mathspec 0.4.0, where missing: neutral is gone and a file names the value that a missing row or a masked variable reads as #2054's three commits, cherry-picked, on the stack's mathspec base (parameter where:). It gives 3 failed, 5756 passed.
  • The 2 failures are on both: test_a_relation_orders_bytewise_not_by_declaration and test_a_where_orders_string_labels_bytewise_not_by_declaration. mathspec main now refuses >= on a string dimension, which feat/mathspec-main predates. They are not this PR's.
• Gates: ruff check . and ruff format --check . are clean. pyrefly check (both targets in typecheck) gives 0 errors.
• Bench: bench/test_harness.py gives 482 passed. It ran with --i-know-another-is-running, because other test runs shared the box. Nothing was timed.
• New tests: tests/test_sets_and_properties.py, 13 cases, each held on both lanes. They cover:
  • the canonical, wide-polars, wide-pandas and mixed forms solving to one optimum
  • the inner join
  • each refusal

Mutation table (tools.mutate for line deletions, the last two by hand with the same three precautions; the base failures deselected)

mutation	result
a property given twice (_sources.py:521-525)	caught
a property row its set lacks (_sources.py:551-556)	caught
a set row without a refused property (_sources.py:559-561)	caught
the inner join of several properties (_engine/relations.py:54-55)	caught
the oracle tests a set by its rows (tests/linopy_lane/where.py:276-278)	caught
a null in a wide column dropped (_sources.py:529, the .filter removed)	caught, 3 tests
a set's property spared the product check (attaching.py:85, the clause removed)	caught, 3 tests

Not run, not done, and departures

Not run

• CI. It runs at the pin.
• The floors env and docs-test. The docs env is not installed here.
• The PyPSA parity harness. PyPSA is not installed, so prep.py and the 18 PyPSA reference scripts were checked by reading, not by running.
• The ladder benchmarks.

Not done

• No support for a parameter's where:, the mathspec change this stack is based on. That is its own PR.
• No dev-dependency tooling for the unreleased mathspec.

Departures from the defaults

• One PR spans the engine and the whole migration. With relations: gone, the suite cannot be green on any smaller cut.
• The history holds one "(in progress)" test commit. Parallel agents wrote the test migration, and a commit hook took a checkpoint mid-way.

Where test coverage moved

• Map-on-the-index tests now assert the wide form and its given-twice refusal.
• One duplicate test in test_label_coords.py was deleted. Its point went into the docstring of its twin.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W6yvGzZN2iR4WyA9RumvvJ

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[read-the-docs-community]

Documentation build overview

📚 specsolve | 🛠️ Build #35369673 | 📁 Comparing 3881186 against latest (5a78e3f)

  🔍 Preview build  

117 files changed · ± 117 modified

± Modified

• development/architecture/index.html
• development/verification/index.html
• examples/facility_location/index.html
• examples/genx_piecewise_fuel/index.html
• examples/monthly_budget/index.html
• examples/multi_period/index.html
• examples/osemosys_utopia/index.html
• examples/piecewise/index.html
• examples/piecewise_conversion/index.html
• examples/piecewise_ragged/index.html
• and 107 more...