[FBumann]

Prompt: Is the coore concern to be able to assert that parameters over mutliple dims are all not needed over all of those dims values, but only over the sparse "connection" dim, which says which combinations are even valid? How could we solve this without multi dims? […] Lets do a PR for it (draft) […] we need to define clearly that where is a mask, and missing row (after masking) use the missing rule. […] Maybe relations should be the second authorative stage…? And parameters can use relations in where, but not the other way round? […] Parameter reading other parameters seems reasonable…

Note

The following content was generated by AI.

What this changes

A parameter takes a where: mask. The mask comes first: outside it the parameter is absent, and its data rows there are dropped. missing: then applies to the coordinates the mask keeps. This gives #944's connection case with no multi-dimension. Stacked on #894.

The connection case from #944, which loads on this branch:

dimensions:
  entity: { dtype: str }
  bus: { dtype: str }
  snapshot: { dtype: int }
  role: { dtype: str }
relations:
  connection: { key: [entity, bus], values: role, missing: absent } # the valid pairs, and their role
parameters:
  efficiency: { dims: [entity, bus, snapshot], where: connection, missing: 1 }
variables:
  injection: { dims: [entity, snapshot] }
constraints:
  nodal_balance:
    dims: [bus, snapshot]
    expression: sum(efficiency * injection, over=entity) == 0
assumptions:
  output_efficiency_is_positive:
    holds: "efficiency > 0"
    where: "connection == 'output'"

• where: connection reads "a row exists". efficiency has coordinates only at the listed pairs, missing: 1 fills only there, and a data row at any other pair is dropped.
• connection == 'output' reads the role column, for the role conditions in Datarecord alignment #944.
• Entities and buses stay real dimensions, so the balance needs no lookups.

Two connections of one entity to one bus (PyPSA/PyPSA#2015) do not fit a relation keyed by (entity, bus), because a key has one row for each pair. Key the relation by port instead, with the bus as its value. The mask is still the relation's key, so the parameter is indexed by port:

relations:
  connection: { key: [entity, port], values: bus, missing: absent } # PyPSA's bus0, bus1, ...
parameters:
  efficiency: { dims: [entity, port, snapshot], where: connection, missing: 1 }
constraints:
  nodal_balance:
    dims: [bus, snapshot]
    expression: sum(efficiency * injection, over=[entity, port], by=connection[bus]) == 0

$$\sum_{e,, o ,:, \mathrm{connection}(e,o) = b} \mathrm{efficiency}_{e,o,t} \cdot \mathrm{injection}_{e,t} = 0$$

Two ports may name one bus. This also loads on this branch, and needs nothing beyond this PR.

The rule, the order of stages, the refusals, and the two load checks that changed

A parameter's coordinates are decided in two steps, in this order:

1. where: is a mask on the dims. It keeps the coordinates where its condition is true. At every other coordinate, the parameter is absent wherever it is read, and a row of the data there is dropped when the data is attached.
2. At each coordinate the mask keeps, the data has a row or it does not. A row is read as it is. A missing row follows missing:: it is refused, absent, or the value missing: names.

missing: never reaches a coordinate that the mask leaves out. The reference page, the engine guide (reading.md), and the docs on ParameterSpec.where and Parameter.where all state this order.

What a mask may read: the stages before it.

Stage	Decides	Reads
dimensions	which labels exist	nothing
relations	which combinations of labels exist. missing: is about the key	dimensions. A relation has no where:
unmasked parameters	values at every coordinate	nothing
masked parameters	values at the coordinates their mask keeps	dimensions, relations, unmasked parameters

A relation stays the authority on which combinations exist, and no parameter overrules it. A coverage rule on a relation is an assumption, a check rather than an authority. The parameters page states the order, and the relations page now states the other half: a relation reads only its dimensions, comes before every mask, and is checked by an assumption, such as {holds: "line_to", where: "closed"}.

Refused at load, each with its own test:

• a mask that reads a variable, directly or through a named condition;
• a mask that reads its own parameter, directly or through a named condition;
• a mask that reads a parameter that has a mask of its own. Its message names the rewrite: "Test what the mask of 'flag' tests, or a relation.";
• a mask over a dimension the parameter does not have.

With no masked parameter in any mask, no two masks wait on each other, so no search for a cycle is needed. Every mask in #944 reads an unmasked flag, a named condition or a relation. A later PR can allow a mask to read a masked parameter, and no file that loads today would then be refused.

Two load checks assumed that a parameter has a row at every coordinate:

• The case check read a missing row as the value missing: names. Outside a mask, the parameter is absent and every comparison on it is false. So two cases that look disjoint under the fill value can both claim a coordinate there. For example, with missing: 0, the cases NOT cost > -1 and NOT cost < 1 both hold where cost is absent. A masked parameter now gives the check no fill value.
• A bare int or str name under refused was refused as a no-op, because such a parameter "has a row at every coordinate". With a mask, the name is true only inside the mask, so it loads.

The divisor check of #894 needs no change. Outside the mask, an absent divisor removes the row and is never 0. Inside the mask, missing: 0 reads as it does today.

Typesetting. The legend prints the mask beside the dimensions, in all three formats: efficiency over $\mathcal{E} \times \mathcal{B} \times \mathcal{T}$, only where $(e, b) \in \mathrm{connection}$, $1$ where the data has no row. The golden spec gains link_eff: { dims: [generator, bus], where: connection, missing: 1 }. Each golden output, and the notation page generated from that spec, gains exactly that one legend line. The golden spec is in the compile-tex corpus, so CI compiles the LaTeX legend.

Composition needs nothing new. A parameter is declared in one fragment only. A sibling reads it under given:, which takes no where: and gives the case check no fill value.

Mutation table

Each guard was deleted in turn, and the full suite was run. The file was restored with git checkout --, and __pycache__ was dropped before and after each run. The tree came back clean. Taken on b062e0f. The later commit ff91d15 changes the relations page only.

Guard deleted	Caught by
a mask that reads a variable is refused	test_a_parameter_mask_that_nothing_can_evaluate_is_refused[a-mask-that-reads-a-variable], […-through-a-named-condition]
a mask that reads a masked parameter, or its own, is refused	[a-mask-that-reads-its-own-parameter], […-through-a-named-condition], [a-mask-that-reads-a-masked-parameter], [two-masks-that-read-each-other]
a mask fits its parameter's dims	[a-mask-over-a-dimension-the-parameter-lacks]
a masked int or str name is not a no-op	test_a_bare_parameter_whose_answer_needs_the_data_loads[a-masked-int]
the case check assumes no fill for a masked parameter	test_a_masked_parameter_is_not_read_as_its_fill_outside_the_mask
the legend prints the mask	test_the_legend_says_where_a_parameter_has_coordinates[latex], [markdown], [typst], the golden outputs, and the notation page

Gates

On the head, aa9ca2c, which merges #894's branch after main (0.4.0, #931, #946, #947) went into it. The merge had no conflict. With pixi:

• pixi run lint passes.
• pixi run test passes: 3120 passed, 1 skipped.
• pixi run docs-build passes.
• compile-tex failed on every document because the container could not download the TeX bundle. CI runs it.

With #949. A trial merge of #949 (the case-check fixes) into #948's head conflicts only in CHANGELOG.md, and pytest gives 3143 passed, 1 skipped on the combined code.

Earlier, on ff91d15, in a uv venv on Python 3.13 with the pinned ruff==0.16.1, pyrefly==1.2.0 and zensical==0.0.63: pytest 3108 passed, 3 skipped. The schema (python -m tools.schema), the golden outputs (python -m tests.typesetting.golden) and the notation page (python -m tools.notation) were regenerated, not edited. python -m mathspec check loads both YAML blocks above, the YAML on the reference page, and the line_to assumption on the relations page.

Open questions, and what was not done

For review: filter or refuse. A data row outside the mask is dropped, not refused. This is what mask means in the glossary, and the targets in #944 need it: p_nom_max: { where: "p_nom_extendable" }, with PyPSA data that gives p_nom_max for every generator. A file that wants such a row to be an error states it once as an assumption, {holds: "connection", where: "efficiency"}. The reference page says so.

Not done:

• specsolve. It must apply the mask when it attaches the data, by dropping the rows outside it, and then apply missing: inside. That is a separate PR in fluxopt/specsolve. Until then, this spelling loads and typesets, but it does not solve.
• where: on a relation. A relation is the stage a mask reads, so it takes none.
• The -1 on a link's input bus. missing: 1 gives every connection the same fill, so the -1 comes from the data or from a two-case expression (Cased missing #940).
• The refusal for a curve values table under points: with missing: refused still ignores a mask. That refusal stays on the safe side, and a file can declare absent.
• The typesetter does not push a parameter's mask into a sum's index set, such as $\sum_{e : (e,b) \in C}$. The mask prints in the legend only.
• What a comparison on a relation reads where the relation has no row is docs(language): a comparison on a relation is false where the relation has no row, whatever its operator #947, now on main and in this branch. fix(language): the case check reads a comparison on a relation as false where the relation has no row #949 makes the case check follow it.

Departures from AGENTS.md:

• The base is feat(language)!: a missing row reads as the value the file names, and missing: neutral is gone #894's branch, not origin/main, so that the change stacks on missing: as it now reads.
• The local worktree branch feat/parameter-where pushes to the session's branch claude/gallant-franklin-6ndbd1.

Why

#935 and #939 propose multi-dimensions and properties, so that a parameter over several dimensions holds only the valid combinations. A mask on a parameter says the same with constructs the language has: one relation lists the valid pairs and their role, and the members stay real dimensions.

🤖 Generated with Claude Code

https://claude.ai/code/session_0113aUbLChfQVBrVR5FVQnJq

[read-the-docs-community]

Documentation build overview

📚 mathspec | 🛠️ Build #35353721 | 📁 Comparing aa9ca2c against latest (13fba32)

  🔍 Preview build  

34 files changed · ± 34 modified

± Modified

• reference/api/index.html
• reference/changelog/index.html
• reference/glossary/index.html
• reference/language/index.html
• reference/notation/index.html
• reference/program/index.html
• reference/reading/index.html
• reference/spec/index.html
• examples/pypsa/carrier/index.html
• examples/pypsa/cost/index.html
• and 24 more...