[FBumann]

Prompt: "Lets fix #863" — then "I think the actual issue is that the pypsa things should use missing: properly!" — then, on a bare name under refused: "But why not allow a parameter in shift/count etc?" and "Lets do it like this!"

Note

The following content was generated by AI.

What this changes

Data can no longer hold inf or -inf. The PyPSA spec declares its caps and budgets missing: absent. A bare name of a refused parameter that stands in a mask is a load error for every dtype except bool. Inside count, shift and at, a bare name loads. Closes #863.

Why

missing: (#810) gives every use of inf in data a spelling in the file. An infinite coefficient is never a number a solver can take (fluxopt/specsolve#1905). Without inf, a bare float name under refused is true at every coordinate, as a bare int or str name already was. The load rule says so for all three. Inside count, shift and at the name still does work, so the rule does not apply there.

Method, gate output, what was not done

Language (docs/reference/language/, program.py docstrings)

• declarations.md: null, NaN and infinite values are refused when the data is attached. missing: .inf reads a missing row as infinite. The float type row says that data cannot hold .inf. The bare-name paragraph gives the new rule.
• expressions.md and absence.md say the same.
• assumptions.md: curve_complete is written only for a values table that is not refused.

Load rule (_where_resolver.py)

• A bare name of a refused parameter is now refused for every dtype except bool. Before, the rule covered only int and str.
• The resolver reads the operand of count, shift and at with an operand flag, so the rule does not apply there. For int and str, this loosens the rule: count(n, over=g) loaded before only for a float.

Piecewise (piecewise.py, lowering.py)

• {curve}_complete names only the values tables that are not refused. A curve whose tables are all refused writes no _complete, because attaching the data already refuses a missing breakpoint. Emitted still reserves the name.
• The refusal of a refused table under points: is now raised before the curve's assumptions resolve. Before, a bare-name refusal inside a generated assumption reached an internal assert.

PyPSA spec (examples/pypsa.yaml, then the 24 fragments from tools.pypsa_split, then the generated pages)

• missing: absent on Generator_p_nom_max, Link_p_nom_max, Process_p_nom_max, StorageUnit_p_nom_max, Store_e_nom_max, Line_s_nom_max, Transformer_s_nom_max, Generator_e_sum_min and Generator_e_sum_max.
• The six maintenance assumptions change from holds: "X_p_nom_max < inf" to holds: "X_p_nom_max". The rows that refuse a network stay the same.
• count(scenario_weight, over=scenario) stays as written.

Tests where the old behaviour moved

• a-float-may-hold-inf is removed. where-a-bare-float-parameter-under-refused replaces it.
• The refused case of test_under_a_reading_a_missing_row_compares_false is removed. That spec is now a load error.
• New: test_a_bare_name_under_refused_loads_inside_count_shift_and_at (float, int and str × count, shift and at). On the old tree, the int and str cases failed.
• New: test_a_curve_whose_tables_are_refused_states_no_completeness.
• These fixtures used a bare float under refused in a mask. They now declare it missing: absent or neutral, or they compare it: the golden typesetting spec (budget > 0, and run_start is NOT shift(load, …)), the test_walk, test_symbols, test_lowering, test_dimensions and test_exclusivity fixtures, and the six tests/expand/curve-*/after.yaml files, which lose curve_complete.

Mutation table (whole suite, each guard removed alone)

Guard removed	Result
not self.operand in the bare-name rule	70 failed, 16 errors
if name not in refused in assumptions_of	128 failed, 4 errors
the refused_under_points raise in lower	4 failed

Gates

• pixi run lint: passes.
• pixi run test: 2861 passed, 1 skipped.
• docs-build and compile-tex did not run here. The sandbox proxy blocks docs.python.org and the tectonic bundle host. GitHub CI runs both.

Not done

• This PR has two changes, against the one-issue-one-PR default: the session pushes to one branch. The bare-name rule follows from the inf refusal.
• missing: .inf stays, as the issue proposes.
• fluxopt/specsolve needs a follow-up to refuse inf at attach and to drop the inf rows in its PyPSA data preparation.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XUtBEYS2M2gYZzMGC5xuLV

[read-the-docs-community]

Documentation build overview

📚 mathspec | 🛠️ Build #35090988 | 📁 Comparing fbb26ad against latest (5d344e2)

  🔍 Preview build  

19 files changed · ± 19 modified

± Modified

• changelog/index.html
• howto/see-an-expansion/index.html
• reference/notation/index.html
• reference/program/index.html
• reference/reading/index.html
• examples/pypsa/generator/index.html
• examples/pypsa/generator_maintenance/index.html
• examples/pypsa/line/index.html
• examples/pypsa/link/index.html
• examples/pypsa/link_maintenance/index.html
• and 9 more...

[FBumann]

@FabianHofmann That's the last PR id like to bring in for v0.3.0

[brynpickering]

@FBumann so if there is a .inf in the data then an error will be raised? Or does missing: absent allow them through? I don't understand what "missing: gives every use of inf in data a spelling in the file." means. Seems like quite a chore to have to filter out infs on the data side before merging data and spec but maybe it's fine because the engines can handle it, the frameworks don't have to.

[FBumann]

The exact meaning of inf, nan etc is somewhat the responsibility of the engine.

If an engine is strict, it should refuse both nan and inf, as both should be rather be expressed by missing:absent.

Users with nan or inf data would then be forced to explicitly filter out inf/nan rules.
Dense dataformats would of course need to allow nan.

Happy to discuss the implications.

I imagine refusing inf and nan makes the data contract much safer.

Pypsa currently passes inf, but that's for legacy reasons I think. An absent row would be the more honest spelling. And more data efficient.
But maybe I misunderstand sth.

[brynpickering]

Yeah, I think inf passes through pypsa (and others) because it isn't as straightforward as nan to remove from arrays. It also doesn't really matter if it passes through to the optimisation problem as it shouldn't impact anything. Agree though that it should be removed as it slims everything down! Preferably by the engine so the framework doesn't have to care, but that's not a problem for us to deal with.

[FBumann]

So we would treat it like:
Mathspec defines what a missing row means.
The engine defines how to interpret inf, nan etc. they might be treated as missing or as actual values. Depending on the engine.
Agree @brynpickering ?

[FBumann]

@brynpickering Im having some second thoughts about this.
See #880

[FBumann]

Closed in favor of #880