From eb5f0758ca2222fe2da4f116e677cb908c501e85 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 23 Oct 2026 18:56:39 +0000 Subject: [PATCH 1/3] feat(language): a file may declare sets of labels and the properties of a dimension or a set `sets:` names which combinations of labels exist, and `properties:` maps each row of a dimension or a set into a dimension. Both lower to the program's relations: a set under its own name, a property under `subject.property` with `Relation.subject` naming the rows its key ranges over. An entry over a set is refused with the rewrite to the member dimensions and `where:`. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6yvGzZN2iR4WyA9RumvvJ --- docs/reference/glossary.md | 2 +- docs/reference/language/file.md | 6 +- docs/reference/language/index.md | 2 +- docs/reference/language/sets.md | 144 ++++++++++++++ mkdocs.yml | 1 + schema/mathspec.schema.json | 113 ++++++++++- src/mathspec/_canonical.py | 4 +- src/mathspec/_composition.py | 25 ++- src/mathspec/_lowering.py | 2 +- src/mathspec/_resolution.py | 34 +++- src/mathspec/_validation.py | 98 +++++++++- src/mathspec/program.py | 10 + src/mathspec/spec.py | 162 ++++++++++++++-- tests/test_sets.py | 324 +++++++++++++++++++++++++++++++ 14 files changed, 897 insertions(+), 30 deletions(-) create mode 100644 docs/reference/language/sets.md create mode 100644 tests/test_sets.py diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index d15900b2..5e2242fa 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -46,7 +46,7 @@ A consumer is responsible for checking that the host provides each given name ([what a program does not build](reading.md#what-a-program-does-not-build)). **Section** -: One of the thirteen top-level keys of a file, such as `parameters:` or +: One of the fifteen top-level keys of a file, such as `parameters:` or `given:`, and the entries under it ([file shape](language/file.md)). **Entry** diff --git a/docs/reference/language/file.md b/docs/reference/language/file.md index ca469836..88a5bd2d 100644 --- a/docs/reference/language/file.md +++ b/docs/reference/language/file.md @@ -5,14 +5,16 @@ SPDX-License-Identifier: CC-BY-4.0 # File shape -A spec file is a YAML mapping with **thirteen section keys**, plus -`version` and `description`, and any subset of the thirteen is accepted. +A spec file is a YAML mapping with **fifteen section keys**, plus +`version` and `description`, and any subset of the fifteen is accepted. | Key | | | ------------- | --------------------------------------------------------------------------------------------------------- | | `given` | what this file reads but does not build ([given](parameters-variables-constraints.md#given)) | | `dimensions` | the dimensions ([dimensions](dimensions.md)) | | `relations` | named relations between dimensions ([relations](relations.md)) | +| `sets` | which combinations of labels exist ([sets](sets.md#sets)) | +| `properties` | a label for each row of a dimension or a set ([properties](sets.md#properties)) | | `parameters` | the data the spec expects ([parameters](parameters-variables-constraints.md#parameters)) | | `conditions` | named conditions, read by name in a `where:`, a `when:` or a `holds:` ([conditions](named.md#conditions)) | | `variables` | what the solver decides | diff --git a/docs/reference/language/index.md b/docs/reference/language/index.md index c6faeeee..7a2903fb 100644 --- a/docs/reference/language/index.md +++ b/docs/reference/language/index.md @@ -47,7 +47,7 @@ names the fix. | # | Rule | | | --- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | -| 1 | A file has thirteen section keys, plus `version` and `description`. An unknown key is refused, with the nearest valid key named. | [File shape](file.md) | +| 1 | A file has fifteen section keys, plus `version` and `description`. An unknown key is refused, with the nearest valid key named. | [File shape](file.md) | | 2 | `to_spec` checks everything it can check without data when the file loads. | [Errors](errors.md) | | 3 | Every name is declared once. A parameter and a dimension both called `snapshot` is refused. | [Names](expressions.md#name-resolution) | | 4 | The kind of a name decides where it may stand. A dimension follows `over=` or `along=`, and is never multiplied. | [Names](expressions.md#name-resolution) | diff --git a/docs/reference/language/sets.md b/docs/reference/language/sets.md new file mode 100644 index 00000000..03153f6a --- /dev/null +++ b/docs/reference/language/sets.md @@ -0,0 +1,144 @@ + + +# Sets and properties + +You declare which combinations of labels exist as a **set**, and a fact about +each row of a dimension or a set as a **property**. The data supplies the rows +of both. + +```yaml +dimensions: + generator: { dtype: str } + bus: { dtype: str } + line: { dtype: str } + period: { dtype: int } + zone: { dtype: str } +sets: + connection: { dims: [generator, bus], missing: absent } # the pairs that are wired + generator_period: { dims: [generator, period] } +properties: + generator: + bus: bus # each generator is on one bus + line: + bus0: bus + bus1: { into: bus, missing: absent } # a line may have an open end + generator_period: + zone: zone # each generator is in one zone in each period +``` + +## `sets` + +A set is a table whose rows are combinations of labels. `connection` holds the +pairs of a generator and a bus that are wired, and every other pair is not in +the set. + +| Field | | | +| ------------- | ------------------------------------------------------------------------ | ----------------- | +| `dims` | required. The dimensions a row combines, as a list or as a mapping | | +| `missing` | `refused` or `absent`: whether every combination of `dims` is a row | default `refused` | +| `description` | free text | default `null` | + +A set has at least two dimensions. For a subset of one dimension, declare a +`bool` parameter. Where two members of a set are over one dimension, the +mapping form names them, and each name is a column of the set's table: + +```yaml +dimensions: + generator: { dtype: str } + snapshot: { dtype: int, ordered: true } +sets: + cover: { dims: { generator: generator, start: snapshot, covered: snapshot }, missing: absent } +``` + +A member is named after its own dimension or after no dimension, so +`dims: {bus: generator, zone: zone}` is refused. A set names dimensions only, +and a set in the `dims:` of another set is refused. + +### Entries over a set + +No parameter, variable, constraint or expression is over a set. An entry is +over the dimensions of the set, and its `where:` names the set: + +```yaml +dimensions: + generator: { dtype: str } + bus: { dtype: str } + snapshot: { dtype: int } +sets: + connection: { dims: [generator, bus], missing: absent } +parameters: + load: { dims: [bus, snapshot] } +variables: + flow: + dims: [generator, bus, snapshot] + where: connection + bounds: { lower: 0 } +constraints: + balance: + dims: [bus, snapshot] + expression: sum(flow, over=generator) == load +``` + +$$\mathit{flow}_{g,b,t} \ge 0 \qquad \forall\, g \in \mathcal{G},\ b \in \mathcal{B},\ t \in \mathcal{T} \,:\, \left( g,\ b \right) \in \mathrm{connection}$$ + +`flow` is [absent](absence.md) at every pair that `connection` does not list. +A parameter's `where:` reads a set in the same way +([a parameter's mask](parameters-variables-constraints.md#a-parameters-mask)). +An entry written over the set is refused, and the message names the rewrite: + +```text +Variable 'flow' is over set 'connection', and no entry is over a set. Write dims: [generator, bus, snapshot] with where: connection. +``` + +## `properties` + +A property maps each row of its **subject** to one label of a dimension. The +subject is a dimension or a set, and `properties:` lists the properties of each +subject under the subject's name. `generator: {bus: bus}` gives each generator +one bus. + +| Field | | | +| ------------- | -------------------------------------------------------------------- | ----------------- | +| `into` | required. The dimension the value is a label of | | +| `missing` | `refused` or `absent`: whether every row of the subject has it | default `refused` | +| `description` | free text | default `null` | + +`bus: bus` is the short form of `bus: {into: bus}`. Two properties of one +subject may map into one dimension, as `bus0` and `bus1` do. A property of a +set maps each row of the set, so `generator_period: {zone: zone}` gives each +generator one zone in each period. + +`to_spec` refuses a property in these cases: + +- The subject is not a declared dimension or set. +- `into:` is not a declared dimension. A property maps into one dimension, so + for a set, declare one property for each dimension of the set. +- The property has the name of a member of its subject. `connection: {bus: zone}` + is refused, because `bus` is a member of `connection`. +- The property is named after a dimension it does not map into. `period: zone` + is refused. Name the property after what it holds. +- The subject is a set with two members over one dimension, such as `cover`. + A subject has one member over each dimension. + +A property name is not declared in the namespace of the file, so two subjects +can both have a property `bus`. + +## A missing row + +`missing:` checks a key at two levels, and never a value. A value that is not a +label of its dimension is refused when the data is attached, so no value fills +a gap. + +| On a | `missing:` asks | `absent` means | +| ---------- | ---------------------------------------------------- | -------------------------------------------------- | +| set | is every combination of `dims` a row? | a combination with no row is not in the set | +| property | does every row of the subject have the property? | a row of the subject with no value lacks it | + +The default `refused` refuses data that leaves a row out. For +`generator_period` above, every pair of a generator and a period is a row, and +every one of those rows has a `zone`. For `connection`, only the listed pairs +are rows, and a property of `connection` with `missing: refused` needs a value +at each listed pair, not at every pair of a generator and a bus. diff --git a/mkdocs.yml b/mkdocs.yml index 66cdef2c..12fd603a 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -52,6 +52,7 @@ nav: - Parameters, variables, constraints and the objective: reference/language/parameters-variables-constraints.md - Dimensions: reference/language/dimensions.md - Relations: reference/language/relations.md + - Sets and properties: reference/language/sets.md - Expressions: reference/language/expressions.md - Named expressions, conditions and macros: reference/language/named.md - Operators: reference/language/operators.md diff --git a/schema/mathspec.schema.json b/schema/mathspec.schema.json index 889ae979..200a731d 100644 --- a/schema/mathspec.schema.json +++ b/schema/mathspec.schema.json @@ -808,6 +808,49 @@ "title": "PiecewiseSpec", "type": "object" }, + "PropertySpec": { + "anyOf": [ + { + "additionalProperties": false, + "description": "A fact about each row of a subject, which is a dimension or a set: one label of the dimension it maps into.\n\nDeclared under ``properties:``, keyed by the subject and then by the\nproperty's name. ``bus: bus`` is the short form of ``bus: {into: bus}``::\n\n properties:\n generator:\n bus: bus\n line:\n bus0: bus\n bus1: {into: bus, missing: absent}\n generator_period:\n zone: zone\n\n``missing:`` asks whether every row of the subject has the property. The\nvalue is never missing: a row that has the property has one label of\n``into``. The table is data, and arrives under ``subject.property``, with\nthe subject's key columns and one column named after the property.", + "properties": { + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Description" + }, + "into": { + "title": "Into", + "type": "string" + }, + "missing": { + "default": "refused", + "enum": [ + "refused", + "absent" + ], + "title": "Missing", + "type": "string" + } + }, + "required": [ + "into" + ], + "title": "PropertySpec", + "type": "object" + }, + { + "type": "string" + } + ] + }, "RelationSpec": { "additionalProperties": false, "description": "A named relation between dimensions: the columns a row is keyed by, and the columns that key determines.\n\nEach side is a dimension, a list of them, or a mapping of column name to\ndimension where two columns share one. ``key:`` is the claim the language\nchecks when the data is attached: one row per key tuple, so every ``values:`` column is a\nfunction of it. A relation with no ``values:`` is **bare** \u2014 every column is\nin its key, a row is its own identity, and nothing reads it::\n\n relations:\n gen_bus: {key: generator, values: bus}\n gen_bt: {key: [generator], values: [bus, technology]}\n zone_of: {key: [generator, period], values: zone}\n ends: {key: line, values: {bus0: bus, bus1: bus}}\n connection: {key: [generator, bus]}\n\nA sum joins the table on the columns ``over=`` names and every other key\ncolumn, and groups by the columns ``by=relation[...]`` names. A lookup\njoins on the columns it names. The entry fixes no direction. A\ncolumn is named after its own dimension or after none, so a name in a\ncall reads the same as a column and as a dimension. The map itself is\ndata, and arrives with the rest of it, under the relation's name, one\ncolumn per role.", @@ -884,6 +927,55 @@ "title": "RelationSpec", "type": "object" }, + "SetSpec": { + "additionalProperties": false, + "description": "A named set: which combinations of the labels of its dimensions exist.\n\n``dims:`` lists the dimensions a row combines, or maps each member's name to\nits dimension where two members share one. No entry is over a set. An entry\nis over the member dimensions, and ``where:`` names the set::\n\n sets:\n connection: {dims: [entity, port], missing: absent}\n cover: {dims: {generator: generator, start: snapshot, covered: snapshot}, missing: absent}\n\nThe rows are data, and arrive under the set's name, one column per member.\nA set is also the subject of the [`PropertySpec`][] entries declared under\nits name.", + "properties": { + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "default": null, + "title": "Description" + }, + "dims": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "additionalProperties": { + "type": "string" + }, + "type": "object" + } + ], + "title": "Dims" + }, + "missing": { + "default": "refused", + "enum": [ + "refused", + "absent" + ], + "title": "Missing", + "type": "string" + } + }, + "required": [ + "dims" + ], + "title": "SetSpec", + "type": "object" + }, "SosSpec": { "additionalProperties": false, "description": "A special-ordered set over one dimension of one variable.\n\nOne set per coordinate of the variable's ``dims`` minus ``along``; the\nmembers are the variable's *existing* coordinates along ``along``, in that\ndimension's declared order.\n\n``type: 1`` admits at most one nonzero member, ``type: 2`` at most two,\nand those two consecutive. A consumer with the concept takes the set as\none; [`Spec.expand`][] states it as binaries instead, and the rows it\nwrites multiply by the member's own ``bounds``, which is why a member\nneeds both.", @@ -1001,7 +1093,7 @@ }, "$schema": "https://json-schema.org/draft/2020-12/schema", "additionalProperties": false, - "description": "The declared math \u2014 one YAML file, or one dict, validated. Nothing here has seen data.\n\nA ``Spec`` that exists has passed the whole language: constructing one by\nany route \u2014 ``to_spec``, [`model_validate`][], the constructor \u2014 runs\nevery load-time check, expression pass included, and raises\n[`LanguageError`][mathspec.errors.LanguageError] on a spec the language refuses.\nHolding one is the proof, so nothing downstream checks it again.\n\nThe API is the thirteen sections plus ``version`` and\n``description``, three ways back out \u2014 [`to_dict`][] for the spec as\ndata, [`to_yaml`][] for the file a reviewer reads, [`expand`][] for the\nspec with its formulations written out as plain rows \u2014 and [`program`][], the\nspec typed, which every reader after load walks. Everything else on this\nclass is pydantic's, not a contract this package keeps.", + "description": "The declared math \u2014 one YAML file, or one dict, validated. Nothing here has seen data.\n\nA ``Spec`` that exists has passed the whole language: constructing one by\nany route \u2014 ``to_spec``, [`model_validate`][], the constructor \u2014 runs\nevery load-time check, expression pass included, and raises\n[`LanguageError`][mathspec.errors.LanguageError] on a spec the language refuses.\nHolding one is the proof, so nothing downstream checks it again.\n\nThe API is the fifteen sections plus ``version`` and\n``description``, three ways back out \u2014 [`to_dict`][] for the spec as\ndata, [`to_yaml`][] for the file a reviewer reads, [`expand`][] for the\nspec with its formulations written out as plain rows \u2014 and [`program`][], the\nspec typed, which every reader after load walks. Everything else on this\nclass is pydantic's, not a contract this package keeps.", "properties": { "assumptions": { "additionalProperties": { @@ -1100,6 +1192,17 @@ "title": "Piecewise", "type": "object" }, + "properties": { + "additionalProperties": { + "additionalProperties": { + "$ref": "#/$defs/PropertySpec" + }, + "type": "object" + }, + "default": {}, + "title": "Properties", + "type": "object" + }, "relations": { "additionalProperties": { "$ref": "#/$defs/RelationSpec" @@ -1108,6 +1211,14 @@ "title": "Relations", "type": "object" }, + "sets": { + "additionalProperties": { + "$ref": "#/$defs/SetSpec" + }, + "default": {}, + "title": "Sets", + "type": "object" + }, "sos": { "additionalProperties": { "$ref": "#/$defs/SosSpec" diff --git a/src/mathspec/_canonical.py b/src/mathspec/_canonical.py index ffc41634..cfe73c8d 100644 --- a/src/mathspec/_canonical.py +++ b/src/mathspec/_canonical.py @@ -222,7 +222,7 @@ def canonical_dict(spec: Spec) -> dict[str, object]: """The spec as plain data, in the form two files that state the same spec share. Entries are sorted by name, the ones under each kind of ``given:`` - too, and every expression is printed from its parsed tree, so what is left of a difference is a difference in the spec. + and each subject of ``properties:`` too, and every expression is printed from its parsed tree, so what is left of a difference is a difference in the spec. A ``where`` string, the order of the regions under ``cases:``, the order of an entry's ``dims`` and the order of a piecewise entry's links are all left as written. @@ -238,7 +238,7 @@ def canonical_dict(spec: Spec) -> dict[str, object]: data = spec.to_dict() built: dict[str, object] = {} for section, value in data.items(): - if section == 'given' and isinstance(value, dict): + if section in ('given', 'properties') and isinstance(value, dict): built[section] = {kind: _sorted_entries(entries) for kind, entries in sorted(value.items())} elif isinstance(value, dict) and section != 'objective': built[section] = _sorted_entries(value, bare_is_expression=section == 'expressions') diff --git a/src/mathspec/_composition.py b/src/mathspec/_composition.py index c9d89975..3db20585 100644 --- a/src/mathspec/_composition.py +++ b/src/mathspec/_composition.py @@ -92,6 +92,8 @@ set to ``null`` is refused, because it removes nothing. * **``given:`` is laid over one kind at a time**, by the same rules as any owned section. +* **``properties:`` is laid over one subject at a time**, by the same rules, + so a patch adds, edits or removes one property and keeps the others. """ from __future__ import annotations @@ -107,7 +109,7 @@ from mathspec._yaml import read_spec from mathspec.errors import LanguageError, did_you_mean, schema_error from mathspec.program import variables_of -from mathspec.spec import GivenSpec, Spec +from mathspec.spec import GivenSpec, PropertySpec, Spec if TYPE_CHECKING: from collections.abc import Callable, Iterable @@ -137,6 +139,7 @@ 'piecewise': 'piecewise curve', 'sos': 'special-ordered set', 'objective': 'objective', + 'properties': 'properties of', } @@ -789,6 +792,8 @@ def _lay_over(base: dict[str, object], patch: dict[str, object], name: str) -> d laid[key] = _given(_mapping(laid.get(key)), _section(value, key, name), name) elif key in SHARED_SECTIONS: laid[key] = _shared(_mapping(laid.get(key)), _section(value, key, name), key, name) + elif key == 'properties': + laid[key] = _properties(_mapping(laid.get(key)), _section(value, key, name), name) elif key in OWNED_SECTIONS: section = _section(value, key, name) if key == 'expressions': @@ -923,6 +928,24 @@ def _owned( return out +def _properties(declared: dict[str, object], patch: dict[str, object], name: str) -> dict[str, object]: + """The ``properties:`` section, one subject laid over at a time, so a patch edits one property and keeps the rest. + + A property the base writes in the short form is read as ``{into: ...}`` + first, so a patch that writes only its ``missing:`` edits it rather than + replacing it. + """ + out = dict(declared) + for subject, over in patch.items(): + if over is None: + _removed(out, subject, 'properties of', name) + continue + standing = {p: {'into': e} if isinstance(e, str) else e for p, e in _mapping(out.get(subject)).items()} + section = _section(over, f'properties: {subject}', name) + out[subject] = _owned(standing, section, 'property', PropertySpec, name) + return out + + def _removed(out: dict[str, object], key: str, label: str, name: str) -> None: """Delete what the patch nulled, refusing a removal its base cannot satisfy.""" if key not in out: diff --git a/src/mathspec/_lowering.py b/src/mathspec/_lowering.py index acb1049f..8f012b2f 100644 --- a/src/mathspec/_lowering.py +++ b/src/mathspec/_lowering.py @@ -230,7 +230,7 @@ def lower(spec: Spec) -> Program: name: Dimension(dtype=ddef.dtype, ordered=ddef.ordered, description=ddef.description) for name, ddef in spec.dimensions.items() }, - relations=ns.relations, + relations=ns.tables, sos={ name: Sos(sdef.variable, sdef.along, sos_type=sdef.type, description=sdef.description) for name, sdef in spec.sos.items() diff --git a/src/mathspec/_resolution.py b/src/mathspec/_resolution.py index 22d14244..313bfe91 100644 --- a/src/mathspec/_resolution.py +++ b/src/mathspec/_resolution.py @@ -88,6 +88,7 @@ class Namespace: 'parameters', 'relations', 'spec', + 'tables', 'variables', ) @@ -125,10 +126,12 @@ def __init__(self, spec: Spec) -> None: self.defaults: dict[str, bool | float] = { p: pd.missing for p, pd in spec.parameters.items() if not isinstance(pd.missing, str) and pd.where is None } - #: relation name -> its columns and key, as declared. - self.relations: dict[str, Relation] = { - n: Relation(lk.pairs, lk.key_roles, lk.missing, lk.description) for n, lk in spec.relations.items() - } + #: Every table the file declares, as [`tables`][] lowers it; what the + #: program carries as its relations. + self.tables = tables(spec) + #: relation or set name -> its columns and key: the tables a file + #: writes by name, which a property's dotted name is not. + self.relations: dict[str, Relation] = {n: lk for n, lk in self.tables.items() if lk.subject is None} #: parameter or variable name -> the ``dims`` it is read through. Stamped onto #: each leaf a condition names, the way a relation leaf carries ``over``. self.leaf_dims: dict[str, tuple[str, ...]] = { @@ -338,6 +341,29 @@ def unknown_constraint(self, name: str, context: str, *, formals: Iterable[str] # --------------------------------------------------------------------------- +def tables(spec: Spec) -> dict[str, Relation]: + """Every table *spec* declares, as the relation it lowers to. + + A set is a bare relation under its own name. A property is a relation + named ``subject.property``, keyed by the subject's members, with one value + column named after the property. A dotted name is not one a file can + write, so it collides with nothing the file declares. Reads a spec that + [`reference_errors`][mathspec._validation.reference_errors] passed, so + every subject is a dimension or a set. + """ + relations = {n: Relation(lk.pairs, lk.key_roles, lk.missing, lk.description) for n, lk in spec.relations.items()} + relations |= {n: Relation(st.pairs, st.members, st.missing, st.description) for n, st in spec.sets.items()} + for subject, properties in spec.properties.items(): + key = spec.sets[subject].pairs if subject in spec.sets else ((subject, subject),) + relations |= { + f'{subject}.{name}': Relation( + (*key, (name, prop.into)), tuple(m for m, _ in key), prop.missing, prop.description, subject + ) + for name, prop in properties.items() + } + return relations + + def mask_of(node: Condition | None) -> Mask | None: """The mask an entry carries for a resolved where: ``None`` where there is none, or where every row passes.""" if node is None or (isinstance(node, BooleanLiteral) and node.value): diff --git a/src/mathspec/_validation.py b/src/mathspec/_validation.py index 34b29343..05dd373a 100644 --- a/src/mathspec/_validation.py +++ b/src/mathspec/_validation.py @@ -27,7 +27,7 @@ from mathspec._sos import Emitted as EmittedSet from mathspec._sos import coefficients from mathspec._yaml import read_spec -from mathspec.errors import SchemaError, unordered +from mathspec.errors import SchemaError, did_you_mean, unordered from mathspec.spec import NUMERIC_DTYPES, Spec, side_columns if TYPE_CHECKING: @@ -84,6 +84,8 @@ def reference_errors(spec: Spec) -> list[str]: *_name_collisions(spec), *_frame_dimensions(spec), *_relation_targets(spec), + *_set_members(spec), + *_property_targets(spec), *_bound_names(spec), *_sos_shapes(spec), *_sos_bounds(spec), @@ -102,6 +104,7 @@ def _flat_namespace(spec: Spec) -> list[tuple[str, Iterable[str]]]: return [ ('dimension', spec.dimensions), ('relation', spec.relations), + ('set', spec.sets), ('parameter', spec.parameters), ('given parameter', spec.given.parameters), ('variable', spec.variables), @@ -154,7 +157,11 @@ def _frame_dimensions(spec: Spec) -> Iterator[str]: *(('Named expression', name, e.dims or []) for name, e in spec.expressions.items()), ] for kind, name, dims in frames: - yield from (undeclared_dimension(kind, name, d) for d in dims if d not in spec.dimensions) + yield from ( + _over_a_set(kind, name, dims, d, spec) if d in spec.sets else undeclared_dimension(kind, name, d) + for d in dims + if d not in spec.dimensions + ) yield from ( f"{kind} '{name}' names dimension '{d}' twice. Delete one." for d, count in Counter(dims).items() @@ -204,6 +211,93 @@ def _relation_targets(spec: Spec) -> Iterator[str]: ) +def _over_a_set(kind: str, name: str, dims: Iterable[str], set_name: str, spec: Spec) -> str: + """The refusal for an entry over a set, naming the frame over its member dimensions and the mask.""" + members = [d for _, d in spec.sets[set_name].pairs] + rewrite = dict.fromkeys(d for written in dims for d in (members if written == set_name else [written])) + return ( + f"{kind} '{name}' is over set '{set_name}', and no entry is over a set. Write dims: " + f'[{", ".join(rewrite)}] with where: {set_name}.' + ) + + +def _members_of_sets(kind: str, name: str, dims: Iterable[str], spec: Spec) -> Iterator[str]: + """A set or a property names dimensions only, and the rewrite for a set it names is that set's dimensions.""" + for d in dict.fromkeys(dims): + if d in spec.sets: + members = ', '.join(dict.fromkeys(m for _, m in spec.sets[d].pairs)) + yield f"{kind} '{name}' names set '{d}', and a set is not a dimension. Name its dimensions: {members}." + elif d not in spec.dimensions: + yield undeclared_dimension(kind, name, d) + + +def _set_members(spec: Spec) -> Iterator[str]: + """A set has at least two members, each over a declared dimension, and named for two over one dimension.""" + for sname, entry in spec.sets.items(): + if len(entry.pairs) < 2: + yield ( + f"Set '{sname}' has {len(entry.pairs)} member(s), and needs at least two under 'dims:'. For a " + f'subset of one dimension, declare a bool parameter instead.' + ) + if isinstance(entry.dims, list): + yield from ( + f"Set '{sname}' names dimension '{d}' twice under 'dims:'. Name the two members: " + f'dims: {{{d}0: {d}, {d}1: {d}}}.' + for d, count in Counter(entry.dims).items() + if count > 1 + ) + yield from _members_of_sets('Set', sname, (d for _, d in entry.pairs), spec) + yield from ( + f"Set '{sname}' names member '{member}' after dimension '{member}', but the member is over " + f"'{dim}'. Rename the member after what it holds." + for member, dim in entry.pairs + if member in spec.dimensions and member != dim + ) + + +def _property_targets(spec: Spec) -> Iterator[str]: + """A property belongs to a dimension or a set, maps into a dimension, and is named apart from its subject's key.""" + for subject, properties in spec.properties.items(): + if subject in spec.sets: + key = spec.sets[subject].pairs + elif subject in spec.dimensions: + key = ((subject, subject),) + else: + yield ( + f"Properties of '{subject}': '{subject}' is neither a dimension nor a set. Declare it under " + f"'dimensions:' or 'sets:'. {did_you_mean(subject, {*spec.dimensions, *spec.sets})}" + ) + continue + twice = {d: [m for m, dim in key if dim == d] for d, count in Counter(d for _, d in key).items() if count > 1} + yield from ( + f"Properties of '{subject}': the set has two members over '{d}' ({members}), and a subject has one " + f'member over each dimension. Declare the properties on a set without one of them.' + for d, members in twice.items() + if properties + ) + members = {m for m, _ in key} + for pname, prop in properties.items(): + label = f'{subject}.{pname}' + if pname in members: + yield ( + f"Property '{label}' has the name of a member of the key of '{subject}', so the two would " + f'read the same. Rename the property.' + ) + if prop.into in spec.sets: + dims = ', '.join(dict.fromkeys(d for _, d in spec.sets[prop.into].pairs)) + yield ( + f"Property '{label}' maps into set '{prop.into}', and a property maps into one dimension. " + f'Declare one property for each of {dims}.' + ) + elif prop.into not in spec.dimensions: + yield undeclared_dimension('Property', label, prop.into) + elif pname in spec.dimensions and pname != prop.into: + yield ( + f"Property '{label}' is named after dimension '{pname}', but maps into '{prop.into}'. " + f'Rename the property after what it holds.' + ) + + def _bound_names(spec: Spec) -> Iterator[str]: """A named bound is a numeric parameter, declared here or given.""" parameters = {**spec.parameters, **spec.given.parameters} diff --git a/src/mathspec/program.py b/src/mathspec/program.py index 64f0a0d3..288ca0d5 100644 --- a/src/mathspec/program.py +++ b/src/mathspec/program.py @@ -508,6 +508,10 @@ class Relation: have one row per key tuple — which keeps a mistyped label from silently dropping its terms in the join that places them, and is what lets ``at`` read one value. + + A ``sets:`` entry is a bare relation under its own name. A ``properties:`` + entry is a relation named ``subject.property``, keyed by the subject's + members, with one value column named after the property. """ columns: tuple[tuple[str, str], ...] @@ -516,6 +520,12 @@ class Relation: #: or ``absent``. missing: RelationMissing = 'refused' description: str | None = None + #: The dimension or set whose rows the key ranges over, on a relation + #: lowered from a property; ``None`` on a set and on a relation. Every row + #: here is a row of the subject, and ``missing`` asks whether every row of + #: the subject is a row here, rather than every combination of the key's + #: dimensions. + subject: str | None = None @property def roles(self) -> tuple[str, ...]: diff --git a/src/mathspec/spec.py b/src/mathspec/spec.py index 776020c7..631d7441 100644 --- a/src/mathspec/spec.py +++ b/src/mathspec/spec.py @@ -76,7 +76,9 @@ 'ParameterSpec', 'PiecewiseLinkSpec', 'PiecewiseSpec', + 'PropertySpec', 'RelationSpec', + 'SetSpec', 'SosSpec', 'Spec', 'VariableSpec', @@ -168,6 +170,19 @@ def _missing_null(kind: str, rewrite: str) -> ValueError: return ValueError(f'missing: null on {kind} says nothing. {rewrite}') +def _refused_or_absent(v: object, kind: str, may: str, outcome: str) -> object: + """*v* where it is ``refused`` or ``absent``, the two words a ``missing:`` on a table takes. + + A label is data, so no value fills a gap in a map: the ``missing:`` of a + relation, a set or a property reads a key, never a value. + """ + if v is None: + raise _missing_null(kind, f'Write absent where {may}, or leave the key out for refused.') + if v not in get_args(RelationMissing): + raise ValueError(f'missing: {_as_yaml(v)} on {kind}, which takes refused or absent. {outcome}') + return v + + #: Every formulation, in the order [`Spec.expand`][] writes them out: a curve #: emits a set, and no set emits a curve. FORMULATIONS: tuple[Formulation, ...] = ('piecewise', 'sos') @@ -243,18 +258,12 @@ class RelationSpec(_StrictSpec): @field_validator('missing', mode='before') @classmethod def _refused_or_absent(cls, v: object) -> object: - """A label is data, so no value fills a gap in a map.""" - if v is None: - raise _missing_null( - 'a relation', 'Write absent where a key combination may have no row, or leave the key out for refused.' - ) - if v not in get_args(RelationMissing): - msg = ( - f'missing: {_as_yaml(v)} on a relation, which takes refused or absent. A key combination with no ' - f'row is refused, or is not in the relation.' - ) - raise ValueError(msg) - return v + return _refused_or_absent( + v, + 'a relation', + 'a key combination may have no row', + 'A key combination with no row is refused, or is not in the relation.', + ) @model_serializer(mode='wrap') def _as_written(self, handler: SerializerFunctionWrapHandler) -> dict[str, object]: @@ -289,6 +298,120 @@ def value_roles(self) -> tuple[str, ...]: return tuple(role for role, _ in side_columns(self.values)) +class SetSpec(_StrictSpec): + """A named set: which combinations of the labels of its dimensions exist. + + ``dims:`` lists the dimensions a row combines, or maps each member's name to + its dimension where two members share one. No entry is over a set. An entry + is over the member dimensions, and ``where:`` names the set:: + + sets: + connection: {dims: [entity, port], missing: absent} + cover: {dims: {generator: generator, start: snapshot, covered: snapshot}, missing: absent} + + The rows are data, and arrive under the set's name, one column per member. + A set is also the subject of the [`PropertySpec`][] entries declared under + its name. + """ + + _label: ClassVar[str] = 'a set entry' + + dims: list[str] | dict[str, str] + #: Whether every combination of the member dimensions is a row: + #: ``refused`` refuses data that leaves one out, and ``absent`` reads it + #: as not in the set. + missing: RelationMissing = 'refused' + description: str | None = None + + @field_validator('missing', mode='before') + @classmethod + def _refused_or_absent(cls, v: object) -> object: + return _refused_or_absent( + v, + 'a set', + 'a combination of its dimensions may have no row', + 'A combination with no row is refused, or is not in the set.', + ) + + @model_serializer(mode='wrap') + def _as_written(self, handler: SerializerFunctionWrapHandler) -> dict[str, object]: + """``missing`` is written where it is not ``refused``: refused is what leaving it out says.""" + return _without_refused(cast('dict[str, object]', handler(self))) + + @property + def pairs(self) -> tuple[tuple[str, str], ...]: + """``(member, dimension)`` per member, in written order.""" + return side_columns(self.dims) + + @property + def members(self) -> tuple[str, ...]: + """The member names, however ``dims:`` was written.""" + return tuple(member for member, _ in self.pairs) + + +class PropertySpec(_StrictSpec): + """A fact about each row of a subject, which is a dimension or a set: one label of the dimension it maps into. + + Declared under ``properties:``, keyed by the subject and then by the + property's name. ``bus: bus`` is the short form of ``bus: {into: bus}``:: + + properties: + generator: + bus: bus + line: + bus0: bus + bus1: {into: bus, missing: absent} + generator_period: + zone: zone + + ``missing:`` asks whether every row of the subject has the property. The + value is never missing: a row that has the property has one label of + ``into``. The table is data, and arrives under ``subject.property``, with + the subject's key columns and one column named after the property. + """ + + _label: ClassVar[str] = 'a property entry' + + into: str + #: Whether every row of the subject has the property: ``refused`` refuses + #: data that leaves one out, and ``absent`` reads it as a row without it. + missing: RelationMissing = 'refused' + description: str | None = None + + @model_validator(mode='before') + @classmethod + def _from_string(cls, data: object) -> object: + return {'into': data} if isinstance(data, str) else data + + @field_validator('missing', mode='before') + @classmethod + def _refused_or_absent(cls, v: object) -> object: + return _refused_or_absent( + v, + 'a property', + 'a row of its subject may lack the property', + 'A row of the subject with no value is refused, or does not have the property.', + ) + + @classmethod + @override + def __get_pydantic_json_schema__(cls, core_schema: CoreSchema, handler: GetJsonSchemaHandler) -> dict[str, object]: + """The published schema admits the dimension name the short form is written as.""" + return _also_written_as(core_schema, handler, {'type': 'string'}) + + @model_serializer + def _as_written(self) -> str | dict[str, object]: + """The short form wherever ``into`` is all the entry says.""" + if self.missing == 'refused' and self.description is None: + return self.into + written: dict[str, object] = {'into': self.into} + if self.missing != 'refused': + written['missing'] = self.missing + if self.description is not None: + written['description'] = self.description + return written + + class DimensionSpec(_StrictSpec): """A declared dimension, the dtype its coordinates must be, and whether their order means anything. @@ -1050,7 +1173,7 @@ class Spec(_StrictSpec): [`LanguageError`][mathspec.errors.LanguageError] on a spec the language refuses. Holding one is the proof, so nothing downstream checks it again. - The API is the thirteen sections plus ``version`` and + The API is the fifteen sections plus ``version`` and ``description``, three ways back out — [`to_dict`][] for the spec as data, [`to_yaml`][] for the file a reviewer reads, [`expand`][] for the spec with its formulations written out as plain rows — and [`program`][], the @@ -1072,6 +1195,10 @@ class is pydantic's, not a contract this package keeps. given: GivenSpec = GivenSpec() dimensions: dict[str, DimensionSpec] = {} relations: dict[str, RelationSpec] = {} + sets: dict[str, SetSpec] = {} + #: subject -> property name -> the property, where the subject is a + #: dimension or a set. + properties: dict[str, dict[str, PropertySpec]] = {} parameters: dict[str, ParameterSpec] = {} conditions: dict[str, ConditionSpec] = {} variables: dict[str, VariableSpec] = {} @@ -1223,9 +1350,14 @@ def _names_are_names(self) -> Spec: section added later cannot be forgotten here — every mapping a Spec carries is keyed by an entry name. ``given:`` nests its five mappings one level down, so they are read off [`GivenSpec`][] the - same way. + same way, and ``properties:`` nests each subject's properties, which + a call writes after the subject as ``subject[property]``. """ - sections = [*self, *((f'given: {kind}', group) for kind, group in self.given)] + sections = [ + *self, + *((f'given: {kind}', group) for kind, group in self.given), + *((f'properties: {subject}', group) for subject, group in self.properties.items()), + ] errors = [ f'{section}: {name!r} is not a name. Start it with a letter or an underscore, ' f'then use only letters, digits or underscores.' diff --git a/tests/test_sets.py b/tests/test_sets.py new file mode 100644 index 00000000..2b91ddd2 --- /dev/null +++ b/tests/test_sets.py @@ -0,0 +1,324 @@ +# SPDX-FileCopyrightText: mathspec Contributors +# +# SPDX-License-Identifier: MIT + +"""`sets:` and `properties:`: what they load as, and every refusal decided at load.""" + +from __future__ import annotations + +from typing import Any + +import pytest + +from mathspec import merge, override, to_spec +from mathspec._typesetting import FORMATS, typeset +from mathspec.errors import LanguageError +from mathspec.program import Relation +from tests.fixtures import varied + +MODEL: dict[str, Any] = { + 'dimensions': { + 'generator': {}, + 'bus': {}, + 'line': {}, + 'entity': {}, + 'port': {}, + 'period': {'dtype': 'int'}, + 'snapshot': {'dtype': 'int', 'ordered': True}, + 'zone': {}, + }, + 'sets': { + 'connection': {'dims': ['entity', 'port'], 'missing': 'absent'}, + 'generator_period': {'dims': ['generator', 'period']}, + 'cover': { + 'dims': {'generator': 'generator', 'start': 'snapshot', 'covered': 'snapshot'}, + 'missing': 'absent', + }, + }, + 'properties': { + 'generator': {'bus': 'bus'}, + 'line': {'bus0': 'bus', 'bus1': {'into': 'bus', 'missing': 'absent'}}, + 'snapshot': {'period': 'period', 'rep': 'snapshot'}, + 'connection': {'bus': 'bus'}, + 'generator_period': {'zone': 'zone'}, + }, + 'parameters': {'eff': {'dims': ['entity', 'port'], 'where': 'connection'}}, + 'variables': {'inj': {'dims': ['entity', 'port', 'snapshot'], 'where': 'connection'}}, +} + + +def _refusal(**patch: Any) -> str: + """The message `to_spec` refuses *MODEL* patched with, and it has to refuse.""" + with pytest.raises(LanguageError) as caught: + to_spec(varied(MODEL, **patch)) + return str(caught.value) + + +@pytest.mark.parametrize( + ('name', 'lowered'), + [ + pytest.param( + 'connection', + Relation((('entity', 'entity'), ('port', 'port')), ('entity', 'port'), 'absent'), + id='a-set-is-a-bare-relation-under-its-name', + ), + pytest.param( + 'cover', + Relation( + (('generator', 'generator'), ('start', 'snapshot'), ('covered', 'snapshot')), + ('generator', 'start', 'covered'), + 'absent', + ), + id='a-set-with-named-members', + ), + pytest.param( + 'generator.bus', + Relation((('generator', 'generator'), ('bus', 'bus')), ('generator',), 'refused', subject='generator'), + id='a-property-of-a-dimension', + ), + pytest.param( + 'line.bus1', + Relation((('line', 'line'), ('bus1', 'bus')), ('line',), 'absent', subject='line'), + id='a-property-a-row-may-lack', + ), + pytest.param( + 'snapshot.rep', + Relation((('snapshot', 'snapshot'), ('rep', 'snapshot')), ('snapshot',), 'refused', subject='snapshot'), + id='a-property-into-its-own-dimension', + ), + pytest.param( + 'connection.bus', + Relation( + (('entity', 'entity'), ('port', 'port'), ('bus', 'bus')), + ('entity', 'port'), + 'refused', + subject='connection', + ), + id='a-property-of-a-set-is-keyed-by-its-members', + ), + ], +) +def test_a_set_and_a_property_lower_to_a_relation(name, lowered): + assert to_spec(MODEL).program.relations[name] == lowered + + +def test_every_set_and_property_is_a_relation_of_the_program(): + assert sorted(to_spec(MODEL).program.relations) == [ + 'connection', + 'connection.bus', + 'cover', + 'generator.bus', + 'generator_period', + 'generator_period.zone', + 'line.bus0', + 'line.bus1', + 'snapshot.period', + 'snapshot.rep', + ], 'one relation per set and one per property, and nothing else' + + +def test_a_mask_reads_a_set(): + program = to_spec(MODEL).program + assert program.variables['inj'].where is not None + assert program.variables['inj'].where.names_read == {'connection'} + assert program.parameters['eff'].where is not None + assert program.parameters['eff'].where.names_read == {'connection'} + + +def test_the_file_writes_back_each_property_as_it_was_written(): + written = to_spec(MODEL).to_dict() + assert written['properties'] == MODEL['properties'], 'the short form where into is all a property says' + assert written['sets']['generator_period'] == {'dims': ['generator', 'period']}, 'missing: refused is left out' + assert to_spec(written) == to_spec(MODEL) + + +def test_the_normal_form_sorts_the_properties_of_a_subject(): + spec = to_spec(varied(MODEL, **{'properties.line': {'bus1': 'bus', 'bus0': 'bus'}})) + assert spec.to_yaml(canonical=True) == to_spec( + varied(MODEL, **{'properties.line': {'bus0': 'bus', 'bus1': 'bus'}}) + ).to_yaml(canonical=True) + + +def test_an_error_lists_no_property_as_a_name_to_write(): + message = _refusal(**{'variables.inj.where': 'conection'}) + assert "Relations: ['connection', 'cover', 'generator_period']" in message, ( + 'a property has no name a file can write, so the listing leaves it out' + ) + + +@pytest.mark.parametrize('fmt', FORMATS) +def test_the_typesetter_prints_a_file_with_sets_and_properties(fmt): + assert 'generator' in typeset(MODEL, fmt) + + +@pytest.mark.parametrize( + ('patch', 'fragment'), + [ + pytest.param( + {'variables.inj.dims': ['connection', 'snapshot']}, + "Variable 'inj' is over set 'connection', and no entry is over a set. Write dims: " + '[entity, port, snapshot] with where: connection.', + id='a-variable-over-a-set', + ), + pytest.param( + {'parameters.cost': {'dims': ['cover']}}, + "Parameter 'cost' is over set 'cover', and no entry is over a set. Write dims: [generator, snapshot] " + 'with where: cover.', + id='a-parameter-over-a-set-with-two-members-over-one-dimension', + ), + pytest.param( + {'given.expressions.flow': {'dims': ['connection']}}, + "Given expression 'flow' is over set 'connection'", + id='a-given-entry-over-a-set', + ), + pytest.param( + {'sets.tag': {'dims': ['generator']}}, + "Set 'tag' has 1 member(s), and needs at least two under 'dims:'. For a subset of one dimension, " + 'declare a bool parameter instead.', + id='a-set-with-one-member', + ), + pytest.param( + {'sets.pair': {'dims': ['bus', 'bus']}}, + "Set 'pair' names dimension 'bus' twice under 'dims:'. Name the two members: dims: {bus0: bus, bus1: bus}.", + id='a-set-over-one-dimension-twice-without-names', + ), + pytest.param( + {'sets.nested': {'dims': ['connection', 'snapshot']}}, + "Set 'nested' names set 'connection', and a set is not a dimension. Name its dimensions: entity, port.", + id='a-set-of-a-set', + ), + pytest.param( + {'sets.odd': {'dims': ['generator', 'nowhere']}}, + "Set 'odd' references undeclared dimension 'nowhere'.", + id='a-set-over-an-undeclared-dimension', + ), + pytest.param( + {'sets.odd': {'dims': {'bus': 'generator', 'zone2': 'zone'}}}, + "Set 'odd' names member 'bus' after dimension 'bus', but the member is over 'generator'.", + id='a-member-named-after-another-dimension', + ), + pytest.param( + {'properties.nowhere': {'bus': 'bus'}}, + "Properties of 'nowhere': 'nowhere' is neither a dimension nor a set.", + id='a-subject-that-is-neither-a-dimension-nor-a-set', + ), + pytest.param( + {'properties.parameter_subject': {'bus': 'bus'}, 'parameters.parameter_subject': {'dims': []}}, + "Properties of 'parameter_subject': 'parameter_subject' is neither a dimension nor a set.", + id='a-parameter-as-a-subject', + ), + pytest.param( + {'properties.generator.at': 'connection'}, + "Property 'generator.at' maps into set 'connection', and a property maps into one dimension. Declare " + 'one property for each of entity, port.', + id='a-property-into-a-set', + ), + pytest.param( + {'properties.generator.where_to': 'nowhere'}, + "Property 'generator.where_to' references undeclared dimension 'nowhere'.", + id='a-property-into-an-undeclared-dimension', + ), + pytest.param( + {'properties.connection.port': 'bus'}, + "Property 'connection.port' has the name of a member of the key of 'connection'", + id='a-property-named-as-a-member-of-its-set', + ), + pytest.param( + {'properties.snapshot.snapshot': 'snapshot'}, + "Property 'snapshot.snapshot' has the name of a member of the key of 'snapshot'", + id='a-property-named-as-its-dimension', + ), + pytest.param( + {'properties.generator.zone': 'bus'}, + "Property 'generator.zone' is named after dimension 'zone', but maps into 'bus'. Rename the property " + 'after what it holds.', + id='a-property-named-after-another-dimension', + ), + pytest.param( + {'properties.cover': {'crew': 'zone'}}, + "Properties of 'cover': the set has two members over 'snapshot' (['start', 'covered'])", + id='a-property-of-a-set-with-two-members-over-one-dimension', + ), + pytest.param( + {'parameters.connection': {'dims': ['entity']}}, + "Parameter 'connection' collides with the set of the same name. Rename one of them.", + id='a-set-shares-the-namespace', + ), + pytest.param( + {'properties.generator': {'2bus': 'bus'}}, + "properties: generator: '2bus' is not a name.", + id='a-property-name-a-call-cannot-write', + ), + pytest.param( + {'sets.connection.key': ['entity', 'port']}, + "unknown key 'key' in a set entry. Valid keys: description, dims, missing.", + id='a-set-written-as-a-relation', + ), + pytest.param( + {'properties.generator.bus': {'into': 'bus', 'values': 'bus'}}, + "unknown key 'values' in a property entry. Valid keys: description, into, missing.", + id='a-property-with-an-unknown-key', + ), + ], +) +def test_a_set_or_a_property_the_language_refuses(patch, fragment): + assert fragment in _refusal(**patch) + + +@pytest.mark.parametrize( + ('path', 'written', 'fragment'), + [ + pytest.param( + 'sets.connection.missing', 0, 'missing: 0 on a set, which takes refused or absent', id='set-a-number' + ), + pytest.param('sets.connection.missing', None, 'missing: null on a set says nothing', id='set-null'), + pytest.param( + 'properties.line.bus1', {'into': 'bus', 'missing': 'bus2'}, 'missing: bus2 on a property', id='a-label' + ), + pytest.param( + 'properties.line.bus1', + {'into': 'bus', 'missing': True}, + 'missing: true on a property, which takes refused or absent', + id='property-a-flag', + ), + pytest.param( + 'properties.line.bus1', + {'into': 'bus', 'missing': None}, + 'missing: null on a property says nothing', + id='property-null', + ), + ], +) +def test_a_missing_that_reads_a_value_is_refused(path, written, fragment): + """`missing:` on a set or a property asks about a key, so it takes only the two words.""" + assert fragment in _refusal(**{path: written}) + + +@pytest.mark.parametrize( + ('patch', 'properties'), + [ + pytest.param('properties: {generator: {zone: zone}}\n', {'bus': 'bus', 'zone': 'zone'}, id='add'), + pytest.param( + 'properties: {generator: {bus: {missing: absent}}}\n', + {'bus': {'into': 'bus', 'missing': 'absent'}}, + id='edit-a-short-form', + ), + pytest.param('properties: {generator: {bus: null}}\n', None, id='remove-the-last'), + pytest.param('properties: {generator: null}\n', None, id='remove-the-subject'), + ], +) +def test_a_patch_lays_over_one_subject_at_a_time(patch, properties): + laid = override(to_spec(MODEL), [patch]).to_dict()['properties'] + assert laid.get('generator') == properties + assert laid['line'] == MODEL['properties']['line'], 'a patch to one subject keeps the others' + + +def test_a_patch_that_creates_a_property_writes_it_whole(): + with pytest.raises(LanguageError, match=r"edits the property 'zone'.*a property needs `into`"): + override(to_spec(MODEL), ['properties: {generator: {zone: {missing: absent}}}\n']) + + +def test_two_fragments_that_declare_one_subject_are_refused(): + fragment = {'dimensions': {'generator': {}, 'bus': {}}, 'properties': {'generator': {'bus': 'bus'}}} + with pytest.raises(LanguageError, match="both declare the properties of 'generator'"): + merge([fragment, fragment]) From 59c7feeeaafbad3dcf71e23e74bc978f71c2fc5e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 23 Oct 2026 19:05:17 +0000 Subject: [PATCH 2/3] docs(changelog): add the line for sets and properties Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6yvGzZN2iR4WyA9RumvvJ --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1106224f..ac9e23c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ it releases that version ([RELEASING.md](https://github.com/energy-models/mathsp ## Upcoming version +- feat(language): a file may declare sets of labels and the properties of a dimension or a set ([#1](https://github.com/fluxopt/mathspec/pull/1)) - feat(language): a parameter may declare the coordinates it has with `where:`, and outside them it is absent ([#945](https://github.com/energy-models/mathspec/pull/945)) - fix(language): a divisor that a where, a case or a masked variable guards loads under missing: 0, and so does a variable number outside its bounds ([#942](https://github.com/energy-models/mathspec/pull/942)) - docs(pypsa): a PyPSA schedule, weight or limit that the data leaves out is absent rather than zero, and a committable extendable unit is assumed to have its big-M ([#912](https://github.com/energy-models/mathspec/pull/912)) From 6a0e9f18580ba08e00ce1654f0ab527e3a977467 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 23 Oct 2026 19:29:22 +0000 Subject: [PATCH 3/3] docs(language): format the sets page, and name a set the file does not declare without a misspelling Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01W6yvGzZN2iR4WyA9RumvvJ --- docs/reference/language/sets.md | 28 ++++++++++++++-------------- tests/test_sets.py | 2 +- 2 files changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/reference/language/sets.md b/docs/reference/language/sets.md index 03153f6a..678b60ae 100644 --- a/docs/reference/language/sets.md +++ b/docs/reference/language/sets.md @@ -35,11 +35,11 @@ A set is a table whose rows are combinations of labels. `connection` holds the pairs of a generator and a bus that are wired, and every other pair is not in the set. -| Field | | | -| ------------- | ------------------------------------------------------------------------ | ----------------- | -| `dims` | required. The dimensions a row combines, as a list or as a mapping | | -| `missing` | `refused` or `absent`: whether every combination of `dims` is a row | default `refused` | -| `description` | free text | default `null` | +| Field | | | +| ------------- | ------------------------------------------------------------------- | ----------------- | +| `dims` | required. The dimensions a row combines, as a list or as a mapping | | +| `missing` | `refused` or `absent`: whether every combination of `dims` is a row | default `refused` | +| `description` | free text | default `null` | A set has at least two dimensions. For a subset of one dimension, declare a `bool` parameter. Where two members of a set are over one dimension, the @@ -100,11 +100,11 @@ subject is a dimension or a set, and `properties:` lists the properties of each subject under the subject's name. `generator: {bus: bus}` gives each generator one bus. -| Field | | | -| ------------- | -------------------------------------------------------------------- | ----------------- | -| `into` | required. The dimension the value is a label of | | -| `missing` | `refused` or `absent`: whether every row of the subject has it | default `refused` | -| `description` | free text | default `null` | +| Field | | | +| ------------- | -------------------------------------------------------------- | ----------------- | +| `into` | required. The dimension the value is a label of | | +| `missing` | `refused` or `absent`: whether every row of the subject has it | default `refused` | +| `description` | free text | default `null` | `bus: bus` is the short form of `bus: {into: bus}`. Two properties of one subject may map into one dimension, as `bus0` and `bus1` do. A property of a @@ -132,10 +132,10 @@ can both have a property `bus`. label of its dimension is refused when the data is attached, so no value fills a gap. -| On a | `missing:` asks | `absent` means | -| ---------- | ---------------------------------------------------- | -------------------------------------------------- | -| set | is every combination of `dims` a row? | a combination with no row is not in the set | -| property | does every row of the subject have the property? | a row of the subject with no value lacks it | +| On a | `missing:` asks | `absent` means | +| -------- | ------------------------------------------------ | ------------------------------------------- | +| set | is every combination of `dims` a row? | a combination with no row is not in the set | +| property | does every row of the subject have the property? | a row of the subject with no value lacks it | The default `refused` refuses data that leaves a row out. For `generator_period` above, every pair of a generator and a period is a row, and diff --git a/tests/test_sets.py b/tests/test_sets.py index 2b91ddd2..4e3fb2d3 100644 --- a/tests/test_sets.py +++ b/tests/test_sets.py @@ -140,7 +140,7 @@ def test_the_normal_form_sorts_the_properties_of_a_subject(): def test_an_error_lists_no_property_as_a_name_to_write(): - message = _refusal(**{'variables.inj.where': 'conection'}) + message = _refusal(**{'variables.inj.where': 'wired'}) assert "Relations: ['connection', 'cover', 'generator_period']" in message, ( 'a property has no name a file can write, so the listing leaves it out' )