Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ it releases that version ([RELEASING.md](https://github.com/energy-models/mathsp

## Upcoming version

- fix(language): a divisor that a where, a case or a masked variable guards loads under missing: 0, and so does a variable number outside its bounds ([#942](https://github.com/energy-models/mathspec/pull/942))
- docs(pypsa): a PyPSA schedule, weight or limit that the data leaves out is absent rather than zero, and a committable extendable unit is assumed to have its big-M ([#912](https://github.com/energy-models/mathspec/pull/912))
- feat(language)!: a missing row reads as the value the file names, and `missing: neutral` is gone ([#894](https://github.com/energy-models/mathspec/pull/894))
- feat(program): each expression node states the dimensions it is read over ([#899](https://github.com/energy-models/mathspec/pull/899))
Expand Down
60 changes: 47 additions & 13 deletions docs/reference/language/absence.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,16 +54,16 @@ for a cost that is zero, `missing: 1` for an efficiency that is one, and
row of an `absent` parameter, or of a parameter with a value, does at each
position:

| Position | `absent` | a value |
| ----------------------------------------------------------- | ------------------------------------- | --------------------------------------- |
| a coefficient, or a constant on either side of a comparison | removes the row | reads the value |
| a factor of a summand in a `sum` | removes the summand, not the row | reads the value |
| a divisor | removes the row | reads the value. `0` is refused at load |
| a `bounds:` entry | leaves that side of the variable open | reads the value |
| the amount of an `offset=` or a `window=` | makes the operator absent there | reads the value |
| a comparison in a `where` | is false | compares the value |
| a bare `bool` name in a `where` | is false | reads the value |
| a bare number or label in a `where` | is false | is false |
| Position | `absent` | a value |
| ----------------------------------------------------------- | ------------------------------------- | -------------------------------------------------------------------------- |
| a coefficient, or a constant on either side of a comparison | removes the row | reads the value |
| a factor of a summand in a `sum` | removes the summand, not the row | reads the value |
| a divisor | removes the row | reads the value. `0` is refused at load, unless a guard leaves the row out |
| a `bounds:` entry | leaves that side of the variable open | reads the value |
| the amount of an `offset=` or a `window=` | makes the operator absent there | reads the value |
| a comparison in a `where` | is false | compares the value |
| a bare `bool` name in a `where` | is false | reads the value |
| a bare number or label in a `where` | is false | is false |

A value of `0` as an amount reads `shift(x, along=t, offset=0)` as `x` itself,
and sums nothing under `sum_back(x, along=t, window=0)`. A curve needs a row at
Expand All @@ -73,6 +73,37 @@ of a table that a curve reads means. For a bound only where the data has one,
mask the variable, or declare the parameter `missing: absent` or
`missing: .inf`.

### A guarded divisor

A divisor that a missing row of a `missing: 0` parameter can make `0` is
refused at load, unless a guard leaves every missing row out where the quotient
is read. A guard is a condition that is false at a missing row, such as `c > 0`,
`c != 0` or the bare name `c`. It guards the quotient when it is one of the
conditions joined by `AND` in one of these places:

- the `where:` of the constraint,
- the `when:` of the case that holds the quotient,
- the `where:` of a variable that multiplies the quotient and is
`missing: absent`, because the product is absent where that variable is
absent. A variable with a `missing:` number reads that number outside its
mask, so its mask guards nothing.

```yaml
parameters:
c: { dims: [g], missing: 0 }
d: { dims: [g] }
constraints:
per_unit:
dims: [g]
where: "c > 0"
expression: y <= d / c
```

A guard holds only at the coordinate where it is read. A `sum`, a window, a
`shift` or an `at` reads other coordinates, so a guard outside one of them does
not guard a divisor inside it. A named expression is read over its own
dimensions, so a guard for its divisor goes in its own `cases:`.

## How absence travels

An absent term in arithmetic removes the whole row. An absent term inside a
Expand Down Expand Up @@ -174,8 +205,10 @@ a storage with inflow and no store, there is no row.

A variable takes `missing: absent` or a finite number, because a variable has
no data to refuse. A number needs a `where:`. It must lie in the `domain:` of
the variable, so a `binary` variable takes `0` or `1`, and inside its literal
`bounds:`. `to_spec` refuses the file otherwise. Inside a summing operator,
the variable, so a `binary` variable takes `0` or `1`, and `to_spec` refuses
the file otherwise. The `bounds:` hold only inside the mask, so the number may
lie outside them: a unit with `lower: 1` where it is built can have
`missing: 0` where it is not. Inside a summing operator,
`missing: 0` changes nothing, and another number adds itself once for each
coordinate that the mask leaves out. The typeset legend prints the number
where the mask leaves it out beside the variable.
Expand All @@ -194,5 +227,6 @@ divisor is absent, and where its divisor is exactly zero, whether a solve or the
data gave that zero. A solved value is read as the engine reads it back, so
an engine may read a value within its solver's tolerance of zero as zero. A
missing row of an `absent` divisor parameter makes the quotient absent, and
`to_spec` refuses a divisor that a `missing: 0` parameter can make zero. A
`to_spec` refuses a divisor that a `missing: 0` parameter can make zero,
unless a [guard](#a-guarded-divisor) leaves the missing row out. A
deleted row has [no dual](named.md#reading-a-constraints-dual).
2 changes: 2 additions & 0 deletions src/mathspec/_condition_resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
UnresolvedCountNode,
)
from mathspec._dimensions import join_dims
from mathspec._divisors import zero_divisor_refusals
from mathspec._expansion import expand
from mathspec._expression_parser import (
ArithmeticNode,
Expand Down Expand Up @@ -411,6 +412,7 @@ def _expression_comparison(self, node: UnresolvedComparisonNode) -> ExpressionCo
dims |= set(side.dims)
except LanguageError as e:
self.errors.append(str(e))
self.errors.extend(zero_divisor_refusals(side, context, ns.spec.parameters, {}))
if len(self.errors) > found:
return node
left, right = sides
Expand Down
171 changes: 171 additions & 0 deletions src/mathspec/_divisors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# SPDX-FileCopyrightText: mathspec Contributors
#
# SPDX-License-Identifier: MIT

"""The load-time refusal of a divisor that a missing row of a ``missing: 0`` parameter makes 0.

The language puts that 0 in, so no data can repair it. A guard spares the
divisor where it is false at every missing row: a conjunct of the ``where``
of the declaration, of the ``when`` of a case, or of the mask of a
``missing: absent`` variable that is a factor of the quotient, since an absent
factor makes the whole product absent. A guard holds only at the coordinate it is read at, so a
sum, a window, a shift or a lookup drops every guard from outside it.
"""

from __future__ import annotations

from typing import TYPE_CHECKING

from mathspec.program import (
Add,
Cases,
ConditionReference,
Divide,
ExpressionReference,
Join,
Mask,
Multiply,
Negate,
ParameterComparison,
ParameterDefined,
ParameterReference,
Power,
Sum,
Translate,
VariableReference,
WindowSum,
children,
)

if TYPE_CHECKING:
from collections.abc import Iterator, Mapping

from mathspec.program import ComparisonOperator, Condition, Expression
from mathspec.spec import ParameterSpec


def zero_divisor_refusals(
expression: Expression,
context: str,
parameters: Mapping[str, ParameterSpec],
masks: Mapping[str, Mask | None],
where: Mask | None = None,
) -> list[str]:
"""A refusal for each ``missing: 0`` parameter that can make a divisor under *expression* 0.

*masks* are the ``where`` masks of the variables that are absent outside
them, and *where* is the mask of the
declaration that holds *expression*. A named expression is not entered:
it is read over its own frame, so it is checked as its own entry.
"""
names = dict.fromkeys(_zero_divisors(expression, parameters, masks, _spared(where)))
return [
f"{context}: the divisor reads '{name}', which is declared missing: 0, so a missing row of '{name}' "
f'makes the divisor 0, which no number divides by. Declare missing: refused, absent, or a value other '
f'than 0, or guard the quotient with a where such as "{name} > 0".'
for name in names
]


def _zero_divisors(
e: Expression, parameters: Mapping[str, ParameterSpec], masks: Mapping[str, Mask | None], spared: frozenset[str]
) -> Iterator[str]:
"""Each parameter not in *spared* whose missing row zeroes a divisor under *e*."""
match e:
case Multiply() | Divide():
for factor in _factors(e):
if isinstance(factor, VariableReference):
spared |= _spared(masks.get(factor.name))
if isinstance(e, Divide) and (name := _zero_where_missing(e.divisor, parameters, spared)) is not None:
yield name
case Cases(regions=regions):
for region in regions:
yield from _zero_divisors(region.value, parameters, masks, spared | _spared(region.when))
return
case Sum() | WindowSum() | Translate() | Join():
spared = frozenset()
case ExpressionReference():
return
for child in children(e):
yield from _zero_divisors(child, parameters, masks, spared)


def _factors(e: Expression) -> tuple[Expression, ...]:
"""The factors of the product *e* heads, through a negation; a divisor is no factor, since it never is absent alone."""
match e:
case Multiply(left=left, right=right):
return _factors(left) + _factors(right)
case Divide(numerator=numerator):
return _factors(numerator)
case Negate(operand=operand):
return _factors(operand)
return (e,)


def _zero_where_missing(e: Expression, parameters: Mapping[str, ParameterSpec], spared: frozenset[str]) -> str | None:
"""The ``missing: 0`` parameter not in *spared* whose missing row makes *e* 0, or ``None`` where no missing row can.

A factor, the numerator of a quotient, the base of a power, and the
operand of a sum, a shift, a lookup or a named expression make *e* 0 where
they are 0; a sum of terms only where every term is, so ``p + 1`` never
is; a case where any region is.
"""
match e:
case ParameterReference(name=name):
declared = parameters.get(name)
return name if declared is not None and declared.missing == 0 and name not in spared else None
case Sum(operand=operand) | WindowSum(operand=operand) | Translate(operand=operand) | Join(operand=operand):
return _zero_where_missing(operand, parameters, frozenset())
case Negate(operand=operand) | ExpressionReference(body=operand):
return _zero_where_missing(operand, parameters, spared)
case Multiply(left=left, right=right):
return _zero_where_missing(left, parameters, spared) or _zero_where_missing(right, parameters, spared)
case Divide(numerator=numerator):
return _zero_where_missing(numerator, parameters, spared)
case Power(base=base):
return _zero_where_missing(base, parameters, spared)
case Add(left=left, right=right):
name = _zero_where_missing(left, parameters, spared)
return name if name is not None and _zero_where_missing(right, parameters, spared) is not None else None
case Cases(regions=regions):
return next(
(n for r in regions if (n := _zero_where_missing(r.value, parameters, spared | _spared(r.when)))),
None,
)
return None


def _spared(mask: Mask | None) -> frozenset[str]:
"""The parameters whose every missing row *mask* leaves out, read off its ``AND`` spine."""
if mask is None:
return frozenset()
return frozenset(name for conjunct in mask.conjuncts for name in _left_out(conjunct))


def _left_out(conjunct: Condition) -> frozenset[str]:
"""The parameter a conjunct is false at wherever its row is missing and it reads as 0."""
match conjunct:
case ParameterDefined(name=name):
return frozenset({name})
case ParameterComparison(name=name, op=op, value=float() | int() as value) if not _holds(0.0, op, value):
return frozenset({name})
case ConditionReference(body=body):
return _spared(Mask(body))
return frozenset()


def _holds(left: float, op: ComparisonOperator, right: float) -> bool:
"""Whether ``left <op> right`` holds."""
match op:
case '<':
return left < right
case '<=':
return left <= right
case '>':
return left > right
case '>=':
return left >= right
case '==':
return left == right
case '!=':
return left != right
49 changes: 0 additions & 49 deletions src/mathspec/_expression_resolver.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,10 @@
from mathspec.program import (
Add,
Axis,
Cases,
Constant,
Divide,
Dual,
Expression,
ExpressionReference,
Join,
JoinColumns,
Multiply,
Expand All @@ -71,7 +69,6 @@
from collections.abc import Mapping

from mathspec._resolution import Namespace
from mathspec.spec import ParameterSpec


#: An ``edge=`` as a translation carries it: whether it wraps, and the number
Expand All @@ -86,38 +83,6 @@
MAX_RESOLVED_DEPTH = 3 * MAX_DEPTH


def _zero_where_missing(e: Expression, parameters: Mapping[str, ParameterSpec]) -> str | None:
"""The ``missing: 0`` parameter whose missing row makes *e* 0, or ``None`` where no missing row can.

The language puts that 0 in, so a divisor it can zero is refused at load.
A factor, the numerator of a quotient, the base of a power, and the
operand of a sum, a shift, a lookup or a named expression make *e* 0 where
they are 0; a sum of terms only where every term is, so ``p + 1`` never
is; a case where any region is.
"""
match e:
case ParameterReference(name=name):
declared = parameters.get(name)
zero = declared is not None and declared.missing == 0
return name if zero else None
case Negate(operand=operand) | Sum(operand=operand) | WindowSum(operand=operand) | Translate(operand=operand):
return _zero_where_missing(operand, parameters)
case Join(operand=operand) | ExpressionReference(body=operand):
return _zero_where_missing(operand, parameters)
case Multiply(left=left, right=right):
return _zero_where_missing(left, parameters) or _zero_where_missing(right, parameters)
case Divide(numerator=numerator):
return _zero_where_missing(numerator, parameters)
case Power(base=base):
return _zero_where_missing(base, parameters)
case Add(left=left, right=right):
name = _zero_where_missing(left, parameters)
return name if name is not None and _zero_where_missing(right, parameters) is not None else None
case Cases(regions=regions):
return next((n for r in regions if (n := _zero_where_missing(r.value, parameters)) is not None), None)
return None


@dataclass(frozen=True)
class ExpressionResolver:
"""One resolution walk over an expression, and the three things every step of it reads.
Expand Down Expand Up @@ -208,26 +173,12 @@ def _binary(self, node: BinaryOperatorNode) -> Expression | None:
case '*':
return Multiply(left, right, dims=dims)
case '/':
if self._zero_divisor(right):
return None
return Divide(left, right, dims=dims)
case '**':
return Power(left, right, dims=dims)
case _:
assert_never(node.op)

def _zero_divisor(self, divisor: Expression) -> bool:
"""Whether a missing row of a ``missing: 0`` parameter can make *divisor* 0, recorded as an error."""
name = _zero_where_missing(divisor, self.ns.spec.parameters)
if name is None:
return False
self.errors.append(
f"{self.context}: the divisor reads '{name}', which is declared missing: 0, so a missing row of '{name}' "
f'makes the divisor 0, which no number divides by. Declare missing: refused, absent, or a value other '
f'than 0.'
)
return True

def _name(self, node: NameNode) -> Expression | None:
"""A bare name as the variable, parameter or named expression it declares; a dimension or relation is not a value.

Expand Down
Loading
Loading