Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 33 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,53 @@
their reference density and heat capacity (36 repeated blocks in
`ECCOV4r4_native.yaml` alone).
- `collect_budgets(..., assert_unit_consistency=True)` enforces units: each
budget that declares a top-level `units` must have its side roots infer units
convertible to the declared ones, raising `UnitError` (exported) otherwise.
Left at its default of `False`, a reconciliation failure only warns; inference
and stamping happen either way.
budget that declares a top-level `units` must have its side roots infer *the
same* units — same dimension **and** same magnitude — raising `UnitError`
(exported) otherwise. Left at its default of `False`, a reconciliation failure
only warns; inference and stamping happen either way. A side root that never
materialized is skipped, since absent inputs are `on_missing`'s business.
- Magnitude is checked because a leading numeric factor is part of a UDUNITS
spec, and so part of what a stamped attribute claims: a root inferring
`0.001 kg s-1` while holding values already in kg/s is mislabelled by a
thousand. That is what a conversion factor written as a bare (dimensionless)
number does when it cancels a scale carried in an input's own `units` — the
case CF's `1e-3` practical salinity creates. Such a factor must declare the
reciprocal scale as its units (`{value: 0.001, units: "1e3"}`), which
`ECCOV4r4_native.yaml` now does; its salt budget previously stamped
`0.001 kg s-1` on correct kg/s values.
- `BudgetQuery` gained `.units(term)` (the stamped units of a materialized
term) and `.budget_units(budget)` (the declared target).
- The recipe display (`show_recipe`, and a `BudgetQuery` shown in a notebook)
annotates every node with its units in square brackets — `density: 1035.
[kg m-3]`, `lateral [×] [W]`. `show_recipe` takes an optional third argument,
the grid/dataset to read raw diagnostics' units from, so a recipe can be read
with its full unit arithmetic before anything is collected.
- `units.infer_units(node, lookup, available=)` takes an optional
"which diagnostics does the dataset hold?" predicate, and with it answers what
a run will *build* rather than what the recipe *describes* — dropping the
operands the evaluator drops instead of letting one absent diagnostic leave
every term above it unannotated. The display passes it whenever it has data.
- Practical salinity (`psu`), which UDUNITS does not define, is treated as
dimensionless (PSS-78); offset temperature (`degC`) is handled by cf-units'
in-product dimensionalization, so heat-content terms reconcile to `W`.
- `cf-units` is now a dependency.

### Engine

- A recipe using `lateral_divergence` without an `xgcm.Grid` now raises the same
clear `ValueError` a grid-less `difference` does, instead of failing later on
the grid-dependent rechunking bookkeeping.
- Missing diagnostics are reported before unit issues: an absent input *causes*
unknown units, so under `on_missing="raise"` the error now names the
diagnostic rather than the symptom.
- Rechunking is a no-op on unchunked (in-memory) data, which is already one chunk
along everything — `difference` and `lateral_divergence` no longer hand back a
lazy result where the caller passed an eager one.
- A budget-level key written on a term (`units:` on a term rather than beside
`lhs`/`rhs`) gets its own warning instead of the generic "you may be missing an
enclosing `product`" hint, which pointed at the wrong fix.
- A `constants` entry with no `value` raises `BudgetParseError` naming the path,
not a bare `KeyError`; and a budget named `constants` says the key is reserved.
- `lateral_divergence` now gets the same `allow_rechunk` treatment `difference`
has always had: its flux operands are put into a single chunk along the
differenced dimensions while xgcm works, and the caller's chunking is restored
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,8 @@ recipe ──parse_budgets──▶ typed tree (nodes.py) ──evaluate_budgets
- **Naming.** One variable per node/operation with operator infixes dropped; no duplicate "copy" variables (108 → 57 on the MOM6 example; 73 on the ECCOv4r4 native-grid recipe, whose volume budget is Eulerian-only — the GM bolus is exposed as `bolus` metadata, not a budget term). `CHANGELOG.md` has the old→new table for anyone migrating from 0.6.x.
- **Missing diagnostics are skipped, not fatal** — a `sum` drops only the missing operand and builds from the rest (flagged `xbudget_incomplete`); a `product` with a missing *required* factor is dropped entirely (an unknown factor is not a zero one). `collect_budgets(on_missing="warn"|"raise"|"ignore")` sets the policy; `optional: true` on a term exempts its subtree. Query it back with `BudgetQuery.missing()`.
- **Rechunking:** `allow_rechunk=True` (default) temporarily rechunks the differenced dimension(s) into a single chunk (required by `grid.diff`) then restores the caller's chunking. This applies to **both** `difference` and `lateral_divergence` — they share `_dataset_chunks`/`_single_chunk_along`/`_restore_chunks` in `evaluate.py`. On the ECCO LLC90 budgets it is numerically identical and roughly halves wall clock.
- **Units.** Every emitted variable is stamped with a `units` attr inferred through the tree (`units.py`, UDUNITS via `cf-units`); `collect_budgets(..., assert_unit_consistency=True)` additionally requires each budget's side roots to reconcile to its declared top-level `units`, raising `UnitError` (it only warns by default). `units.infer_units(node, lookup)` answers the same question statically, off the typed tree, which is what lets `show_recipe(recipe, budget, data)` and a notebook-displayed `BudgetQuery` annotate every node with `[units]`.
- **Units.** Every emitted variable is stamped with a `units` attr inferred through the tree (`units.py`, UDUNITS via `cf-units`); `collect_budgets(..., assert_unit_consistency=True)` additionally requires each budget's side roots to reconcile to its declared top-level `units` — same dimension **and** same magnitude — raising `UnitError` (it only warns by default). A side root that never materialized is exempt (that is `on_missing`'s job). Magnitude matters because a UDUNITS spec may carry a leading numeric scale: a conversion factor cancelling a scale in an input's own units (CF writes practical salinity as `1e-3`) must declare the reciprocal scale as *its* units (`{value: 0.001, units: "1e3"}`), or the term stamps itself `0.001 kg s-1` over correct kg/s values.
- **Static unit inference.** `units.infer_units(node, lookup, available=)` answers the units question off the typed tree, without evaluating — which is what lets `show_recipe(recipe, budget, data)` and a notebook-displayed `BudgetQuery` annotate every node with `[units]`. Passing `available` (a `name -> bool` over the dataset) makes it mirror the *run* rather than the recipe: operands the evaluator would drop are dropped here too, instead of one absent diagnostic leaving every ancestor unannotated. The `_DROPPED` sentinel is what distinguishes "not built" from "built but unknown".
- **Lenient parser.** `parse.py` **warns and skips** unavailable-diagnostic placeholders (e.g. a `null`-source `difference`) and stray non-operation keys instead of failing, so real recipes with such terms still load. A stray key usually means an enclosing `product:` was omitted (its factors are then silently dropped rather than multiplied), so those warnings are worth reading rather than filtering.
- **xgcm version:** requires **xgcm >= 0.10.0** — `lateral_divergence` needs native face-connected differencing, first shipped there. 0.10.0 also removed `xgcm.Grid`'s `periodic` argument and renamed `boundary` to `padding` (both now raise `ValueError`), so every grid construction in `examples/` and `tests/` uses `padding=`; `periodic=False` translates to `padding="fill"`.

Expand Down
11 changes: 10 additions & 1 deletion docs/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,14 @@ Each term is named after its path through the recipe, joined with underscores
`heat` / `rhs` / `advection` becomes `heat_rhs_advection`. Nothing to memorize,
and nothing to parse back out.

```{note}
A budget's metadata sits alongside its sides in that result, so a recipe
declaring `units: "kg s-1"` or `lambda: "thetao"` gives
`{'mass': {'units': 'kg s-1', 'lhs': {...}, 'rhs': {...}}}`. Iterate with
`q.aggregate()["mass"]["rhs"]` rather than over every value, or you will hit a
metadata string where you expected a dict of terms.
```

## Does it close?

This is the question xbudget exists to answer:
Expand Down Expand Up @@ -205,7 +213,8 @@ Two things this page skipped, both of which real budgets need:

- **Grids.** Terms built by differencing a staggered flux (or by a lateral
divergence) need an `xgcm.Grid` instead of a plain `Dataset`, so xbudget knows
the discretization. Pass the grid to `collect_budgets` in exactly the same way.
the discretization. Pass the grid to `collect_budgets` in exactly the same way;
[Writing a recipe](the-grid) shows how to build one.
- **Scale.** The examples below run against real model output, with dask.

For a full worked budget, see the
Expand Down
108 changes: 94 additions & 14 deletions docs/recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,29 @@ product:
area: "areacello"
```

Only genuinely dimensional constants need this (`density`, `specific_heat_capacity`);
`sign`, salinity-to-mass conversion factors, and the like are dimensionless and
stay bare. See the *Units* section below.
A genuinely dimensionless factor — a `sign`, a fraction — stays bare.

````{warning}
A **unit-conversion factor is the one thing that looks dimensionless and is
not.** If it exists to cancel a scale that an input carries in its own `units`
string, it has to declare the reciprocal scale, or the scale survives into the
result and the stamped `units` misreport the values by exactly that factor.

CF writes practical salinity as a scale: ECCO's salt fluxes are `1e-3 m3 s-1`.
A recipe multiplying one by `0.001` to reach kg/s must say so:

```yaml
constants:
salinity_scale:
value: 0.001
units: "1e3" # value x units == 1: numerically the identity,
# dimensionally the thing that cancels the input's 1e-3
```

Written as a bare `0.001` instead, the values are still right but the term
stamps itself `0.001 kg s-1` — a thousand-fold lie to anyone who reads the
attribute. `assert_unit_consistency=True` catches this (see *Units* below).
````

**Shared constants live at the root.** A reference density recurs at nearly every
term of a budget, and repeating the mapping at each use is both tedious and a
Expand All @@ -114,10 +134,14 @@ mass:

Entries take the same two forms an inline constant does (a bare number, or a
`{value:, units:}` mapping). `constants` is the one top-level key that is not a
budget. A string operand is looked up there first and otherwise treated as a
diagnostic name, so a constant sharing a name with a dataset variable shadows it
— name them distinctly. All the shipped recipes use this for their reference
density and heat capacity.
budget, so a budget cannot be named that.

References are resolved for the operands of a `sum` or a `product`: a string
there is looked up in the table first, and otherwise treated as a diagnostic
name. (The operand of a `difference` or a `reciprocal` is always a diagnostic —
differencing a constant is not a thing you want.) A constant sharing a name with
a dataset variable therefore *shadows* it — name them distinctly. All the shipped
recipes use the table for their reference density and heat capacity.

[udunits]: https://www.unidata.ucar.edu/software/udunits/

Expand Down Expand Up @@ -173,9 +197,20 @@ mass:
```

With `assert_unit_consistency=True`, this is enforced: each budget that declares
`units` must have its `lhs`/`rhs` roots infer units convertible to the declared
ones, or `collect_budgets` raises `xbudget.UnitError`. Left at its default of
`False`, a mismatch only warns — inference and stamping happen regardless.
`units` must have its `lhs`/`rhs` roots infer *the same* units — same dimension
and same magnitude — or `collect_budgets` raises `xbudget.UnitError`. Left at its
default of `False`, a mismatch only warns — inference and stamping happen
regardless.

Magnitude counts because a leading numeric factor is part of a UDUNITS spec, and
therefore part of what a stamped attribute claims: a root inferring
`0.001 kg s-1` while holding values already in kg/s is mislabelled by a thousand.
That is the conversion-factor trap above, and this is what catches it.

One thing it does *not* check: a side root that never materialized, because a
diagnostic it needs was absent, is skipped rather than failed. Absent inputs are
`on_missing`'s business — reporting the same problem twice under the name "units"
would only obscure it.

```python
xbudget.collect_budgets(grid, recipe, assert_unit_consistency=True) # UnitError on a clash
Expand Down Expand Up @@ -210,10 +245,16 @@ inspect one before ever running it.

Each node also carries its units in square brackets — `density: 1035. [kg m-3]`,
`lateral [×] [W]` — composed up the tree by the same arithmetic
`assert_unit_consistency` checks. Units enter at the leaves, so pass the data you
intend to collect against (as the third argument above) and the whole tree
resolves; reading it top-down then shows exactly *where* a unit stops being
inferable. Without it, only the recipe's declared constants are annotated.
`assert_unit_consistency` checks (`xbudget.units.infer_units`, if you want to
call it yourself). Units enter at the leaves, so pass the data you intend to
collect against (as the third argument above) and the whole tree resolves;
reading it top-down then shows exactly *where* a unit stops being inferable.
Without it, only the recipe's declared constants are annotated.

Given the data, the annotation answers *what a run will build*, not *what the
recipe describes*: a term whose diagnostics are absent is left blank, and its
parents still show their units, because that is what the run will produce from
the operands that remain.

Once you *have* run a budget, displaying the query does the same but annotated
with each term's resolved variable name — with terms whose diagnostics were
Expand All @@ -237,6 +278,45 @@ xbudget.BudgetQuery(grid, recipe) # same tree, annotated with run state
`sum` and `product` take any number of operands, each of which may itself be a
sub-term, so budgets nest as deeply as you need.

(the-grid)=
### The grid

`sum`, `product` and `reciprocal` work on a plain `xarray.Dataset`. The two
operations that cross cell faces — `difference` and `lateral_divergence` — need
an `xgcm.Grid`, because only the grid knows which dimension is staggered against
which. Pass one in place of the dataset and everything else is unchanged:

```python
import xgcm

grid = xgcm.Grid(
ds,
coords={ # your staggered dimension names
"X": {"center": "xh", "outer": "xq"},
"Y": {"center": "yh", "outer": "yq"},
},
metrics={("X", "Y"): ["areacello"]}, # optional
padding="fill", # boundary condition
autoparse_metadata=False,
)
xbudget.collect_budgets(grid, recipe) # derived vars land in grid._ds
```

Position names are xgcm's (`center` plus one of `left`/`right`/`inner`/`outer`).
Without a grid, a recipe using either operation fails immediately with
`Input 'data' must be an 'xgcm.Grid' instance…` rather than producing something
subtly wrong.

```{note}
xbudget requires **xgcm ≥ 0.10**, which renamed `boundary=` to `padding=` and
removed `periodic=` — both now raise `ValueError`. Older tutorials and answers
still show the previous spelling; `periodic=False` becomes `padding="fill"`.
```

For a face-connected topology (the ECCO LLC90 tiles, say), also pass
`face_connections`; `lateral_divergence` then stitches the seams correctly. See
`examples/load_example_ecco_grid.py` for a complete 13-tile grid.

**`difference`** takes a single operand and differences it across whichever axis
it is staggered on — the axis is inferred from the operand's dimensions, so you
do not name it:
Expand Down
Loading