diff --git a/CHANGELOG.md b/CHANGELOG.md index 90de451..a8b69fa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,10 +25,20 @@ their reference density and heat capacity (36 repeated blocks in `ECCOV4r4_native.yaml` alone). - `collect_budgets(..., assert_unit_consistency=True)` enforces units: each - budget that declares a top-level `units` must have its side roots infer units - convertible to the declared ones, raising `UnitError` (exported) otherwise. - Left at its default of `False`, a reconciliation failure only warns; inference - and stamping happen either way. + budget that declares a top-level `units` must have its side roots infer *the + same* units — same dimension **and** same magnitude — raising `UnitError` + (exported) otherwise. Left at its default of `False`, a reconciliation failure + only warns; inference and stamping happen either way. A side root that never + materialized is skipped, since absent inputs are `on_missing`'s business. +- Magnitude is checked because a leading numeric factor is part of a UDUNITS + spec, and so part of what a stamped attribute claims: a root inferring + `0.001 kg s-1` while holding values already in kg/s is mislabelled by a + thousand. That is what a conversion factor written as a bare (dimensionless) + number does when it cancels a scale carried in an input's own `units` — the + case CF's `1e-3` practical salinity creates. Such a factor must declare the + reciprocal scale as its units (`{value: 0.001, units: "1e3"}`), which + `ECCOV4r4_native.yaml` now does; its salt budget previously stamped + `0.001 kg s-1` on correct kg/s values. - `BudgetQuery` gained `.units(term)` (the stamped units of a materialized term) and `.budget_units(budget)` (the declared target). - The recipe display (`show_recipe`, and a `BudgetQuery` shown in a notebook) @@ -36,6 +46,11 @@ [kg m-3]`, `lateral [×] [W]`. `show_recipe` takes an optional third argument, the grid/dataset to read raw diagnostics' units from, so a recipe can be read with its full unit arithmetic before anything is collected. +- `units.infer_units(node, lookup, available=)` takes an optional + "which diagnostics does the dataset hold?" predicate, and with it answers what + a run will *build* rather than what the recipe *describes* — dropping the + operands the evaluator drops instead of letting one absent diagnostic leave + every term above it unannotated. The display passes it whenever it has data. - Practical salinity (`psu`), which UDUNITS does not define, is treated as dimensionless (PSS-78); offset temperature (`degC`) is handled by cf-units' in-product dimensionalization, so heat-content terms reconcile to `W`. @@ -43,6 +58,20 @@ ### Engine +- A recipe using `lateral_divergence` without an `xgcm.Grid` now raises the same + clear `ValueError` a grid-less `difference` does, instead of failing later on + the grid-dependent rechunking bookkeeping. +- Missing diagnostics are reported before unit issues: an absent input *causes* + unknown units, so under `on_missing="raise"` the error now names the + diagnostic rather than the symptom. +- Rechunking is a no-op on unchunked (in-memory) data, which is already one chunk + along everything — `difference` and `lateral_divergence` no longer hand back a + lazy result where the caller passed an eager one. +- A budget-level key written on a term (`units:` on a term rather than beside + `lhs`/`rhs`) gets its own warning instead of the generic "you may be missing an + enclosing `product`" hint, which pointed at the wrong fix. +- A `constants` entry with no `value` raises `BudgetParseError` naming the path, + not a bare `KeyError`; and a budget named `constants` says the key is reserved. - `lateral_divergence` now gets the same `allow_rechunk` treatment `difference` has always had: its flux operands are put into a single chunk along the differenced dimensions while xgcm works, and the caller's chunking is restored diff --git a/CLAUDE.md b/CLAUDE.md index 18ca834..197b388 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -64,7 +64,8 @@ recipe ──parse_budgets──▶ typed tree (nodes.py) ──evaluate_budgets - **Naming.** One variable per node/operation with operator infixes dropped; no duplicate "copy" variables (108 → 57 on the MOM6 example; 73 on the ECCOv4r4 native-grid recipe, whose volume budget is Eulerian-only — the GM bolus is exposed as `bolus` metadata, not a budget term). `CHANGELOG.md` has the old→new table for anyone migrating from 0.6.x. - **Missing diagnostics are skipped, not fatal** — a `sum` drops only the missing operand and builds from the rest (flagged `xbudget_incomplete`); a `product` with a missing *required* factor is dropped entirely (an unknown factor is not a zero one). `collect_budgets(on_missing="warn"|"raise"|"ignore")` sets the policy; `optional: true` on a term exempts its subtree. Query it back with `BudgetQuery.missing()`. - **Rechunking:** `allow_rechunk=True` (default) temporarily rechunks the differenced dimension(s) into a single chunk (required by `grid.diff`) then restores the caller's chunking. This applies to **both** `difference` and `lateral_divergence` — they share `_dataset_chunks`/`_single_chunk_along`/`_restore_chunks` in `evaluate.py`. On the ECCO LLC90 budgets it is numerically identical and roughly halves wall clock. -- **Units.** Every emitted variable is stamped with a `units` attr inferred through the tree (`units.py`, UDUNITS via `cf-units`); `collect_budgets(..., assert_unit_consistency=True)` additionally requires each budget's side roots to reconcile to its declared top-level `units`, raising `UnitError` (it only warns by default). `units.infer_units(node, lookup)` answers the same question statically, off the typed tree, which is what lets `show_recipe(recipe, budget, data)` and a notebook-displayed `BudgetQuery` annotate every node with `[units]`. +- **Units.** Every emitted variable is stamped with a `units` attr inferred through the tree (`units.py`, UDUNITS via `cf-units`); `collect_budgets(..., assert_unit_consistency=True)` additionally requires each budget's side roots to reconcile to its declared top-level `units` — same dimension **and** same magnitude — raising `UnitError` (it only warns by default). A side root that never materialized is exempt (that is `on_missing`'s job). Magnitude matters because a UDUNITS spec may carry a leading numeric scale: a conversion factor cancelling a scale in an input's own units (CF writes practical salinity as `1e-3`) must declare the reciprocal scale as *its* units (`{value: 0.001, units: "1e3"}`), or the term stamps itself `0.001 kg s-1` over correct kg/s values. +- **Static unit inference.** `units.infer_units(node, lookup, available=)` answers the units question off the typed tree, without evaluating — which is what lets `show_recipe(recipe, budget, data)` and a notebook-displayed `BudgetQuery` annotate every node with `[units]`. Passing `available` (a `name -> bool` over the dataset) makes it mirror the *run* rather than the recipe: operands the evaluator would drop are dropped here too, instead of one absent diagnostic leaving every ancestor unannotated. The `_DROPPED` sentinel is what distinguishes "not built" from "built but unknown". - **Lenient parser.** `parse.py` **warns and skips** unavailable-diagnostic placeholders (e.g. a `null`-source `difference`) and stray non-operation keys instead of failing, so real recipes with such terms still load. A stray key usually means an enclosing `product:` was omitted (its factors are then silently dropped rather than multiplied), so those warnings are worth reading rather than filtering. - **xgcm version:** requires **xgcm >= 0.10.0** — `lateral_divergence` needs native face-connected differencing, first shipped there. 0.10.0 also removed `xgcm.Grid`'s `periodic` argument and renamed `boundary` to `padding` (both now raise `ValueError`), so every grid construction in `examples/` and `tests/` uses `padding=`; `periodic=False` translates to `padding="fill"`. diff --git a/docs/quickstart.md b/docs/quickstart.md index 46e745e..be22d74 100644 --- a/docs/quickstart.md +++ b/docs/quickstart.md @@ -96,6 +96,14 @@ Each term is named after its path through the recipe, joined with underscores `heat` / `rhs` / `advection` becomes `heat_rhs_advection`. Nothing to memorize, and nothing to parse back out. +```{note} +A budget's metadata sits alongside its sides in that result, so a recipe +declaring `units: "kg s-1"` or `lambda: "thetao"` gives +`{'mass': {'units': 'kg s-1', 'lhs': {...}, 'rhs': {...}}}`. Iterate with +`q.aggregate()["mass"]["rhs"]` rather than over every value, or you will hit a +metadata string where you expected a dict of terms. +``` + ## Does it close? This is the question xbudget exists to answer: @@ -205,7 +213,8 @@ Two things this page skipped, both of which real budgets need: - **Grids.** Terms built by differencing a staggered flux (or by a lateral divergence) need an `xgcm.Grid` instead of a plain `Dataset`, so xbudget knows - the discretization. Pass the grid to `collect_budgets` in exactly the same way. + the discretization. Pass the grid to `collect_budgets` in exactly the same way; + [Writing a recipe](the-grid) shows how to build one. - **Scale.** The examples below run against real model output, with dask. For a full worked budget, see the diff --git a/docs/recipes.md b/docs/recipes.md index e8f26a3..f6154a1 100644 --- a/docs/recipes.md +++ b/docs/recipes.md @@ -86,9 +86,29 @@ product: area: "areacello" ``` -Only genuinely dimensional constants need this (`density`, `specific_heat_capacity`); -`sign`, salinity-to-mass conversion factors, and the like are dimensionless and -stay bare. See the *Units* section below. +A genuinely dimensionless factor — a `sign`, a fraction — stays bare. + +````{warning} +A **unit-conversion factor is the one thing that looks dimensionless and is +not.** If it exists to cancel a scale that an input carries in its own `units` +string, it has to declare the reciprocal scale, or the scale survives into the +result and the stamped `units` misreport the values by exactly that factor. + +CF writes practical salinity as a scale: ECCO's salt fluxes are `1e-3 m3 s-1`. +A recipe multiplying one by `0.001` to reach kg/s must say so: + +```yaml +constants: + salinity_scale: + value: 0.001 + units: "1e3" # value x units == 1: numerically the identity, + # dimensionally the thing that cancels the input's 1e-3 +``` + +Written as a bare `0.001` instead, the values are still right but the term +stamps itself `0.001 kg s-1` — a thousand-fold lie to anyone who reads the +attribute. `assert_unit_consistency=True` catches this (see *Units* below). +```` **Shared constants live at the root.** A reference density recurs at nearly every term of a budget, and repeating the mapping at each use is both tedious and a @@ -114,10 +134,14 @@ mass: Entries take the same two forms an inline constant does (a bare number, or a `{value:, units:}` mapping). `constants` is the one top-level key that is not a -budget. A string operand is looked up there first and otherwise treated as a -diagnostic name, so a constant sharing a name with a dataset variable shadows it -— name them distinctly. All the shipped recipes use this for their reference -density and heat capacity. +budget, so a budget cannot be named that. + +References are resolved for the operands of a `sum` or a `product`: a string +there is looked up in the table first, and otherwise treated as a diagnostic +name. (The operand of a `difference` or a `reciprocal` is always a diagnostic — +differencing a constant is not a thing you want.) A constant sharing a name with +a dataset variable therefore *shadows* it — name them distinctly. All the shipped +recipes use the table for their reference density and heat capacity. [udunits]: https://www.unidata.ucar.edu/software/udunits/ @@ -173,9 +197,20 @@ mass: ``` With `assert_unit_consistency=True`, this is enforced: each budget that declares -`units` must have its `lhs`/`rhs` roots infer units convertible to the declared -ones, or `collect_budgets` raises `xbudget.UnitError`. Left at its default of -`False`, a mismatch only warns — inference and stamping happen regardless. +`units` must have its `lhs`/`rhs` roots infer *the same* units — same dimension +and same magnitude — or `collect_budgets` raises `xbudget.UnitError`. Left at its +default of `False`, a mismatch only warns — inference and stamping happen +regardless. + +Magnitude counts because a leading numeric factor is part of a UDUNITS spec, and +therefore part of what a stamped attribute claims: a root inferring +`0.001 kg s-1` while holding values already in kg/s is mislabelled by a thousand. +That is the conversion-factor trap above, and this is what catches it. + +One thing it does *not* check: a side root that never materialized, because a +diagnostic it needs was absent, is skipped rather than failed. Absent inputs are +`on_missing`'s business — reporting the same problem twice under the name "units" +would only obscure it. ```python xbudget.collect_budgets(grid, recipe, assert_unit_consistency=True) # UnitError on a clash @@ -210,10 +245,16 @@ inspect one before ever running it. Each node also carries its units in square brackets — `density: 1035. [kg m-3]`, `lateral [×] [W]` — composed up the tree by the same arithmetic -`assert_unit_consistency` checks. Units enter at the leaves, so pass the data you -intend to collect against (as the third argument above) and the whole tree -resolves; reading it top-down then shows exactly *where* a unit stops being -inferable. Without it, only the recipe's declared constants are annotated. +`assert_unit_consistency` checks (`xbudget.units.infer_units`, if you want to +call it yourself). Units enter at the leaves, so pass the data you intend to +collect against (as the third argument above) and the whole tree resolves; +reading it top-down then shows exactly *where* a unit stops being inferable. +Without it, only the recipe's declared constants are annotated. + +Given the data, the annotation answers *what a run will build*, not *what the +recipe describes*: a term whose diagnostics are absent is left blank, and its +parents still show their units, because that is what the run will produce from +the operands that remain. Once you *have* run a budget, displaying the query does the same but annotated with each term's resolved variable name — with terms whose diagnostics were @@ -237,6 +278,45 @@ xbudget.BudgetQuery(grid, recipe) # same tree, annotated with run state `sum` and `product` take any number of operands, each of which may itself be a sub-term, so budgets nest as deeply as you need. +(the-grid)= +### The grid + +`sum`, `product` and `reciprocal` work on a plain `xarray.Dataset`. The two +operations that cross cell faces — `difference` and `lateral_divergence` — need +an `xgcm.Grid`, because only the grid knows which dimension is staggered against +which. Pass one in place of the dataset and everything else is unchanged: + +```python +import xgcm + +grid = xgcm.Grid( + ds, + coords={ # your staggered dimension names + "X": {"center": "xh", "outer": "xq"}, + "Y": {"center": "yh", "outer": "yq"}, + }, + metrics={("X", "Y"): ["areacello"]}, # optional + padding="fill", # boundary condition + autoparse_metadata=False, +) +xbudget.collect_budgets(grid, recipe) # derived vars land in grid._ds +``` + +Position names are xgcm's (`center` plus one of `left`/`right`/`inner`/`outer`). +Without a grid, a recipe using either operation fails immediately with +`Input 'data' must be an 'xgcm.Grid' instance…` rather than producing something +subtly wrong. + +```{note} +xbudget requires **xgcm ≥ 0.10**, which renamed `boundary=` to `padding=` and +removed `periodic=` — both now raise `ValueError`. Older tutorials and answers +still show the previous spelling; `periodic=False` becomes `padding="fill"`. +``` + +For a face-connected topology (the ECCO LLC90 tiles, say), also pass +`face_connections`; `lateral_divergence` then stitches the seams correctly. See +`examples/load_example_ecco_grid.py` for a complete 13-tile grid. + **`difference`** takes a single operand and differences it across whichever axis it is staggered on — the axis is inferred from the operand's dimensions, so you do not name it: diff --git a/examples/MOM6_budget_examples_mass_heat_salt.ipynb b/examples/MOM6_budget_examples_mass_heat_salt.ipynb index cf37f27..cebbc7e 100644 --- a/examples/MOM6_budget_examples_mass_heat_salt.ipynb +++ b/examples/MOM6_budget_examples_mass_heat_salt.ipynb @@ -6,10 +6,10 @@ "id": "182a8d3a-09f8-448e-a029-d03db69bbe44", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:33.362422Z", - "iopub.status.busy": "2026-07-28T19:27:33.362027Z", - "iopub.status.idle": "2026-07-28T19:27:33.523729Z", - "shell.execute_reply": "2026-07-28T19:27:33.523413Z" + "iopub.execute_input": "2026-07-28T20:48:59.697600Z", + "iopub.status.busy": "2026-07-28T20:48:59.697309Z", + "iopub.status.idle": "2026-07-28T20:48:59.858677Z", + "shell.execute_reply": "2026-07-28T20:48:59.858334Z" } }, "outputs": [], @@ -24,10 +24,10 @@ "id": "a5f0eef4-7368-4f12-bc35-e09387046539", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:33.525645Z", - "iopub.status.busy": "2026-07-28T19:27:33.525528Z", - "iopub.status.idle": "2026-07-28T19:27:35.076743Z", - "shell.execute_reply": "2026-07-28T19:27:35.076428Z" + "iopub.execute_input": "2026-07-28T20:48:59.860799Z", + "iopub.status.busy": "2026-07-28T20:48:59.860664Z", + "iopub.status.idle": "2026-07-28T20:49:01.297665Z", + "shell.execute_reply": "2026-07-28T20:49:01.297334Z" } }, "outputs": [], @@ -61,10 +61,10 @@ "id": "5aa1b5c9-60d4-4b5e-9c7d-d0c25efb5093", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:35.078522Z", - "iopub.status.busy": "2026-07-28T19:27:35.078366Z", - "iopub.status.idle": "2026-07-28T19:27:36.164349Z", - "shell.execute_reply": "2026-07-28T19:27:36.163987Z" + "iopub.execute_input": "2026-07-28T20:49:01.299719Z", + "iopub.status.busy": "2026-07-28T20:49:01.299517Z", + "iopub.status.idle": "2026-07-28T20:49:02.608288Z", + "shell.execute_reply": "2026-07-28T20:49:02.607930Z" } }, "outputs": [ @@ -95,10 +95,10 @@ "id": "449ee6d3-57ca-4e3d-94dc-076887309454", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:36.166023Z", - "iopub.status.busy": "2026-07-28T19:27:36.165928Z", - "iopub.status.idle": "2026-07-28T19:27:36.212256Z", - "shell.execute_reply": "2026-07-28T19:27:36.211968Z" + "iopub.execute_input": "2026-07-28T20:49:02.610725Z", + "iopub.status.busy": "2026-07-28T20:49:02.610614Z", + "iopub.status.idle": "2026-07-28T20:49:02.660840Z", + "shell.execute_reply": "2026-07-28T20:49:02.660417Z" } }, "outputs": [], @@ -123,10 +123,10 @@ "id": "2d158238-b8a5-4fe5-8a59-7198d9e759ce", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:36.213742Z", - "iopub.status.busy": "2026-07-28T19:27:36.213659Z", - "iopub.status.idle": "2026-07-28T19:27:36.234348Z", - "shell.execute_reply": "2026-07-28T19:27:36.234132Z" + "iopub.execute_input": "2026-07-28T20:49:02.662529Z", + "iopub.status.busy": "2026-07-28T20:49:02.662441Z", + "iopub.status.idle": "2026-07-28T20:49:02.689895Z", + "shell.execute_reply": "2026-07-28T20:49:02.689609Z" } }, "outputs": [ @@ -305,10 +305,10 @@ "id": "527f1b10", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:36.235573Z", - "iopub.status.busy": "2026-07-28T19:27:36.235494Z", - "iopub.status.idle": "2026-07-28T19:27:36.380448Z", - "shell.execute_reply": "2026-07-28T19:27:36.380186Z" + "iopub.execute_input": "2026-07-28T20:49:02.691270Z", + "iopub.status.busy": "2026-07-28T20:49:02.691161Z", + "iopub.status.idle": "2026-07-28T20:49:02.850291Z", + "shell.execute_reply": "2026-07-28T20:49:02.850022Z" } }, "outputs": [], @@ -334,10 +334,10 @@ "id": "aa8797da-0f49-40aa-be98-b03ba49b380a", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:36.382102Z", - "iopub.status.busy": "2026-07-28T19:27:36.382019Z", - "iopub.status.idle": "2026-07-28T19:27:36.392036Z", - "shell.execute_reply": "2026-07-28T19:27:36.391784Z" + "iopub.execute_input": "2026-07-28T20:49:02.851801Z", + "iopub.status.busy": "2026-07-28T20:49:02.851715Z", + "iopub.status.idle": "2026-07-28T20:49:02.863201Z", + "shell.execute_reply": "2026-07-28T20:49:02.862959Z" } }, "outputs": [ @@ -380,10 +380,10 @@ "id": "7c9bfd4c-aaff-44ca-b360-d3b43f3addf1", "metadata": { "execution": { - "iopub.execute_input": "2026-07-28T19:27:36.393349Z", - "iopub.status.busy": "2026-07-28T19:27:36.393269Z", - "iopub.status.idle": "2026-07-28T19:27:36.411833Z", - "shell.execute_reply": "2026-07-28T19:27:36.411594Z" + "iopub.execute_input": "2026-07-28T20:49:02.864542Z", + "iopub.status.busy": "2026-07-28T20:49:02.864464Z", + "iopub.status.idle": "2026-07-28T20:49:02.886341Z", + "shell.execute_reply": "2026-07-28T20:49:02.886097Z" } }, "outputs": [ @@ -851,7 +851,7 @@ " provenance: ['heat_lhs_advection_lateral', 'heat_lhs_advection_interfa...\n", " xbudget_path: ['heat', 'lhs', 'advection']\n", " xbudget_op: sum\n", - " units: W