diff --git a/CLAUDE.md b/CLAUDE.md index 18669cf05..1b15f241c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,7 +14,7 @@ Four docs, kept short on purpose — **reference pages carry rules, design notes - `docs/SPEC.md` — the language reference: what a YAML file may contain and what it means. It opens with **§0, the ten laws**; every section below elaborates one. - `docs/ARCHITECTURE.md` — how it fits together, the hard rules, the module map. Update it in any PR that changes structure. - `docs/design/ceiling.md` — what may enter the language: the two tiers, the admissibility test, why sink capability is a second axis. -- `docs/ROADMAP.md` — what we build toward and what we have decided never to build. +- `docs/ROADMAP.md` — why the project exists and where it is going. No work items: those are issues. A PR that adds, renames, or retires a construct updates `docs/SPEC.md` — **§0 if it changes a law, the section if it changes a detail**. Rationale belongs in the PR description or a code comment, not in a new doc section; historical "this used to work differently" notes belong in git. @@ -22,7 +22,7 @@ A PR that adds, renames, or retires a construct updates `docs/SPEC.md` — **§0 Everything under `docs/` is also published as an mkdocs-material site (`mkdocs.yml`), built from those same files. Two rules: **a new page under `docs/` needs a `nav:` entry** or `mkdocs build --strict` fails in CI, and **a link to anything outside `docs/` is written as a full GitHub URL**, never as `../CONTRIBUTING.md` — the relative form resolves in the repo and 404s on the site, silently. Links inside `docs/` stay relative. `tests/test_docs_site.py` enforces both; see *the docs* in [CONTRIBUTING.md](CONTRIBUTING.md). -**Before proposing a new language feature**, triage it: **macro, primitive, or escape?** Most requests are compositions (macro, free); a genuinely new shape earns a primitive only if it clears the expressive ceiling in `docs/design/ceiling.md` (degree 1 ∩ relational ∩ local); unsayable math goes to a declared `escape:` island (#38) rather than into the language. Check the deliberate non-primitives in `docs/ROADMAP.md` first — parity with another tool is not by itself a reason to add anything. +**Before proposing a new language feature**, triage it: **macro, primitive, or escape?** Most requests are compositions (macro, free); a genuinely new shape earns a primitive only if it clears the expressive ceiling in `docs/design/ceiling.md` (degree 1 ∩ relational ∩ local); unsayable math goes to a declared `escape:` island (#38) rather than into the language. Check the deliberate non-primitives in `docs/design/ceiling.md` first — parity with another tool is not by itself a reason to add anything. ## Common Commands diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f9007df72..14d40fb07 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -156,7 +156,7 @@ declared `escape:` island rather than into the language. Read, in order: -1. [the deliberate non-primitives in docs/ROADMAP.md](docs/ROADMAP.md) — parity with +1. [the deliberate non-primitives](docs/design/ceiling.md#deliberate-non-primitives) — parity with another tool is not by itself a reason to add anything, and several plausible-sounding features are refused there on purpose; 2. [the ceiling in docs/design/ceiling.md](docs/design/ceiling.md#two-tiers-and-the-ceiling) — diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 7aa485d0d..5df46a494 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -22,7 +22,7 @@ property makes everything else legal: the whole model can be compiled — to eag xarray/linopy calls, or to a logical plan executed relationally and streamed to a sink — with both paths provably meaning the same thing. Every rule below protects it. (A *declared* memory ceiling is not something we have; see -[Track 4](ROADMAP.md#track-4--the-memory-axis).) +[the memory axis](ROADMAP.md#where-it-is-going).) **Four directories, four fences.** One produces the AST; three consume it and know nothing of each other. Each box below is a directory, and its subtitle is @@ -367,7 +367,7 @@ streams and no more: `cols` (bounds, objective coefficients, integrality), and `genconstr` — plus a semi-continuous threshold on `cols`. Unlike the three that exist, those two would land *unevenly*, because the destinations differ per sink (see "Capability is not the ceiling"); that unevenness is what -[Track 3](ROADMAP.md#track-3--capabilities-and-the-degree-line) exists to make declared rather +[Track 3](https://github.com/FBumann/lpspec/issues/472) exists to make declared rather than discovered at solve time. **A sink is one of two things, and the directory says which.** A **solver** diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index de3063932..7830c9646 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,175 +1,70 @@ # Roadmap -What we build toward and what we have decided never to build. **This page is the -index; the issues carry the argument.** - -## The ceiling, and what is not in it - -The closure that protects streaming is **relational ∩ local** — filter / join / -group-by-aggregate over tidy tables, pointwise or bounded-halo. That is -solver-independent and it is not moving; the argument is -[docs/design/ceiling.md](design/ceiling.md). - -Two things used to be stated as part of it and are not: - -- **Degree.** The language is degree 1 *today*. That is a scope line with a - sequence behind it (Track 3), not an admissibility rule — nothing about - `x * y` is relational or non-local. -- **What a sink can ingest.** SOS, indicator, semi-continuous and a Hessian are - **capability** questions per sink, not language questions. Conflating them let - one solver's limits read as architectural law. - -What *is* refused is at the [bottom of this page](#what-we-will-not-build), and -it is a shorter list than it used to be. - -**How we measure.** Not "can we say what other tools say?" but "what is sayable -in a language that must stay streamable?" A primitive is done when `lowering.py` -accepts it and the differential test against linopy passes. Coverage against -Calliope and PyPSA is generated by a test, not by hand -([#27](https://github.com/FBumann/lpspec/issues/27)); no numbers are published -until it produces them. The claim we lead on is **end-to-end cost to a loaded -solver** ([benchmarks](benchmarks.md)), not expressiveness. - -## Track 1 — primitives - -| # | Primitive | Surface | Unlocks | Locality | Status | -|---|---|---|---|---|---| -| 1–2 | Indexed access | `at(x, over=dim, index=value\|map)` | boundary conditions, multi-period linking, DC power flow, hydro cascades | pointwise | [#185](https://github.com/FBumann/lpspec/issues/185) | -| 3 | Coordinate helpers in `where` | `index(dim, i)`, negative from the end | "first/last snapshot" without hardcoding a label | coordinate-space | [#301](https://github.com/FBumann/lpspec/issues/301) | -| 4 | Window sum | `sum_next_n(x, over=dim, n=N)` | rolling horizons, DSR, reserves, min up/down | bounded-halo | needs 3 | -| 5 | Component-aware `where` | `defined(v)` over variables | masks compose across components instead of drifting | pointwise | needs design | -| 6 | `all`-reduction in `where` | `all(x, over=dim)` | reducing a mask over an out-of-`foreach` dim at all | pointwise | small | -| 7 | Bounds as expressions | — | signs, bidirectional flows, component libraries | pointwise | [#31](https://github.com/FBumann/lpspec/issues/31) | -| 8–9 | Namespacing, schema merge | — | component libraries in one lower/stream pass | n/a | [#29](https://github.com/FBumann/lpspec/issues/29), [#30](https://github.com/FBumann/lpspec/issues/30) | -| 10 | `cumsum`, data only | `cumsum(p, over=dim)`, variable-free | investment schedules, cumulative budgets | coordinate-space | [#384](https://github.com/FBumann/lpspec/issues/384) | -| 11 | Semi-continuous variables | `semi_continuous: true` | "off, or between min-stable and max", without the big-M pair | n/a | [#383](https://github.com/FBumann/lpspec/issues/383) | - -**Items 1–4 are the big one** — indexed access plus the masks that make it -usable, and the difference between "dispatch and balances" and "network and -multi-period math". Item 2 is the adjoint of `group_sum`: same mapping table, -join without the aggregate. - -**The window family is three questions.** `diff` is -`x - shift(x, over=d, by=1)` and is a macro today. Fixed-width windows are item -4, bounded-halo, the same class as `shift`. `cumsum` splits on what it reduces: -over data it is one column (item 10, free); over a **variable** row *t* carries -*t* terms, so an 8760-snapshot horizon emits ~38M nonzeros where the -state-variable recurrence emits ~26k. - -That cost is **not ours to optimise away** — build throughput is ours to fix and -must never gate what the language can say, but an inflated *emitted* model is -paid by the solver on every solve. So it is **priced, not refused**, and the -mechanism is [#380](https://github.com/FBumann/lpspec/issues/380). - -## Track 2 — the operational surface - -Everything a model needs after "it builds". Mostly queries over frames we -already materialise. - -- **Reading results.** Duals have shipped; reduced costs and slacks ride the - same join and have not. Derived results (LCOE, curtailment, emissions by - group) are SQL over the solution tables. -- **Infeasibility.** HiGHS has no IIS — Gurobi does, but an answer only the - opt-in sink can give is not the answer — so it is **elastic relaxation** - ([#80](https://github.com/FBumann/lpspec/issues/80)) — slacks with penalty - costs, then a query of nonzero slacks grouped by block. Needs no solver - feature and works on every sink. Taxed like a primitive, since new variables - mean a schema-level expansion pass. -- **The REPL gap is verbs, not structure.** A built model is four polars frames, - which answers "what is in row 12" with a filter where a labelled `Dataset` - needs a scatter — `build()` already returns the live executor, it just has - nothing readable on it. Render a **bound** row, preview which rows survive a - `where`, evaluate an expression against bound data, coefficient ranges, model - statistics. All **read-only**: inspecting a built model, never declaring one, - which is the line that keeps this cheap and rule 5 intact. -- **Lifecycle.** `var_label` *is* the solver column index with no remapping, so - value-only re-solve is a label query plus `changeColsBounds`, integrality is - the same contract (`changeColsIntegrality`, already called at build), and - **appending rows moves no label at all**. The session that holds the handle is - [#204](https://github.com/FBumann/lpspec/issues/204); warm starts are - [#382](https://github.com/FBumann/lpspec/issues/382), and they are what make - rebuild-instead-of-edit cheap rather than merely correct. -- **Decomposition.** Benders and successive substitution, and **the shape - favours us** — worth saying because the opposite is the natural assumption. - Decomposition wants sparse triplets plus label tables; that *is* the model - here, so the master/sub split is a `GROUP BY` over `A` - ([#39](https://github.com/FBumann/lpspec/issues/39)), cut coefficients are - `duals ⋈ subproblem rows`, and a cut lands with zero translation. The open - question is not feasibility but **who writes the cut** - ([#381](https://github.com/FBumann/lpspec/issues/381)). -- **AST consumers.** The first has shipped — `to_latex` / `to_typst` / - `to_markdown`, one tree walk, no data, no solver. Remaining: CLI - ([#35](https://github.com/FBumann/lpspec/issues/35)), observability - ([#34](https://github.com/FBumann/lpspec/issues/34)). - -## Track 3 — capabilities, and the degree line - -**The ceiling and sink capability are two axes.** A declared capability set per -**sink** (not per solver — `lp_file` is not a solver but has capabilities), -modelled on linopy's `Solver.features` with two divergences: entries are -three-valued (`native` / `reformulated` / `absent`) so satisfying one by -reformulation is additive later, and the model expresses **conjunction -exclusions**, because linopy declares HiGHS with `INTEGER_VARIABLES` *and* -`QUADRATIC_OBJECTIVE` in one flat `frozenset` while HiGHS refuses the pair. -`check(model, sink=...)` takes the sink optionally. Design: -[#89](https://github.com/FBumann/lpspec/issues/89). - -That unblocks three things, in order of effort: - -| | Blocked on | Note | -|---|---|---| -| **Semi-continuous** | nothing | HiGHS has `kSemiContinuous` natively and linopy has the oracle — [#383](https://github.com/FBumann/lpspec/issues/383) | -| **SOS / indicator** | the capability model | `lp_file` carries SOS as a text section, Gurobi natively, HiGHS not at all — [#23](https://github.com/FBumann/lpspec/issues/23) | -| **Quadratic** | the capability model — the second solver ships | below | - -**Quadratic is planned, not refused.** The cost side is settled and small, and -the oracle is free — linopy's `QuadraticExpression` builds the comparison, which -is normally the expensive half of a primitive. Performance is not the question. - -The blocker is *where it can land*: HiGHS returns `kError` for -`Hessian + integrality`, and `binary:`, `integer:` and nonconvex `piecewise:` all -ship today, so on the default path quadratic conflicts with features already in -the language. That is a capability finding, not a reason to refuse the math — so -it needs the table above **and** a solver without the exclusion. The second -solver is no longer the blocking half: the `gurobi` solver ships -([#106](https://github.com/FBumann/lpspec/issues/106)), and Gurobi takes a -Hessian alongside integrality. What is left is the capability table, without -which landing the primitive would ship math the *default* solver refuses. - -Until then `piecewise: {convex: true}` and the epigraph pattern are the answer -for convex 1-D curves — and they keep the LP duals, warm starts and MILP -compatibility a quadratic objective gives up, so they stay the *preferred* -spelling even after quadratic lands. The scope, the lowering, and whether -*coordinate-aligned* is the right restriction at all, are -[#261](https://github.com/FBumann/lpspec/issues/261) and -[#84](https://github.com/FBumann/lpspec/issues/84). - -## Track 4 — the memory axis - -The engine holds the model it builds, so peak tracks the model rather than a -number the caller sets. That is the right default and it is what makes the -lifetime disappear from the API, but there is **no declared ceiling** — no way -to say "build this within N gigabytes or fail". The honest version is -partition-wise execution, which the locality closure already guarantees is safe. -Worth most for the write path: the solver is the larger term by roughly an order -of magnitude at 10⁷ variables ([benchmarks](benchmarks.md)). - -## What we will not build - -| Request | Why | Instead | -|---|---|---| -| Data prep — resampling, clustering, IO, units | not math | preprocess; pass a parameter | -| Arbitrary array ops (`merge`, `reindex`) | unbounded; xarray with extra steps | data prep | -| Domain helpers (`reduce_carrier_dim`) | encodes one domain into the language | component libraries over generic primitives | -| Normalisation (`x / sum(x)`) | a *variable divisor* is rational, not polynomial — no sink takes it at any degree | state the ratio as a constraint, or fix the denominator | -| Conditionals, iteration, data-dependent structure **inside one plan** | destroys the closed AST | `where` masks + `foreach` dims. A *process* may loop over plans — that is Track 2 | -| A Python API for constructing models | hard rule 5 — the model is the file you review and diff | YAML. Whether Python may *emit* declarations is [#381](https://github.com/FBumann/lpspec/issues/381) | - -Genuinely unsayable math goes to a declared `escape:` island -([#38](https://github.com/FBumann/lpspec/issues/38)) — named in the file, -bounded by the preceding `where`, terminal, and billed against a label budget -before any Python runs. It buys back *relational* and *local*; it cannot buy -back degree, since it returns affine COO rows either way. +Why this project exists, and what it is becoming. **No work items live here.** +The work is issues, grouped under three parents: + +- [Track 1 — primitives](https://github.com/FBumann/lpspec/issues/470) +- [Track 2 — the operational surface](https://github.com/FBumann/lpspec/issues/471) +- [Track 3 — capabilities, and the degree line](https://github.com/FBumann/lpspec/issues/472) + +This page used to carry a table per track, and it lost to the tracker it +duplicated: rows described work that had already shipped, and code and tests +linked to sections — *Track 5*, *The degree axis*, *ROADMAP 2c* — that had never +existed or no longer did. An index maintained by hand beside an issue tracker is +a second copy that drifts. It is not one any more. + +## Why + +An optimisation model is math, and math is worth reading. It usually arrives as +Python that *builds* math — the equations entangled with the loops, the frames +and the library that assembled them, so a diff shows scaffolding rather than +constraints and nothing can read the model except the program that wrote it. +Reviewing such a model means reviewing a program, which is a different and much +harder job. + +lpspec makes the math the artifact. A YAML file says what the variables, +constraints and objective *are*; the file is validated at load time, built at +runtime, and can be reviewed by someone who understands the math without +understanding the builder. That is the whole thesis, and every rule below is +downstream of it. + +## Where it is going + +**One language, more than one place to run it.** The same file builds natively +on the relational engine or onto a `linopy.Model` that already exists in memory. +Not a fallback and not a dialect — one language, so a differential test between +the two lanes is an oracle rather than a comparison. + +**A build that streams, with a ceiling you can declare.** The model is frames +and the build is relational, so nothing dense is ever materialised and peak +tracks the model rather than a number someone guessed. What is missing is the +*declaration*: there is no way to say "build this within N gigabytes or fail". +The honest version is partition-wise execution, which the locality closure +already guarantees is safe. + +**Answers, not just solutions.** A solved model should tell you why it is +infeasible, what a row costs, and what changed since the last solve — without +opening a file no editor can hold. Most of that is a query over frames that +already exist. + +**Component libraries, composed rather than generated.** A fixed set of +parametrised templates agreeing on a port/flow convention, merged into one +program before a single build pass. **Topology is data** — wiring a system is +rows in a connectivity table, never generated YAML — so structure stays bounded +by the number of component *types* while cardinality lives entirely in data. + +## What it will not become + +**Two durable losses, and they are the price of the closed AST.** Structure that +needs the solver's *answer* to decide the next row, inside one plan; and +imperative modeling at all. What that price buys is load-time validation, two +lanes on one language, and a build that streams. Everything else is scheduling. + +The specific refusals — data prep, arbitrary array ops, domain helpers, +normalisation, in-plan conditionals, a Python modeling API — are in +[the ceiling](design/ceiling.md#deliberate-non-primitives), with the reason and +the rewrite for each. Read them before proposing a feature: parity with another +tool is not by itself a reason to add anything. ## Honest snapshot @@ -189,15 +84,10 @@ string and *uncalled* macro template. (labelled DataArrays vs tidy tables — `to_dataarray` bridges), debugging (IIS via Gurobi, `print()` of a row), lifecycle (mutate, re-solve, warm start, `relax`/`fix`), solver breadth (ten backends and four handoffs vs HiGHS-direct -plus LP files, [#106](https://github.com/FBumann/lpspec/issues/106)), and the -variable types and capabilities in Track 3. +plus LP files), and the variable types and capabilities behind the capability +model. -**The ranking this page implies:** indexed access blocks whole model classes -today; the Track 2 verbs block using the engine at 3am; solver breadth blocks +**The ranking this implies:** indexed access blocks whole model classes today; +the operational verbs block using the engine at 3am; solver breadth blocks arrival from linopy at all; semi-continuous and `cumsum`-over-data are cheap, unblocked and unscheduled. - -**Two durable losses.** Structure that needs the solver's *answer* to decide the -next row **inside one plan**, and imperative modeling at all (rule 5). Both are -the price of the closed AST — which is what buys load-time validation, two lanes -on one language, and a build that streams. Everything else here is scheduling. diff --git a/docs/SPEC.md b/docs/SPEC.md index 46b33f8da..a769603d0 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -503,7 +503,7 @@ language: nothing there changes what a file means. |---|---| | time-series processing (resample, cluster, interpolate, align), file IO, units | data prep; pass a parameter | | solver breadth | two solver sinks — HiGHS, which ships, and Gurobi via the `[gurobi]` extra — chosen with `solver_name` at the call, never in the file; LP files for everything else ([#106](https://github.com/FBumann/lpspec/issues/106)) | -| SOS and indicator constraints | planned, as a *sink capability* rather than a language question — the default solver has no such concept, `lp_file` and Gurobi do ([#23](https://github.com/FBumann/lpspec/issues/23), [ROADMAP Track 3](ROADMAP.md#track-3--capabilities-and-the-degree-line)). `piecewise:` (§4) covers SOS2's usual purpose today | +| SOS and indicator constraints | planned, as a *sink capability* rather than a language question — the default solver has no such concept, `lp_file` and Gurobi do ([#23](https://github.com/FBumann/lpspec/issues/23), [Track 3](https://github.com/FBumann/lpspec/issues/472)). `piecewise:` (§4) covers SOS2's usual purpose today | | multi-objective | one objective — declaring a second is a load error (§2); weight them into one expression | | schema migrations | — | | arbitrary array ops (`merge`, `reindex`, `apply_ufunc`) | data prep, or a declared `escape:` island — the closed AST is what makes streaming possible | diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 350e99a09..559e063a7 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -494,7 +494,7 @@ Wall time behaves throughout: our advantage grows as the model thins, 1.0x to What each sink can ingest, measured against the shipped solvers rather than assumed. The architectural reading is in [docs/design/ceiling.md](design/ceiling.md#capability-is-not-the-ceiling); the plan is -[ROADMAP Track 3](ROADMAP.md#track-3--capabilities-and-the-degree-line). +[Track 3](https://github.com/FBumann/lpspec/issues/472). | | `lp_file` | HiGHS direct | Gurobi direct | |---|---|---|---| diff --git a/docs/design/ceiling.md b/docs/design/ceiling.md index bfcac74d7..3dac605fa 100644 --- a/docs/design/ceiling.md +++ b/docs/design/ceiling.md @@ -2,10 +2,11 @@ **What may enter the language, and what may never.** The structure that holds it is [ARCHITECTURE](../ARCHITECTURE.md); the rules a model must obey are -[SPEC §0](../SPEC.md#0-the-laws); what is planned or refused is -[ROADMAP](../ROADMAP.md). This page is the argument in between: the test a +[SPEC §0](../SPEC.md#0-the-laws); why the project exists and where it is going +is [ROADMAP](../ROADMAP.md). This page is the argument in between: the test a candidate primitive has to pass, why capability is a second axis rather than -part of the ceiling, and what composition would force. +part of the ceiling, [what has been refused and why](#deliberate-non-primitives), +and what composition would force. It is a claim, so it carries evidence — math a ported model needed and this language could not state becomes a ledger row in @@ -40,7 +41,7 @@ made for a while. Nothing about `variable × variable` is non-relational or non-local — a coordinate-aligned product is a pointwise self-join. Degree 1 is where the language *is*, not where it must stay: what actually gates quadratic is **what a sink can ingest**, which is the second axis below, and the sequence is -[ROADMAP Track 3](../ROADMAP.md#track-3--capabilities-and-the-degree-line). The +[Track 3](https://github.com/FBumann/lpspec/issues/472). The same is true of SOS, indicator and semi-continuous. Read this page as the *streamability* closure and nothing more. @@ -103,7 +104,7 @@ generation — because there is no "before" for it to happen in. **Outside the plan is not outside the engine**, and the difference is the whole of decomposition. A plan cannot contain a loop; a *process* may loop over plans, each with its shape fixed before its own data. Rolling horizon is that shape and -is in scope ([ROADMAP Track 2](../ROADMAP.md#track-2--the-operational-surface)); so are +is in scope ([Track 2](https://github.com/FBumann/lpspec/issues/471)); so are Benders and successive substitution. Nor does appending a cut cost the label contract the way removal would: `var_label` is a `ROW_NUMBER()` over the rows surviving the `where` mask, so adding *rows* moves no column and renumbers no @@ -141,7 +142,29 @@ so capability is not a flat set. The whole-Hessian handoff is an implementation difference, not a rule-4 violation. Making this a declared per-sink capability set, with `check` taking an optional -sink, is [ROADMAP Track 3](../ROADMAP.md#track-3--capabilities-and-the-degree-line). +sink, is [Track 3](https://github.com/FBumann/lpspec/issues/472). + +## Deliberate non-primitives + +The admissibility test above says what *may* enter. This is what has been asked +for and refused, with the reason and the rewrite — so a request that has already +been answered is answered once rather than re-argued. Parity with another tool +is not by itself a reason to add anything. + +| Request | Why | Instead | +|---|---|---| +| Data prep — resampling, clustering, IO, units | not math | preprocess; pass a parameter | +| Arbitrary array ops (`merge`, `reindex`) | unbounded; xarray with extra steps | data prep | +| Domain helpers (`reduce_carrier_dim`) | encodes one domain into the language | component libraries over generic primitives | +| Normalisation (`x / sum(x)`) | a *variable divisor* is rational, not polynomial — no sink takes it at any degree | state the ratio as a constraint, or fix the denominator | +| Conditionals, iteration, data-dependent structure **inside one plan** | destroys the closed AST | `where` masks + `foreach` dims. A *process* may loop over plans | +| A Python API for constructing models | hard rule 5 — the model is the file you review and diff | YAML. Whether Python may *emit* declarations is [#381](https://github.com/FBumann/lpspec/issues/381) | + +Genuinely unsayable math goes to a declared `escape:` island +([#38](https://github.com/FBumann/lpspec/issues/38)) — named in the file, +bounded by the preceding `where`, terminal, and billed against a label budget +before any Python runs. It buys back *relational* and *local*; it cannot buy +back degree, since it returns affine COO rows either way. ## Composition (component libraries) diff --git a/docs/models/monthly_budget.md b/docs/models/monthly_budget.md index 783de6a5f..7458a23d4 100644 --- a/docs/models/monthly_budget.md +++ b/docs/models/monthly_budget.md @@ -211,7 +211,7 @@ and a group with no members contributes nothing. What it cannot express is an **overlapping** aggregate — *"trailing twelve months, at every month"* — because each snapshot would belong to twelve groups and no single column can say so. That is a sliding window over a variable, and -it is [ROADMAP](../ROADMAP.md#track-1--primitives) Track 1 item 4. +it is the fixed-width window, [#468](https://github.com/FBumann/lpspec/issues/468). The same split shows up one level up, where a *process* loops over plans rather than an expression looping over rows diff --git a/examples/walkthrough.out b/examples/walkthrough.out index 3d50eb646..77797df7f 100644 --- a/examples/walkthrough.out +++ b/examples/walkthrough.out @@ -90,6 +90,6 @@ model: examples/walkthrough.yaml variable x variable — above the degree-1 ceiling: caught by check() as LanguageError - objective 'total_cost': both factors of a product contain variables, which is degree 2. Multiply the variable by a parameter instead, or model the curve with a piecewise: block — see ROADMAP, "The degree axis". + objective 'total_cost': both factors of a product contain variables, which is degree 2. Multiply the variable by a parameter instead, or model the curve with a piecewise: block — see docs/design/ceiling.md. docs/ARCHITECTURE.md has the rules these stages enforce. diff --git a/src/lpspec/language/degree.py b/src/lpspec/language/degree.py index 18edbbfd2..b48f10ecd 100644 --- a/src/lpspec/language/degree.py +++ b/src/lpspec/language/degree.py @@ -4,7 +4,7 @@ the resolved core AST with no data bound, which is what makes ``lps.check()`` a real gate rather than a syntax pass, and it is the one admissibility rule that is a **scope choice** rather than a consequence of streaming -(docs/design/ceiling.md, ROADMAP "The degree axis"). +(docs/design/ceiling.md). That is why it lives here and not in ``lowering.py``. Degree is a property of the *language*, not of any plan — the ceiling doc says so in as many words — @@ -97,14 +97,14 @@ def check_binary(node: BinaryOperatorNode, context: str | None = None) -> None: raise LanguageError( f"{where}operator '{node.op}' is not in the language. Multiply the " f'term out, or precompute it as a parameter — a variable base would ' - f'make the model nonlinear (see ROADMAP, "The degree axis").' + f'make the model nonlinear (see docs/design/ceiling.md).' ) if node.op == '*' and carries_variable(node.left) and carries_variable(node.right): raise LanguageError( f'{where}both factors of a product contain variables, which ' f'is degree 2. Multiply the variable by a parameter instead, or ' - f'model the curve with a piecewise: block — see ROADMAP, ' - f'"The degree axis".' + f'model the curve with a piecewise: block — see ' + f'docs/design/ceiling.md.' ) if node.op == '/' and carries_variable(node.right): raise LanguageError( diff --git a/src/lpspec/language/validation.py b/src/lpspec/language/validation.py index daec0069c..2bde0200e 100644 --- a/src/lpspec/language/validation.py +++ b/src/lpspec/language/validation.py @@ -313,7 +313,7 @@ def _check_template_names( errors.append(f'{context}: {unknown_helper_message(node.name)}') for arg in node.args: _check_template_names(arg, template, context, ns, formals, errors) - # a formal may be bound to a dimension at the call site (ROADMAP 5c) + # a formal may be bound to a dimension at the call site known_dims = ns.dimensions | formals for kwarg in builtin.dimension_kwargs if builtin else (): value = node.kwargs.get(kwarg) diff --git a/src/lpspec/relational/chunking.py b/src/lpspec/relational/chunking.py index b0f1a616b..0697dfa34 100644 --- a/src/lpspec/relational/chunking.py +++ b/src/lpspec/relational/chunking.py @@ -13,7 +13,8 @@ model and a hundred in another, so what the pass held tracked the model's shape rather than the budget — the one thing a *batched* pass exists to stop doing. (The engine's own peak tracks the model, deliberately: there is no -configured ceiling any more — ROADMAP Track 5. This is the narrower promise +configured ceiling any more, and declaring one is the memory axis in +docs/ROADMAP.md. This is the narrower promise that a pass over a model holds a bounded slice of it, not the whole.) Requiring a width at every call site is the point of this module. A pass whose unit really does cost one element says so, in one character, where a diff --git a/src/lpspec/relational/sinks/README.md b/src/lpspec/relational/sinks/README.md index afd91648b..2bf0c1d66 100644 --- a/src/lpspec/relational/sinks/README.md +++ b/src/lpspec/relational/sinks/README.md @@ -53,7 +53,7 @@ no name on the Python surface. ## When Track 3 lands -[Track 3](../../../../docs/ROADMAP.md#track-3--capabilities-and-the-degree-line) +[Track 3](https://github.com/FBumann/lpspec/issues/472) gives each sink a declared capability table so `check(model, sink=...)` can answer "will this sink take it". The table belongs in the sink's own module, collected by the family `__init__` rather than owned by it — and it stops diff --git a/tests/oracle.py b/tests/oracle.py index 5081479f9..57c10a8c7 100644 --- a/tests/oracle.py +++ b/tests/oracle.py @@ -74,7 +74,7 @@ # re-exports the *real* ``linopy`` above, so the shim needs a name that cannot # be confused with it. ``lpspec_linopy`` names the module it actually is, which # also keeps it unambiguous if a second non-streaming execution mode ever -# lands (ROADMAP Track 5) and "eager" stops picking out one thing. +# lands and "eager" stops picking out one thing. from lpspec import linopy as lpspec_linopy # noqa: E402 — must follow the guard above from lpspec.linopy import builder, loader # noqa: E402 diff --git a/tests/test_api.py b/tests/test_api.py index c76ca5c7d..f88651346 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -130,7 +130,7 @@ def test_check_and_load_schema_need_no_data(dispatch_yaml): ('expression', 'match'), [ ('sum(p ** 2, over=generator)', r"operator '\*\*'"), - # the CI verb enforces degree 1 with no data bound (ROADMAP, degree axis) + # the CI verb enforces degree 1 with no data bound (docs/design/ceiling.md) ('sum(p * p, over=generator)', 'degree 2'), ], ) @@ -251,7 +251,7 @@ def test_a_second_solve_does_not_rewrite_the_first_result(dispatch_yaml, dispatc so `objective` was a snapshot while `primal` was live — one result disagreeing with itself after a second solve, silently and with plausible numbers. Nothing supported re-binds data yet, so the bound has to be moved - the way the planned in-place update will (ROADMAP 2c: `changeColsBounds` + the way the planned in-place update will (#382: `changeColsBounds` against labels that are already solver indices). """ key = ['snapshot', 'generator'] # a read is a join, so compare on coordinates diff --git a/tests/test_resolution_parity.py b/tests/test_resolution_parity.py index 2a0c52f3b..1246cadbb 100644 --- a/tests/test_resolution_parity.py +++ b/tests/test_resolution_parity.py @@ -84,7 +84,7 @@ def test_both_lanes_refuse_the_same_where(tmp_path, data, coords, where, match, #: sweep masks ``variables.p.where`` on the dispatch model, and a bare variable #: name fits neither slot: in a variable's own where it is a self-reference #: (rejected at load), and on ``balance`` it spans a dim the constraint does not -#: (a DimensionError, correctly — reducing it needs ROADMAP Track 1 item 6). +#: (a DimensionError, correctly — reducing it needs an `all`-reduction, #469). #: Mapped rather than skipped so the coverage guard below still names a test. COVERED_ELSEWHERE = { 'VariableDefinedNode': ('tests/test_relational.py::test_a_bare_variable_name_in_a_where_asks_whether_it_exists'), diff --git a/tests/test_validation.py b/tests/test_validation.py index ea36cc636..7f344b153 100644 --- a/tests/test_validation.py +++ b/tests/test_validation.py @@ -212,7 +212,7 @@ def test_extend_flags_unknown_variable(self, tmp_path): class TestDimensionKwargs: - """ROADMAP 5c: a dim kwarg that names nothing is a silent no-op, not an error. + """A dim kwarg that names nothing is a silent no-op, not an error. ``sum(p, over=snapshto)`` used to build a model that solved and was wrong — both lanes agree on the no-op, so nothing downstream caught it.