Skip to content

Prove table soundness from a finishing order of channels and of message classes - #6

Draft
jotabulacios wants to merge 14 commits into
pr1-upstream-v2from
pr2-upstream-v2
Draft

jotabulacios wants to merge 14 commits into
pr1-upstream-v2from
pr2-upstream-v2

Conversation

@jotabulacios

Copy link
Copy Markdown
Collaborator

Stacked on #5: the base of this PR is pr1-upstream-v2, so the diff below is this PR's own work —
one new file in Clean/Air/, one worked example, 40 lines of Balance.lean, two imports and a
README paragraph.

Why

OrderedChannel orders tables: on each channel, every table that pushes comes before every
table that pulls, and no table does both. Two shapes break that.

The first has no workaround. A table that both pulls from a channel and pushes to it, on different
messages, is not a cycle: what it reads and what it writes are disjoint. A bus shared by several
operations is where this happens — a table that answers one operation by using the answers to
others pulls those and pushes its own, all under one channel name. Ordering whole channels cannot
express that, and giving each operation a channel of its own costs a channel per operation.

The second is a table whose requirements on one channel follow from its guarantees on another.
Clean/Examples/CountedSquares, from #1, is the case in miniature: its consumer pulls a square
with multiplicity count and owes count = 0 ∨ CountUpTo 2 count, which it pays with a constraint
of its own, (count - 1) * (count - 2) = 0. That only scales while the bound is small enough to
write as a polynomial. A chip counting to 255 range-checks its multiplicity on a second channel
instead, and then no order of its tables says where the bound came from.

This PR orders stages: a channel together with a class of its messages. finished lists them
newest first, and a component's requirements on one may use its guarantees on those finished before
it. By induction from the oldest, every one's requirements hold, so by consistency its guarantees
hold, and every table's spec follows. Balance is message by message (BalancedUpTo.filter_msg), so
a stage balances by itself.

What

The stage form is the argument; the channel form is read off it.

  • Stage, a channel together with a class of its messages, with Table.GuaranteesOn /
    RequirementsOn and Component.RequirementsAfterOn / StagedRequirementsOn. The soundness is
    Ensemble.guaranteesOn_of_stagedRequirementsOn and
    Ensemble.tableSoundness_of_stagedRequirementsOn.
  • A whole channel is the stage whose class is every message, so the channel form is the stage form
    through that correspondence rather than a second induction: Stage.ofChannel and
    Component.stagedRequirementsOn_of_stagedRequirements carry it over, and
    Ensemble.tableSoundness_of_stagedRequirements is seven lines on top. The statement is kept
    because an ensemble whose channels finish whole is the simpler thing to say, and to use.
  • Discharging helpers at both levels: requirementsAfter_of_not_mem /
    requirementsAfter_of_guarantees_subset / stagedRequirements_of_forall, and their stage
    counterparts, plus requirementsAfterOn_of_rowOperations, which reads a stage's obligation one
    row and one interaction at a time — the form a component's own circuit_norm lemmas produce —
    and stagedRequirementsOn_of_no_guarantees for a component that receives no guarantee at all.
  • BalancedUpTo.filter_msg in Balance.lean: balance holds message by message, so it restricts to
    any class of messages.
  • Clean/Examples/StagedSquares: the second shape above, built. It reuses the squaring chip and
    counted channel of CountedSquares and changes one thing — the consumer pulls its count from a
    second channel instead of constraining it, so its requirement on squares follows from its
    guarantee on counts, and counts finished before squares is what says so. It generates, and
    its WitnessGenerationTest pins the run.
  • Clean/Examples/SameBus: the first shape, built. One bus carries (tag, a, b) and guarantees
    b = a * a at tag = 0, b = a⁴ at tag = 1. A fourth-power chip pulls a square and pushes a
    fourth power on that one bus; its push owes b = a⁴ and one constraint of its own is not enough,
    because z = y * y says nothing about x until the bus says y = x * x. Ordering whole
    channels cannot express it — powers would have to come before itself — and the two classes can.
    This is what exercises Stage, tableSoundness_of_stagedRequirementsOn and the
    requirementsAfterOn_* helpers.
  • Clean/Air/README.md gains a paragraph for the file, since its Organization section says the
    library provides two arguments for ensemble soundness and this is a third.

Both staged theorems take their per-channel balance from
EnsembleWitness.balancedChannels_of_balanced, the same named step that OrderedChannel.lean and
Vm.lean use, rather than inlining the derivation over the ensemble's channels.

Scope

Two limits, both deliberate.

The verifier must be empty. Ensemble.verifier defaults to the empty program, so for an
ensemble of tables the hypothesis is discharged by rfl; it is not free for a VM, where
Ensemble.addVm propagates a verifier that pushes the initial state and pulls the final one.
Neither staged predicate says what a verifier owes, so the hypothesis is what lets the proofs use
interactionsWith_of_verifier_empty. It stands for two obligations: that every interaction on a
stage come from a table, so the requirements the induction establishes cover the whole list
consistency is applied to, and that the verifier's own spec follow from its guarantees.
OrderedChannel.lean discharges the first through PartialBalancedChannel, which tolerates
interactions from outside the list it is stated for, and the second through
verifierChannelGuarantees_of_tableRequirements.

The induction itself needs less than it asks for: only that the verifier not interact with the
stages being finished, which is per channel rather than global. The hypothesis is left as it is
because nothing in the tree calls the induction directly, and weakening it without a caller would
add generality nobody uses.

It does not compose with SoundEnsemble. The argument is made for a whole ensemble at once
rather than built up table by table, so StagedSquares assembles its Ensemble directly. Relating
the two is a separate piece of work.

Checks

…ge classes.

A table's requirements on a channel may follow from its guarantees on channels finished before it,
not only from its constraints. A counted pull needs this: its multiplicity bound comes from a
lookup on another channel, which OrderedChannel can't express. Component.StagedRequirements states
it for a finishing order, and Ensemble.tableSoundness_of_staged proves soundness by induction from
the oldest channel.

A stage is a channel together with a class of its messages. A table that pulls some messages of a
channel and pushes others, like a division chip pulling LT and MUL results and pushing DIVREM
results on the ALU bus, is not a cycle when the classes are finished in order.
Ensemble.tableSoundness_of_stagedOn proves soundness from a finishing order of stages; balance is
message by message, so each stage balances by itself (BalancedUpTo.filter_msg). Genuine cycles
still have no valid order.
…ma too.

Both staged soundness theorems inlined the same derivation over the ensemble's channels
that `balancedChannels_of_balanced` names, which is the pattern just removed from
`OrderedChannel.lean` and `Vm.lean`. All four call sites now go through the lemma, and
`Balance.balancedUpTo_of_balanced` is applied in one place.
…eady does.

`Component.StagedRequirements` has `@[circuit_norm]` nil and cons lemmas, so a caller
discharging `∀ component ∈ ens.tables, component.StagedRequirements finished` with
`simp [circuit_norm]` gets the list unfolded. `StagedRequirementsOn` had neither, so the
identical proof over stages left a stuck goal.
The file held two inductions of the same shape, `channelGuarantees_of_staged` over a list of
channels and `guaranteesOn_of_stagedOn` over a list of stages, forty-nine lines each, with a
soundness corollary apiece. A whole channel is the stage whose class is every message, which the
section already said in prose, so the channel pair is the stage pair read through that
correspondence and does not need an induction of its own.

`Stage.ofChannel` names the stage that is a whole channel, and
`Component.stagedRequirementsOn_of_stagedRequirements` carries a finishing order of channels to
the finishing order of those stages, one step at a time through
`requirementsAfterOn_of_requirementsAfter`, which the file already had. On top of those,
`channelGuarantees_of_staged` is ten lines and `tableSoundness_of_staged` is seven.

Both statements are unchanged, so callers do not notice. What the stage theorem asks for and the
channel one does not is `covered`, that every pull of every table lie in some finished stage;
`subset_finished` discharges it, since the stage of a whole channel accepts every message. The
stage theorems themselves are untouched.
…corollary.

`REVIEWERS.md` asks that each underscore-separated part of a theorem name correspond to
a declaration in its statement rather than to an invented description, and that nothing
inside a namespace be qualified with that namespace.

`staged` and `stagedOn` named no declaration. The hypotheses are
`Component.StagedRequirements` and `Component.StagedRequirementsOn`, so the three
theorems become `guaranteesOn_of_stagedRequirementsOn`,
`tableSoundness_of_stagedRequirementsOn` and `tableSoundness_of_stagedRequirements`,
which is the shape `Ensemble.tableSoundness_of_soundChannels` already has in
`OrderedChannel.lean`.

`Stage.ofChannel_channel` named the projection of a constructor inside out; Mathlib reads
it outside in, so it is `Stage.channel_ofChannel`.

Six references were qualified with the namespace of the declaration they appear in. The
file was inconsistent about it: `stagedRequirementsOn_of_stagedRequirements` already
called itself by its bare name on one line and qualified a sibling on the next. The
`Table.` prefixes inside `Component.*` lemmas stay, since those are a different namespace.

`channelGuarantees_of_staged` had no caller, in this repo or in any consumer: the staged
soundness theorem is reached through `tableSoundness_of_stagedRequirements`, which does
not go through it. It is removed rather than kept as an entry point nobody enters.
…annel orders.

The module docstring named `LT`, `IS_HALF` and `ARE_BYTES`, and the stages section named
`LT`, `MUL` and `DIVREM` on an ALU bus. No circuit in this repo carries those names, so
they ask the reader to take the motivation on faith. The counted pull of
`Clean/Examples/CountedSquares` is the same shape and is in the tree: its multiplicity
owes `CountUpTo N`, and a bound too large to constrain arithmetically has to come from a
range check on another channel. The bus is described by what it is rather than by one
VM's opcodes.

The first sentence also said only half of what `OrderedChannel` requires. Its defining
property is an order between tables, as `Clean/Air/README.md` puts it: every table that
pushes to a channel comes before every table that pulls from it. "A table may not both
pull and owe requirements" is a consequence, not the condition.
Its `covered` hypothesis asked that every possible message of every guarantee channel lie
in some finished stage. The proof passes it to `Table.guarantees_of_guaranteesOn`, which
only ever needs it at the pulls a table actually makes, and reconstructed the weaker form
from the stronger one at the call site.

It now asks for exactly that weaker form, which is the shape
`Component.requirementsAfterOn_of_covered` already uses a few lines above, and the call
site hands it over unchanged. An ensemble whose finished stages cover every message still
discharges it; one whose last stage is not a catch-all now also can.
…yway.

`Component.RequirementsAfterOn` quantifies over the interactions of a whole table, which
is not the form a component's own `circuit_norm` lemmas produce. Every chip proving a
staged obligation therefore starts by turning it into rows times the component's row
operations, through `Table.forall_interactionsWith_iff`, `Component.interactions_eq` and
`AbstractInteraction.eval_requirements`. `requirementsAfterOn_of_rowOperations` is that
step, stated once.

`stagedRequirementsOn_of_no_guarantees` is the other end: a component that receives no
guarantee owes nothing on any stage, so a provider discharges its whole staged obligation
in one application. The channel form already had `stagedRequirements_of_forall` for this
shape; the stage form had nothing.

Both were written against this API outside the library and are stated here over an
arbitrary `FiniteField`, which is what they always needed; neither mentions the
characteristic or the size of the field.
The Organization section documents every file in `Clean/Air/` and says the library
provides two arguments for ensemble soundness. `StagedChannels.lean` is a third, so it
gets a paragraph in the same register: what it orders, what `StagedRequirements` and
`Stage` say, and which theorem concludes.

The paragraph also records the two limits that a reader comparing it with
`OrderedChannel.lean` will want stated. The staged argument is made for a whole ensemble
at once, so it does not compose with the incremental `SoundEnsemble` builder; and it
applies only when the verifier is empty, since neither predicate says what a verifier
owes on a staged channel.

`OrderedChannel.lean` was described as "a staged channel construction", which now names
the wrong file. The sentence after it already states the property that defines it, an
order between tables, so the word is dropped rather than replaced.
The paragraph said only that neither staged predicate says what a verifier owes. The
hypothesis is doing two separate jobs. It makes every interaction on a stage come from a
table, so that the requirements the induction establishes cover the whole list that
`RawChannel.ConsistentUpTo.consistent` is applied to; and it discharges the verifier's own
spec, which is owed even once the first is settled.

`OrderedChannel.lean` handles them with `PartialBalancedChannel`, which tolerates
interactions from outside the list it is stated for, and with
`verifierChannelGuarantees_of_tableRequirements`. Naming both says what lifting the
hypothesis would actually take.
Nothing in the repo used the staged soundness theorems, so the API had no in-tree
consumer: a reader could see what `StagedRequirements` says but not what discharging it
looks like, and the `circuit_norm` lemmas and the two `requirementsAfter_of_*` helpers
were reached by nothing.

`StagedSquares` is the smallest case that needs a finishing order. It reuses the squaring
chip and the counted channel of `Clean/Examples/CountedSquares` and changes one thing:
where that consumer discharges its pull's requirement with a constraint of its own,
`(count - 1) * (count - 2) = 0`, this one pulls its count from a second channel. A chip
provides that channel, and its own constraint is what pays for the guarantee.

The consumer therefore owes a requirement it cannot pay from its constraints. Its
`requirementsChannelsLawful` declares the channel rather than proving the requirement,
and `consumer_requirementsAfter` pays it with the guarantee the consumer was handed on
`counts`. That is the shape `OrderedChannel` cannot express: it orders tables, and no
order of these three says where the bound comes from. `counts` finished before `squares`
does say it.

The ensemble is assembled as a plain `Ensemble` rather than through `SoundEnsemble`,
which the staged argument does not compose with, and generation is not exercised.
`StagedSquares` could not generate. Its range chip emitted `mult + 1`, copied from the
squaring chip of `CountedSquares`, where the `+ 1` is what seeds a run: that chip is
`preallocated`, its rows exist before any demand and their multiplicity cells start at
`0`. The range chip is `demand`, so a row is created only by a pull it has to answer and
is born with that pull counted. The `+ 1` pushed once more than anyone pulled, and
`generate` stopped with `unhandled channel imbalance on: counts`.

The chip now emits the multiplicity as it stands, and the component gets a padding row
that answers nothing: multiplicity `0`, and `c := 1` so that the chip's own constraint
holds, which `0` would not.

`WitnessGenerationTest.lean` pins the run at heights `[1, 4, 2]` with the constraints, the
balance check and the interaction bound all satisfied. The example said only that
generation was not exercised, which was true and beside the point: nothing ran it, so
nothing could say it was broken. Now something does.
…hes one bus.

`StagedSquares` orders whole channels, so half of this file had no consumer: `Stage`,
`GuaranteesOn` / `RequirementsOn`, `RequirementsAfterOn`, the `requirementsAfterOn_of_*`
helpers and `tableSoundness_of_stagedRequirementsOn` could be read but not seen at work.
That half is the argument; the channel form is read off it.

`SameBus` is the smallest case that needs it. One bus carries `(tag, a, b)` and
guarantees `b = a * a` at `tag = 0`, `b = a⁴` at `tag = 1`. A squaring chip provides the
squares. A fourth-power chip pulls a square and pushes a fourth power, on the same bus:
its push owes `b = a⁴`, and one constraint of its own is not enough, because `z = y * y`
says nothing about `x` until the bus says `y = x * x`.

Ordering whole channels cannot express that — `powers` would have to come before itself.
The two classes can: the squares are finished before the fourth powers, and the chip's
only pull lies in the earlier one, which is what `requirementsAfterOn_of_covered` asks
for. `chip_shape` is the step that says so, that every interaction the chip makes on the
bus is its pull at `tag = 0` or its push at `tag = 1` and nothing else.

Interactions are plain pushes and pulls here. Multiplicity is what `CountedSquares` is
about and it is orthogonal to the ordering question.

The ensemble is not generated. Telling the two classes apart during generation needs
demand guards, which nothing in the tree exercises yet; `StagedSquares` is the staged
example that generates, and its test pins the run.
The stage half of this file now has an example, but the shape it was written for is
larger than either: lambda_vm, a RISC-V zkVM built on Clean, whose lookup channels admit
a finishing order with exactly one cycle, an arithmetic bus that one table both reads and
writes. Breaking that cycle with message classes is why `Stage` exists.

Saying so is the answer to "where is this needed?", which
`Clean/Circuit/ExpressionPolynomial` gives in the same way for leanerVM.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant