Skip to content

Commit c93a8a8

Browse files
authored
perf(beacon): write epoch balances through an in-order copy-on-write cursor (#647)
## Motivation An epoch boundary writes every balance. `get_mut` on a tree list buffers a copy of each element in a dense map, sorts the map into a second `Vec`, then rebuilds every touched leaf and recomputes the root of every written element, changed or not. The altair rewards step calls it up to eight times per validator (four reward/penalty pairs, each as increase then decrease), and `process_effective_balance_updates` reads `balances()[i]` per validator, which was only cheap because the rewards pass had just filled the map. This PR adds an in-order lazy copy-on-write write cursor to `ethlambda-ssz-tree` and moves the epoch balance writes onto it. Expected gain (estimate, not measured on a real chain): tens of ms off each epoch-start block at mainnet registry size. The import-replay benchmark was not run for this PR. ## Changes | commit | what | |---|---| | `perf(ssz-tree): add an in-order copy-on-write write cursor` | `List`/`Vector::iter_cow`, `IterCow::next_cow`, `ElemCow::{index, make_mut, set}`, `try_update_each`; `with_updated_leaves` generalized over a leaf source; unit tests and model-based property tests | | `perf(beacon): apply epoch balance and effective-balance writes through the cursor` | `BeaconState::registry_mut` (disjoint borrows of validators and balances); `apply_balance_deltas` used by the altair+ and phase0 `process_rewards_and_penalties`; both `process_effective_balance_updates`; new `tree_bench` rows; docs | How the cursor works: it walks leaves in order and copies a leaf on the first write that reaches it. At the leaf's end it compares only the touched offsets with the original; if all are equal, the original leaf (its `Arc` and cached hash) is kept. Otherwise the new leaf carries over the element roots of unchanged elements and computes the changed ones. On drop, one descent swaps the changed leaves in and rebuilds only their paths. Pending buffered writes are applied before the pass starts. ## Design decisions - `Drop` does the work, and the tree is untouched during the pass, so an early return or a panic leaves a consistent list holding the writes so far (tested with `catch_unwind`). - Names as in the design: `iter_cow`, `next_cow`, `make_mut`, `set`, `try_update_each`. No `set_with`. - Changed composite elements get their root computed eagerly, on the state-transition thread. - A kept `Arc` or carried root always follows an actual comparison. The cursor only creates fresh empty hash cells, or carried roots for a leaf it owns; it never fills a shared `OnceLock`. - Spec order for balances: per index, each pair is `saturating_add(reward)` then `saturating_sub(penalty)`, pairs in order. Deltas are never netted. Each balance depends only on its own deltas, so index-by-index equals the spec's pair-by-pair loop. - `apply_balance_deltas` lives in `helpers/mutators.rs` and is shared by the phase0 and altair+ paths. It keeps the old error for a balances list shorter than the registry, and does not touch balances past the registry length. - Rewards and both effective-balance passes move together: once rewards bypass the map, the effective-balance reads would otherwise become tree descents. - Effective-balance updates only copy a leaf when the new effective balance differs from the old one (it can equal it when capped at the maximum). - No leaf-slice API, no parallel bulk update. ## Not in this PR - Switching `Balances` to a sparse `BTreeMap` update map: to be decided after measuring whether a dense writer remains. - `List::filled`: its only consumer is the participation-list work, which is deferred. - Migrating slashings, inactivity, and registry updates: sparse, flat lists, or part of the epoch single-pass rewrite (#630, #632, #633). Expect a merge conflict with #632 in `process_rewards_and_penalties`. - Import-replay benchmark and any remote profiling. ## Testing | command | result | |---|---| | `cargo test -p ethlambda-ssz-tree --profile release-fast` | 71 unit + 31 model tests passed (before the second commit) | | `cargo test -p ethlambda-types --profile release-fast --lib` | 197 passed | | `cargo test -p ethlambda-state-transition --profile release-fast --lib` | 324 passed, 16 ignored | | beacon spec suite, mainnet preset, full | 5734 passed, 4 failed | | beacon spec suite, minimal preset, full | 40038 passed, 4 failed | | `cargo clippy --locked --workspace --all-targets --profile release-fast -- -D warnings` and the state-transition `beacon-spec-tests` variant | clean | | `cargo check -p ethlambda --profile release-fast` | ok | The 4 spec failures in each preset are the `gossip/*` suites (`gossip_beacon_block`, `gossip_data_column_sidecar`, `gossip_beacon_aggregate_and_proof`, `gossip_beacon_attestation`): the local gossip fixture directory is intentionally empty, so they match no cases. Every other suite, including `epoch_processing`, `rewards`, `sanity`, `finality`, `random` and `transition`, passes. ### tree_bench (2.4M validators, `--ignored`, single run) Noisy: run while the machine was under a load average above 100 from other builds, so absolute numbers (and even the order of the old rows) are unreliable; read them as rough ratios only. | row | ms | |---|---| | `epoch_sweep_tree` (old path) | 333.8 | | `epoch_sweep_tree_phases` | write 58.1, apply 70.0, hash 83.1 | | `epoch_sweep_cursor_tree` | 26.8 | | `epoch_sweep_noop_cursor_tree` | 90.4 | | `epoch_sweep_rebuild_tree` (rebuild baseline) | 105.8 | | `rewards_apply_4_pairs_get_mut` | 342.5 | | `rewards_apply_4_pairs_cursor` | 158.9 | | `eb_updates_indexed` | 253.8 | | `eb_updates_cursor` | 115.4 | | `lockstep_three_lists_cursor` | 136.2 | The bench asserts the cursor rows produce the same roots as the old paths, and that a no-op sweep keeps the tree (`ptr_eq`).
1 parent b30257c commit c93a8a8

17 files changed

Lines changed: 1400 additions & 78 deletions

File tree

‎CLAUDE.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -647,7 +647,10 @@ existing once per chain.
647647
- **`Validators` and `Balances` are `ethlambda_ssz_tree::List`s** (gloas:
648648
`ethlambda_ssz_tree::ProgressiveList`, same method set), persistent
649649
Merkle trees that cache node hashes and share unchanged subtrees between
650-
states through `Arc`; they have no slices and no `iter_mut`. The registry is
650+
states through `Arc`; they have no slices and no `iter_mut` (a pass that
651+
rewrites most of a list uses the lazy copy-on-write cursor, `iter_cow` /
652+
`try_update_each`, with `BeaconState::registry_mut` for disjoint borrows of
653+
both lists). The registry is
651654
therefore reached through element-level `BeaconState` accessors
652655
(`validator`/`validator_mut`, `balance`/`balance_mut`, `push_validator`,
653656
`iter_validators`/`iter_balances`, `validator_count`, `validators_root`), so

‎crates/blockchain/state_transition/src/beacon/helpers/mutators.rs‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,47 @@ pub fn decrease_balance(state: &mut BeaconState, index: ValidatorIndex, delta: G
3333
Ok(())
3434
}
3535

36+
/// Applies reward and penalty vectors to every balance in one in-order pass.
37+
///
38+
/// Equivalent to calling [`increase_balance`] then [`decrease_balance`] for
39+
/// every index, one `(rewards, penalties)` pair after another: each balance
40+
/// depends on its own deltas only, so going index by index and pair by pair
41+
/// within an index gives the same result as the specification's pair-by-pair
42+
/// loop. The saturation stays per step, in spec order (`saturating_add` of the
43+
/// reward, then `saturating_sub` of the penalty, for each pair): netting the
44+
/// deltas first would let a reward mask a penalty that should have driven a low
45+
/// balance to zero.
46+
///
47+
/// Goes through the balances' write cursor instead of a `get_mut` per call, so
48+
/// nothing is buffered per element and a leaf whose balances all come out
49+
/// unchanged keeps its hash. Every vector is indexed by validator index and
50+
/// must cover the registry.
51+
pub fn apply_balance_deltas(
52+
state: &mut BeaconState,
53+
deltas: &[(Vec<Gwei>, Vec<Gwei>)],
54+
) -> Result<()> {
55+
let validator_count = state.validator_count();
56+
let balance_count = state.iter_balances().len();
57+
if balance_count < validator_count {
58+
return Err(Error::UnknownValidator(balance_count as ValidatorIndex));
59+
}
60+
state.try_update_balances(|balance| {
61+
let index = balance.index();
62+
// Balances past the registry are not the specification's to touch.
63+
if index >= validator_count {
64+
return Ok(());
65+
}
66+
let mut value = **balance;
67+
for (rewards, penalties) in deltas {
68+
value = value
69+
.saturating_add(rewards[index])
70+
.saturating_sub(penalties[index]);
71+
}
72+
balance.set(value);
73+
Ok(())
74+
})
75+
}
76+
3677
/// Puts a validator into the exit queue.
3778
///
3879
/// Does nothing if it is already exiting, so this is safe to call more than once

‎crates/blockchain/state_transition/src/beacon/stf/epoch/altair.rs‎

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -194,11 +194,11 @@ pub(super) fn next_inactivity_score(
194194
/// [`constants::PARTICIPATION_FLAG_WEIGHTS`]) plus the same kind of inactivity
195195
/// penalty, computed from [`crate::beacon::helpers::altair::get_inactivity_penalty_deltas`]
196196
/// against the scores [`process_inactivity_updates`] just brought up to date.
197-
/// Applying rewards and penalties as two separate passes (through
198-
/// [`increase_balance`] and [`decrease_balance`], not one netted delta) is
199-
/// unchanged from phase0, and for the same reason: [`decrease_balance`] floors
200-
/// at zero, so netting first would let a reward mask a penalty that should
201-
/// have driven a low balance all the way down.
197+
/// Rewards and penalties are applied as separate saturating steps, in spec
198+
/// order (see [`crate::beacon::helpers::mutators::apply_balance_deltas`]), not as one netted delta: that is
199+
/// unchanged from phase0, and for the same reason: a penalty floors at zero,
200+
/// so netting first would let a reward mask a penalty that should have driven
201+
/// a low balance all the way down.
202202
///
203203
/// Skipped entirely at the genesis epoch: rewards pay for participation
204204
/// recorded during the previous epoch, and genesis has none.
@@ -265,10 +265,18 @@ fn apply_rewards_and_penalties(
265265

266266
// The iterator is opaque, so its borrow of `state` lasts until it drops.
267267
drop(balance_iter);
268-
for (index, updated) in changes {
269-
*state.balance_mut(index as ValidatorIndex)? = updated;
270-
}
271-
Ok(())
268+
// The changes are in index order, so one cursor pass applies them: nothing
269+
// is buffered per change, and the leaves it does not touch keep their hash.
270+
let mut changes = changes.into_iter().peekable();
271+
state.try_update_balances(|balance| {
272+
if let Some(&(index, updated)) = changes.peek()
273+
&& index == balance.index()
274+
{
275+
balance.set(updated);
276+
changes.next();
277+
}
278+
Ok::<(), Error>(())
279+
})
272280
}
273281

274282
/// Steps 1-3 of every altair-through-fulu epoch, over one registry scan.

‎crates/blockchain/state_transition/src/beacon/stf/epoch/electra.rs‎

Lines changed: 14 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -763,37 +763,28 @@ pub fn process_pending_consolidations(state: &mut BeaconState, _config: &Config)
763763
/// hysteresis test itself is [`super::leaves_hysteresis_band`], shared with
764764
/// that version.
765765
pub fn process_effective_balance_updates(state: &mut BeaconState) -> Result<()> {
766-
// Two passes for the same reason `super::process_effective_balance_updates`
767-
// needs them: `validator_mut` clones the element into the update buffer
768-
// on every call, whether or not it is then written (see its own doc), so
769-
// deciding and writing in one combined pass would buffer and rehash the
770-
// whole registry instead of only the validators that actually move; that
771-
// also keeps `get_max_effective_balance`, called here on the validator
772-
// being decided on, reading rather than fighting a live mutable borrow.
773766
// The specification reads `state.balances[index]` for every validator, so
774767
// a balance list shorter than the registry fails at its first missing
775-
// entry. Checked here because the zip below would stop at it silently.
768+
// entry. Checked here because the paired pass below would stop at it
769+
// silently.
776770
let balance_count = state.iter_balances().len();
777771
if balance_count < state.validator_count() {
778772
return Err(Error::UnknownValidator(balance_count as ValidatorIndex));
779773
}
780-
let mut updates = Vec::new();
781-
// Zipped rather than indexed: step 3 only writes the balances that changed,
782-
// so `balance(index)` would be a tree descent for most validators.
783-
for (index, (validator, balance)) in state
784-
.iter_validators()
785-
.zip(state.iter_balances())
786-
.enumerate()
787-
{
774+
// Same shape as `super::process_effective_balance_updates`: the registry's
775+
// write cursor, with the balances read in step.
776+
// `get_max_effective_balance` reads the validator being decided on straight
777+
// from the cursor's element.
778+
state.try_update_validators_with_balances(|validator, balance| {
788779
if let Some(effective) = updated_effective_balance(validator, balance)? {
789-
updates.push((index as ValidatorIndex, effective));
780+
// Equal to the old value when capped at the ceiling: checked before
781+
// the copy, so a validator already there dirties nothing.
782+
if effective != validator.effective_balance {
783+
validator.make_mut().effective_balance = effective;
784+
}
790785
}
791-
}
792-
793-
for (index, effective) in updates {
794-
state.validator_mut(index)?.effective_balance = effective;
795-
}
796-
Ok(())
786+
Ok(())
787+
})
797788
}
798789

799790
/// The effective balance `validator` moves to given its `balance`, or `None`

‎crates/blockchain/state_transition/src/beacon/stf/epoch/mod.rs‎

Lines changed: 15 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -291,39 +291,31 @@ pub fn process_eth1_data_reset(state: &mut BeaconState) -> Result<()> {
291291
/// and since effective balance feeds the shuffling seed's weighting and every
292292
/// reward, that would churn far more than it measures.
293293
pub fn process_effective_balance_updates(state: &mut BeaconState) -> Result<()> {
294-
// Decided in one pass and applied in another: `validator_mut` clones the
295-
// element into the update buffer on every call, whether or not it is
296-
// then written (see its own doc), so deciding and writing in one combined
297-
// pass would buffer and rehash the whole registry instead of only the
298-
// validators that actually move. Collecting the decisions first also
299-
// keeps this fork-independent, which matters because every fork runs
300-
// this step unchanged.
301294
// The specification reads `state.balances[index]` for every validator, so
302295
// a balance list shorter than the registry fails at its first missing
303-
// entry. Checked here because the zip below would stop at it silently.
296+
// entry. Checked here because the paired pass below would stop at it
297+
// silently.
304298
let balance_count = state.iter_balances().len();
305299
if balance_count < state.validator_count() {
306300
return Err(Error::UnknownValidator(balance_count as ValidatorIndex));
307301
}
308-
let mut updates = Vec::new();
309-
// Zipped rather than indexed: step 3 only writes the balances that changed,
310-
// so `balance(index)` would be a tree descent for most validators.
311-
for (index, (validator, balance)) in state
312-
.iter_validators()
313-
.zip(state.iter_balances())
314-
.enumerate()
315-
{
302+
// One in-order pass through the registry's write cursor, with the balances
303+
// read in step. It copies a leaf only for a validator whose effective
304+
// balance changes and keeps every other leaf (and its hash) as it was;
305+
// this stays fork-independent, which matters because every fork runs this
306+
// step unchanged.
307+
state.try_update_validators_with_balances(|validator, balance| {
316308
if leaves_hysteresis_band(validator.effective_balance, balance)? {
317309
let effective = (balance - balance % preset::EFFECTIVE_BALANCE_INCREMENT)
318310
.min(preset::MAX_EFFECTIVE_BALANCE);
319-
updates.push((index as ValidatorIndex, effective));
311+
// Equal to the old value when capped at the maximum: checked before
312+
// the copy, so a validator already at its ceiling dirties nothing.
313+
if effective != validator.effective_balance {
314+
validator.make_mut().effective_balance = effective;
315+
}
320316
}
321-
}
322-
323-
for (index, effective) in updates {
324-
state.validator_mut(index)?.effective_balance = effective;
325-
}
326-
Ok(())
317+
Ok(())
318+
})
327319
}
328320

329321
/// Whether `balance` has moved far enough from `effective_balance` for the

‎crates/blockchain/state_transition/src/beacon/stf/epoch/rewards.rs‎

Lines changed: 8 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@
1212
//! a fixed amount per validator regardless of how many there are.
1313
//!
1414
//! [`process_rewards_and_penalties`] applies the results as two separate
15-
//! passes, rewards then penalties, each through [`increase_balance`] and
16-
//! [`decrease_balance`] rather than a single netted delta. That distinction is
17-
//! load-bearing: [`decrease_balance`] floors at zero, and netting the two
15+
//! saturating steps, rewards then penalties (see [`apply_balance_deltas`]),
16+
//! rather than a single netted delta. That distinction is
17+
//! load-bearing: a penalty floors at zero, and netting the two
1818
//! before applying them would let a reward mask a penalty that should have
1919
//! driven a low balance all the way down.
2020
//!
@@ -48,7 +48,7 @@ use crate::beacon::helpers::finality::{
4848
get_eligible_validator_indices, get_finality_delay, is_in_inactivity_leak,
4949
};
5050
use crate::beacon::helpers::math::integer_squareroot;
51-
use crate::beacon::helpers::mutators::{decrease_balance, increase_balance};
51+
use crate::beacon::helpers::mutators::apply_balance_deltas;
5252
use crate::beacon::preset;
5353
use crate::beacon::primitives::{Gwei, ValidatorIndex};
5454

@@ -308,21 +308,17 @@ pub fn get_attestation_deltas(
308308
///
309309
/// Skipped entirely at the genesis epoch: rewards pay for attestations cast in
310310
/// the previous epoch, and genesis has none. Rewards and penalties are two
311-
/// separate passes over [`increase_balance`] and [`decrease_balance`], not one
312-
/// netted delta, because [`decrease_balance`] floors at zero: netting first
311+
/// separate saturating steps (see [`apply_balance_deltas`]), not one
312+
/// netted delta, because a penalty floors at zero: netting first
313313
/// would let a reward mask a penalty that should have driven a low balance all
314314
/// the way down.
315315
pub fn process_rewards_and_penalties(state: &mut BeaconState, config: &Config) -> Result<()> {
316316
if get_current_epoch(state) == constants::GENESIS_EPOCH {
317317
return Ok(());
318318
}
319319

320-
let (rewards, penalties) = get_attestation_deltas(state, config)?;
321-
for index in 0..state.validator_count() as ValidatorIndex {
322-
increase_balance(state, index, rewards[index as usize])?;
323-
decrease_balance(state, index, penalties[index as usize])?;
324-
}
325-
Ok(())
320+
let deltas = [get_attestation_deltas(state, config)?];
321+
apply_balance_deltas(state, &deltas)
326322
}
327323

328324
#[cfg(test)]

0 commit comments

Comments
 (0)