Skip to content

feat!: use Keccak-256 instead of SHA-256 for hash_tree_root (experiment) - #39

Draft
MegaRedHand wants to merge 6 commits into
mainfrom
feat/keccak-htr
Draft

feat!: use Keccak-256 instead of SHA-256 for hash_tree_root (experiment)#39
MegaRedHand wants to merge 6 commits into
mainfrom
feat/keccak-htr

Conversation

@MegaRedHand

@MegaRedHand MegaRedHand commented Nov 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Warning

Experimental. This makes hash_tree_root not consensus-compatible: every root that involves hashing changes. It is a draft to measure the cost/benefit of Keccak-256 vs SHA-256, not to merge as-is.

Result: SHA-256 is still faster on consensus data, even with SIMD Keccak. On x86_64 (Ryzen 9 9950X3D, SHA-NI, AVX-512), against the batched SHA-256 in perf/batched-sha256-htr:

  • BeaconState: Keccak-256 is 1.16–1.45x slower, and 1.51–1.90x slower than that branch's hashtree feature. Validators and small containers are worse, up to 2.7x and 3.3x.
  • Wide trees (merkleize, big lists and vectors of u64 or Bytes32): Keccak-256 is up to 2x faster, and up to 1.45x faster than hashtree.
  • ARM: the M3 numbers at the end are for the first version of this PR; the current code isn't benchmarked on ARM.

Full results and caveats: results comment.

What changes

This branch is built on top of perf/batched-sha256-htr (merged in), which makes merkleize hash a whole tree level per call through a new hash_pairs method. Until that branch lands, this diff also includes its batching changes. On top of that:

  • Hash function: sha2::Sha256 → Keccak-256 for hash_nodes, merkleize and mix_in_*. This is the original Keccak padding used by Ethereum's execution layer, not NIST SHA3-256.
  • New crates/ssz-merkle/src/keccak256.rs: a 64-byte pair fits in one Keccak block, so each pair costs one Keccak-f[1600] permutation on a pre-padded state, skipping the sponge. hash_pairs picks a backend at runtime with cpufeatures:
    • AVX-512: 8 pairs per permutation, part-filled batches included.
    • AVX2: 4 pairs per permutation; 2 or 3 leftover pairs also run in SIMD, a single one runs scalar.
    • ARMv8 SHA3: 2 pairs per permutation; a leftover pair runs scalar.
    • Otherwise: the scalar permutation from the keccak crate.
    • MerkleHasher::hash, for input of any length, still uses sha3::Keccak256.
  • Dependencies: sha2 is replaced by sha3 0.10, keccak 0.1.6 and cpufeatures 0.2. keccak's asm feature uses the ARMv8 SHA3 instructions through inline assembly, with no C toolchain.
  • ZERO_HASHES: the build-time table in crates/ssz-merkle/build.rs is regenerated with Keccak-256, so zero-subtree elision in merkleize stays consistent with hash_nodes.
  • Breaking renames (the old names would be misleading):
    • trait Sha256Hasher → MerkleHasher (also in the ssz-derive output)
    • struct Sha2Hasher → KeccakHasher
    • feature sha2-backend → keccak-backend
  • Docs: docs/technical-design.md is updated. docs/ssz-spec-summary.md still says SHA-256 because it describes the upstream spec.

Tests

  • New test keccak_hasher_is_legacy_keccak256 pins Keccak256("") = c5d24601…5d85a470, which tells it apart from SHA3-256.
  • Each SIMD backend is checked against sha3::Keccak256 for every batch size from 0 to 41 pairs, and on all-zero pairs against ZERO_HASHES[1]. A backend the CPU lacks is skipped, so I ran the tests on a Zen 5 (AVX-512 and AVX2) and on an Apple M3 (SHA3).
  • Two unit tests that pin SHA-256 reference roots are #[ignore]d with a reason rather than deleted: merkleize_progressive_matches_reference_roots (remerkleable) and merkleize_progressive_matches_consensus_spec_vectors.
  • CI: Build, Test (default and alloc-only), Clippy, Doc, no_std Check and the fuzz smoke runs pass, except:
    • Expected to fail: Spec Tests (consensus vectors are SHA-256) and smoke / diff_htr_container (compares container roots with Lighthouse and ssz_rs). diff_hash_tree_root only compares primitives, which don't hash, so it passes.
    • Fails on main too: Cargo Audit, on crypto-bigint (RUSTSEC-2026-0387).

Results (x86_64)

AMD Ryzen 9 9950X3D (Zen 5, SHA-NI, AVX-512), Linux 6.12, rustc 1.95.0, bench profile, criterion medians, each bench pinned to one core.

  • SHA-256 main: 958a90f.
  • SHA-256 new: perf/batched-sha256-htr @ 72b2889, with and without its hashtree feature.
  • Keccak old: the first version of this PR, 58d16bd.
  • Keccak SIMD: this PR @ bcb6a86. 135e403 after it only changes comments and docs.

Brackets: SHA-256 columns vs SHA-256 main, Keccak SIMD vs Keccak old (higher is faster).

benchmark SHA-256 main SHA-256 new SHA-256 + hashtree Keccak old Keccak SIMD
merkle/hash_nodes/single 53.23 ns 64.27 ns (0.83x) 57.81 ns (0.92x) 207.04 ns 146.51 ns (1.41x)
merkle/mix_in_length/single 67.26 ns 63.65 ns (1.06x) 57.37 ns (1.17x) 206.88 ns 144.51 ns (1.43x)
hash_tree_root/consensus_containers/checkpoint 77.21 ns 65.30 ns (1.18x) 58.56 ns (1.32x) 220.10 ns 155.31 ns (1.42x)
hash_tree_root/containers/validator 496.06 ns 393.15 ns (1.26x) 298.56 ns (1.66x) 1.66 µs 626.83 ns (2.65x)
hash_tree_root/list_validator/10000 5.51 ms 4.29 ms (1.28x) 3.28 ms (1.68x) 18.72 ms 6.39 ms (2.93x)
hash_tree_root/beacon_state/16384 14.50 ms 11.20 ms (1.29x) 8.63 ms (1.68x) 49.42 ms 13.00 ms (3.80x)
hash_tree_root/beacon_state/300000 174.02 ms 135.91 ms (1.28x) 104.04 ms (1.67x) 591.53 ms 193.79 ms (3.05x)
diff/htr/beacon_state/libssz/1000000 569.14 ms 445.94 ms (1.28x) 340.35 ms (1.67x) 1.94 s 646.62 ms (3.00x)
hash_tree_root/list_u64/100000 1.62 ms 1.05 ms (1.54x) 792.71 µs (2.04x) 5.37 ms 736.28 µs (7.30x)
merkle/merkleize/4096 218.85 µs 157.61 µs (1.39x) 115.82 µs (1.89x) 828.14 µs 79.38 µs (10.43x)
merkle/merkleize_with_limit/chunks/65536_of_262144 3.52 ms 2.53 ms (1.39x) 1.86 ms (1.89x) 13.28 ms 1.28 ms (10.39x)
control: diff/htr/beacon_state/lighthouse/1000000 842.09 ms 843.77 ms 843.83 ms 843.15 ms 854.76 ms
control: diff/htr/beacon_state/ssz_rs/1000000 960.19 ms 959.13 ms 964.57 ms 953.03 ms 954.99 ms
no hashing: merkle/pack/32768 1.59 µs 1.59 µs 1.60 µs 1.70 µs 1.59 µs

Keccak SIMD time divided by SHA-256 time (bold = Keccak is faster):

Keccak SIMD time ÷ SHA-256 main SHA-256 new SHA-256 + hashtree
merkle/hash_nodes/single 2.75 2.28 2.53
merkle/mix_in_length/single 2.15 2.27 2.52
hash_tree_root/consensus_containers/checkpoint 2.01 2.38 2.65
hash_tree_root/containers/validator 1.26 1.59 2.10
hash_tree_root/list_validator/10000 1.16 1.49 1.95
hash_tree_root/beacon_state/16384 0.90 1.16 1.51
hash_tree_root/beacon_state/300000 1.11 1.43 1.86
diff/htr/beacon_state/libssz/1000000 1.14 1.45 1.90
hash_tree_root/list_u64/100000 0.46 0.70 0.93
merkle/merkleize/4096 0.36 0.50 0.69
merkle/merkleize_with_limit/chunks/65536_of_262144 0.36 0.51 0.69

All 91 benchmarks, noise checks and caveats are in the results comment. In short:

  • The SHA-256 branch's single hash_nodes got slower, for reasons not yet found.
  • AVX2 Keccak has only rough numbers from a probe on this machine.
  • The current code isn't benchmarked on ARM.

First version: one pair at a time through sha3

Measurements of 58d16bd on Apple M3 Pro and x86_64

These are the original measurements of this PR's first commit, which hashed one pair at a time through sha3::Keccak256, against main @ 958a90f. The sha2 0.11 bump suggested in the M3 section is done in perf/batched-sha256-htr, but that branch isn't benchmarked on ARM yet.

Apple M3 Pro: SHA-256 (main @ 958a90f) vs Keccak-256 (this branch)

Setup: Apple M3 Pro, macOS, rustc 1.99.1, bench profile (opt-level=3, thin LTO), criterion 0.5. I ran cargo bench -p ssz-bench --bench hash_tree_root --bench merkle plus --bench differential -- diff/htr, saving one baseline per side and comparing criterion medians. Encode/decode benches were skipped because they never hash.

What this compares: on aarch64, sha2 0.10 without its asm feature uses its software implementation, and so does sha3. So this is software SHA-256 vs software Keccak-256. On x86_64 with SHA-NI (e.g. GitHub runners), sha2 picks hardware SHA-256 automatically, so the result there would likely be very different. See M3 hardware acceleration below.

Noise: the machine had heavy background load during the runs. One burst inflated a contiguous block of the Keccak run (beacon_state, hash_nodes, pack), including benches that never hash. Those rows (marked †) were re-run alternating SHA-256 → Keccak → SHA-256 → Keccak; both rounds agreed, and the table uses round 1. The lighthouse / ssz_rs rows always use SHA-256, so they act as a control and show the run-to-run noise (0–7%).

Summary

benchmark sha2 keccak keccak / sha2
merkle/hash_nodes/single † 264.52 ns 158.71 ns 0.60x
merkle/mix_in_length/single 269.81 ns 155.97 ns 0.58x
merkle/merkleize/1024 283.99 µs 160.15 µs 0.56x
merkle/merkleize_with_limit/chunks/65536_of_262144 17.55 ms 10.44 ms 0.59x
hash_tree_root/containers/validator 2.32 µs 1.41 µs 0.61x
hash_tree_root/list_validator/10000 25.35 ms 15.37 ms 0.61x
hash_tree_root/vec_u64/100000 6.99 ms 4.24 ms 0.61x
hash_tree_root/beacon_state/300000 † 788.48 ms 491.77 ms 0.62x
diff/htr/beacon_state/libssz/1000000 2.62 s 1.57 s 0.60x
control: diff/htr/beacon_state/lighthouse/1000000 857.02 ms 850.70 ms 0.99x
control: diff/htr/beacon_state/ssz_rs/1000000 3.11 s 3.03 s 0.98x
no hashing: merkle/pack/32768 † 3.69 µs 3.66 µs 0.99x
no hashing: hash_tree_root/primitives/u64 0.45 ns 0.43 ns 0.96x

Takeaway (software only): anything that hashes is consistently ~38–44% faster with Keccak-256, and benches that don't hash are unchanged. This fits the cost of one 64-byte hash_nodes input: SHA-256 runs two compressions (the data block plus a padding block), while Keccak-256 (136-byte rate) absorbs it in a single Keccak-f[1600] permutation.

Note that libssz with soft SHA-256 is ~3x slower than Lighthouse's tree_hash on beacon_state here (Lighthouse hashes through ethereum_hashing, which uses ring's assembly SHA-256 on aarch64). Switching to soft Keccak narrows that to ~1.8x but doesn't close it.

All 91 benchmarks
benchmark sha2 keccak keccak / sha2
diff/htr/beacon_state/libssz/100000 280.61 ms 163.54 ms 0.58x (-41.7%)
diff/htr/beacon_state/libssz/1000000 2.62 s 1.57 s 0.60x (-40.3%)
diff/htr/beacon_state/libssz/16384 66.40 ms 38.75 ms 0.58x (-41.6%)
diff/htr/beacon_state/libssz/300000 809.57 ms 479.71 ms 0.59x (-40.7%)
diff/htr/beacon_state/lighthouse/100000 92.13 ms 86.61 ms 0.94x (-6.0%)
diff/htr/beacon_state/lighthouse/1000000 857.02 ms 850.70 ms 0.99x (-0.7%)
diff/htr/beacon_state/lighthouse/16384 21.41 ms 20.76 ms 0.97x (-3.0%)
diff/htr/beacon_state/lighthouse/300000 261.78 ms 254.37 ms 0.97x (-2.8%)
diff/htr/beacon_state/ssz_rs/100000 366.57 ms 339.45 ms 0.93x (-7.4%)
diff/htr/beacon_state/ssz_rs/1000000 3.11 s 3.03 s 0.98x (-2.4%)
diff/htr/beacon_state/ssz_rs/16384 112.37 ms 104.71 ms 0.93x (-6.8%)
diff/htr/beacon_state/ssz_rs/300000 979.53 ms 944.40 ms 0.96x (-3.6%)
diff/htr/consensus_containers/libssz/attestation_data 2.27 µs 1.32 µs 0.58x (-41.9%)
diff/htr/consensus_containers/libssz/checkpoint 296.31 ns 175.96 ns 0.59x (-40.6%)
diff/htr/consensus_containers/libssz/eth1_data 842.18 ns 499.49 ns 0.59x (-40.7%)
diff/htr/consensus_containers/libssz/fork 824.93 ns 491.30 ns 0.60x (-40.4%)
diff/htr/libssz/bool 0.68 ns 0.67 ns 0.97x (-2.7%)
diff/htr/libssz/bytes32 0.63 ns 0.62 ns 0.99x (-1.4%)
diff/htr/libssz/u64 0.43 ns 0.43 ns 0.99x (-1.1%)
diff/htr/lighthouse/bool 0.56 ns 0.56 ns 1.00x (-0.4%)
diff/htr/lighthouse/bytes32 0.64 ns 0.63 ns 0.99x (-1.3%)
diff/htr/lighthouse/u64 0.65 ns 0.65 ns 1.00x (-0.4%)
diff/htr/ssz_rs/bool 0.49 ns 0.48 ns 0.98x (-1.6%)
diff/htr/ssz_rs/bytes32 79.27 ns 75.90 ns 0.96x (-4.2%)
diff/htr/ssz_rs/u64 41.61 ns 40.96 ns 0.98x (-1.6%)
hash_tree_root/beacon_state/100000 † 282.85 ms 171.72 ms 0.61x (-39.3%)
hash_tree_root/beacon_state/16384 † 65.78 ms 41.12 ms 0.63x (-37.5%)
hash_tree_root/beacon_state/300000 † 788.48 ms 491.77 ms 0.62x (-37.6%)
hash_tree_root/bitfields/bitlist/2048 2.32 µs 1.34 µs 0.58x (-42.0%)
hash_tree_root/bitfields/bitlist/4096 4.61 µs 2.64 µs 0.57x (-42.8%)
hash_tree_root/bitfields/bitlist/64 301.81 ns 184.60 ns 0.61x (-38.8%)
hash_tree_root/bitfields/bitvector/4096 4.27 µs 2.48 µs 0.58x (-42.1%)
hash_tree_root/bitfields/bitvector/512 339.19 ns 206.86 ns 0.61x (-39.0%)
hash_tree_root/bitfields/bitvector/64 35.00 ns 34.80 ns 0.99x (-0.6%)
hash_tree_root/consensus_containers/attestation_data 2.29 µs 1.34 µs 0.58x (-41.5%)
hash_tree_root/consensus_containers/checkpoint 307.98 ns 185.95 ns 0.60x (-39.6%)
hash_tree_root/consensus_containers/eth1_data 843.94 ns 499.93 ns 0.59x (-40.8%)
hash_tree_root/consensus_containers/fork 845.87 ns 510.77 ns 0.60x (-39.6%)
hash_tree_root/consensus_containers/pending_attestation 5.52 µs 3.27 µs 0.59x (-40.8%)
hash_tree_root/containers/beacon_block_header 1.76 µs 974.42 ns 0.55x (-44.7%)
hash_tree_root/containers/validator 2.32 µs 1.41 µs 0.61x (-39.1%)
hash_tree_root/list_u64/100 11.67 µs 7.08 µs 0.61x (-39.3%)
hash_tree_root/list_u64/1000 72.31 µs 44.40 µs 0.61x (-38.6%)
hash_tree_root/list_u64/100000 6.95 ms 4.18 ms 0.60x (-39.9%)
hash_tree_root/list_validator/100 255.95 µs 155.34 µs 0.61x (-39.3%)
hash_tree_root/list_validator/1000 2.57 ms 1.51 ms 0.59x (-41.0%)
hash_tree_root/list_validator/10000 25.35 ms 15.37 ms 0.61x (-39.4%)
hash_tree_root/nested_container/10 28.61 µs 17.01 µs 0.59x (-40.5%)
hash_tree_root/nested_container/100 259.06 µs 151.56 µs 0.59x (-41.5%)
hash_tree_root/nested_container/1000 2.54 ms 1.52 ms 0.60x (-40.1%)
hash_tree_root/primitives/bool 0.70 ns 0.68 ns 0.97x (-2.9%)
hash_tree_root/primitives/bytes32 0.69 ns 0.69 ns 0.99x (-1.2%)
hash_tree_root/primitives/u128 0.43 ns 0.42 ns 0.97x (-2.7%)
hash_tree_root/primitives/u16 0.44 ns 0.42 ns 0.96x (-3.9%)
hash_tree_root/primitives/u32 0.44 ns 0.44 ns 0.99x (-0.9%)
hash_tree_root/primitives/u64 0.45 ns 0.43 ns 0.96x (-3.9%)
hash_tree_root/primitives/u8 0.42 ns 0.43 ns 1.01x (+1.3%)
hash_tree_root/unions/0 276.23 ns 157.88 ns 0.57x (-42.8%)
hash_tree_root/unions/1 270.92 ns 156.21 ns 0.58x (-42.3%)
hash_tree_root/unions/2 1.98 µs 1.13 µs 0.57x (-42.9%)
hash_tree_root/unions/3 2.88 µs 1.74 µs 0.60x (-39.6%)
hash_tree_root/variable_container/100 10.57 µs 6.41 µs 0.61x (-39.3%)
hash_tree_root/variable_container/1000 83.98 µs 49.72 µs 0.59x (-40.8%)
hash_tree_root/variable_container/10000 806.39 µs 498.83 µs 0.62x (-38.1%)
hash_tree_root/vec_u64/100 8.18 µs 4.85 µs 0.59x (-40.7%)
hash_tree_root/vec_u64/1000 70.73 µs 41.97 µs 0.59x (-40.7%)
hash_tree_root/vec_u64/100000 6.99 ms 4.24 ms 0.61x (-39.3%)
hash_tree_root/vector_bytes32/100 28.61 µs 16.77 µs 0.59x (-41.4%)
hash_tree_root/vector_bytes32/1000 281.78 µs 162.63 µs 0.58x (-42.3%)
merkle/hash_nodes/single † 264.52 ns 158.71 ns 0.60x (-40.0%)
merkle/merkleize/1 22.46 ns 21.87 ns 0.97x (-2.6%)
merkle/merkleize/1024 283.99 µs 160.15 µs 0.56x (-43.6%)
merkle/merkleize/16 4.24 µs 2.41 µs 0.57x (-43.0%)
merkle/merkleize/256 68.66 µs 39.98 µs 0.58x (-41.8%)
merkle/merkleize/4 829.68 ns 493.56 ns 0.59x (-40.5%)
merkle/merkleize/4096 1.13 ms 633.85 µs 0.56x (-43.7%)
merkle/merkleize/64 17.50 µs 10.05 µs 0.57x (-42.6%)
merkle/merkleize_with_limit/chunks/1024_of_1048576 277.58 µs 162.07 µs 0.58x (-41.6%)
merkle/merkleize_with_limit/chunks/256_of_4096 69.95 µs 42.39 µs 0.61x (-39.4%)
merkle/merkleize_with_limit/chunks/64_of_1024 18.05 µs 10.50 µs 0.58x (-41.9%)
merkle/merkleize_with_limit/chunks/65536_of_262144 17.55 ms 10.44 ms 0.59x (-40.5%)
merkle/merkleize_with_limit/chunks/65537_of_262144 17.57 ms 10.43 ms 0.59x (-40.6%)
merkle/mix_in_length/single 269.81 ns 155.97 ns 0.58x (-42.2%)
merkle/pack/1024 † 137.50 ns 134.28 ns 0.98x (-2.3%)
merkle/pack/256 † 39.59 ns 39.97 ns 1.01x (+1.0%)
merkle/pack/32 † 13.26 ns 13.50 ns 1.02x (+1.8%)
merkle/pack/32768 † 3.69 µs 3.66 µs 0.99x (-1.0%)
merkle/pack_bits/2048 † 40.30 ns 40.32 ns 1.00x (+0.1%)
merkle/pack_bits/512 † 18.26 ns 18.56 ns 1.02x (+1.6%)
merkle/pack_bits/64 † 12.26 ns 12.59 ns 1.03x (+2.7%)
merkle/pack_bits/8192 † 136.86 ns 134.35 ns 0.98x (-1.8%)

† Re-run alternating with the other side (see "Noise" above).

Apple M3 Pro, hardware acceleration: SHA-256 wins by ~3x

On aarch64, sha2's asm feature turns on the ARMv8 SHA-256 instructions (x86 already gets SHA-NI by default), and sha3's asm feature turns on the ARMv8 SHA3 instructions (the M3 has both). Neither is enabled in this repo. I built each side with its asm feature in throwaway worktrees (not committed) and ran all four variants back to back, twice in alternation.

Round 2 was clean: load average under 5, and the no-hash control merkleize/1 is flat across all four variants. Round 1 hit another load spike that doubled the control for soft SHA-256, so its soft rows are discarded; its two asm columns reproduced round 2 within ~4%.

benchmark (round 2) sha2 (soft) keccak (soft) sha2 + asm keccak + asm
merkle/hash_nodes/single 269.55 ns 158.55 ns (0.59x) 44.80 ns (0.17x) 147.59 ns (0.55x)
merkle/merkleize/1024 281.88 µs 164.31 µs (0.58x) 46.43 µs (0.16x) 154.03 µs (0.55x)
merkle/merkleize/4096 1.12 ms 657.87 µs (0.59x) 188.48 µs (0.17x) 622.89 µs (0.56x)
hash_tree_root/list_validator/1000 2.58 ms 1.56 ms (0.60x) 522.41 µs (0.20x) 1.41 ms (0.55x)
hash_tree_root/beacon_state/16384 67.19 ms 39.76 ms (0.59x) 12.86 ms (0.19x) 37.54 ms (0.56x)
control (no hashing): merkle/merkleize/1 22.51 ns 22.16 ns (0.98x) 22.19 ns (0.99x) 22.24 ns (0.99x)

Ratios are relative to soft SHA-256.

  • Hardware SHA-256 is ~5–6x faster than software SHA-256, and ~3x faster than Keccak-256 with or without asm.
  • sha3's asm gives Keccak only ~5–7% (159 → 148 ns per hash_nodes).
  • So Keccak-256 only wins when SHA-256 runs in software. On CPUs with SHA extensions (x86 SHA-NI, ARMv8 SHA2) switching to Keccak would make hash_tree_root ~3x slower, besides breaking consensus compatibility.

Side finding, worth its own PR: using the ARM SHA-256 instructions makes libssz's hash_tree_root ~5x faster on Apple Silicon / ARM. On beacon_state/16384 that's 12.86 ms, against 21.41 ms for Lighthouse's tree_hash in the first run above (a cross-run comparison). The way to get this is probably bumping sha2 to 0.11 rather than enabling asm on 0.10: 0.11.1 detects the ARMv8 SHA-256 instructions (and x86 SHA-NI) at runtime in its default build, with no asm feature or C toolchain. It moves to digest 0.11, so it's an API bump, not a drop-in. I haven't benchmarked 0.11 itself.

x86_64 (Ryzen 9 9950X3D): SHA-256 wins by ~3.5x

Setup: AMD Ryzen 9 9950X3D (Zen 5, SHA-NI, AVX-512), Linux 6.12, rustc 1.95.0, same bench profile and benches as above.

  • Each bench process was pinned to one core with taskset -c 4.
  • The box was also running two mainnet nodes; load was ~2.4 on 32 threads during the runs.
  • On x86, sha2 0.10 picks SHA-NI at runtime by default, so main is hardware SHA-256 here with no feature changes. sha3's asm feature only affects aarch64, so this branch is software Keccak either way.
  • The software SHA-256 reference is main with sha2's force-soft feature (not committed).

Noise: results are very stable. The full run and two alternating rounds agree within ~1–2%, and the controls (Lighthouse, ssz_rs, pack, primitives) are flat.

Summary

benchmark sha2 (SHA-NI) keccak keccak / sha2
merkle/hash_nodes/single 53.29 ns 206.87 ns 3.88x
merkle/mix_in_length/single 67.13 ns 215.02 ns 3.20x
merkle/merkleize/1024 54.74 µs 207.30 µs 3.79x
merkle/merkleize_with_limit/chunks/65536_of_262144 3.52 ms 13.29 ms 3.78x
hash_tree_root/containers/validator 502.43 ns 1.66 µs 3.31x
hash_tree_root/list_validator/10000 5.49 ms 19.49 ms 3.55x
hash_tree_root/vec_u64/100000 1.58 ms 5.33 ms 3.37x
hash_tree_root/beacon_state/300000 175.51 ms 592.77 ms 3.38x
diff/htr/beacon_state/libssz/1000000 575.99 ms 1.94 s 3.36x
control: diff/htr/beacon_state/lighthouse/1000000 862.00 ms 865.12 ms 1.00x
control: diff/htr/beacon_state/ssz_rs/1000000 968.76 ms 964.36 ms 1.00x
no hashing: merkle/pack/32768 1.60 µs 1.61 µs 1.00x
no hashing: hash_tree_root/primitives/u64 2.49 ns 2.44 ns 0.98x

Software SHA-256 reference

benchmark (round 2) sha2 (SHA-NI) keccak (soft) sha2 force-soft
hash_tree_root/beacon_state/16384 14.42 ms 49.32 ms (3.42x) 51.30 ms (3.56x)
hash_tree_root/list_validator/1000 549.12 µs 1.87 ms (3.40x) 2.00 ms (3.64x)
merkle/hash_nodes/single 53.15 ns 207.40 ns (3.90x) 210.02 ns (3.95x)
merkle/merkleize/1024 54.75 µs 207.84 µs (3.80x) 211.24 µs (3.86x)
merkle/merkleize/4096 219.09 µs 830.43 µs (3.79x) 845.53 µs (3.86x)
control (no hashing): merkle/merkleize/1 12.72 ns 12.74 ns (1.00x) 12.81 ns (1.01x)

Round 1 matched round 2 within 3.5%.

  • With SHA-NI, Keccak-256 makes hash_tree_root 2.6–3.9x slower across all 57 benches that hash. The low end is small objects that only do a few hashes (bitfields, Checkpoint, unions); large lists and BeaconState are 3.3–3.6x; a single 64-byte hash_nodes is 3.9x (53 ns vs 207 ns). The 20 benches that don't hash are within ±5%.
  • Software vs software, Keccak barely wins on Zen 5: from ~2% slower to ~7% faster, against ~40% faster on the M3. So on x86 even a SHA-256 build without SHA-NI would gain little from Keccak.
  • Nothing in this repo needs changing for x86 hardware SHA-256: main already gets it by default. The gap is aarch64 only (see the side finding above).
All 91 benchmarks (x86_64)
benchmark sha2 (SHA-NI) keccak keccak / sha2
diff/htr/beacon_state/libssz/100000 61.57 ms 209.91 ms 3.41x (+240.9%)
diff/htr/beacon_state/libssz/1000000 575.99 ms 1.94 s 3.36x (+236.1%)
diff/htr/beacon_state/libssz/16384 14.12 ms 49.56 ms 3.51x (+251.0%)
diff/htr/beacon_state/libssz/300000 175.20 ms 593.48 ms 3.39x (+238.7%)
diff/htr/beacon_state/lighthouse/100000 93.65 ms 93.76 ms 1.00x (+0.1%)
diff/htr/beacon_state/lighthouse/1000000 862.00 ms 865.12 ms 1.00x (+0.4%)
diff/htr/beacon_state/lighthouse/16384 22.10 ms 22.09 ms 1.00x (-0.1%)
diff/htr/beacon_state/lighthouse/300000 264.32 ms 265.48 ms 1.00x (+0.4%)
diff/htr/beacon_state/ssz_rs/100000 141.49 ms 140.31 ms 0.99x (-0.8%)
diff/htr/beacon_state/ssz_rs/1000000 968.76 ms 964.36 ms 1.00x (-0.5%)
diff/htr/beacon_state/ssz_rs/16384 65.05 ms 64.95 ms 1.00x (-0.2%)
diff/htr/beacon_state/ssz_rs/300000 323.77 ms 318.70 ms 0.98x (-1.6%)
diff/htr/consensus_containers/libssz/attestation_data 517.44 ns 1.67 µs 3.23x (+223.5%)
diff/htr/consensus_containers/libssz/checkpoint 76.89 ns 218.86 ns 2.85x (+184.6%)
diff/htr/consensus_containers/libssz/eth1_data 197.97 ns 627.25 ns 3.17x (+216.8%)
diff/htr/consensus_containers/libssz/fork 198.07 ns 625.37 ns 3.16x (+215.7%)
diff/htr/libssz/bool 2.26 ns 2.36 ns 1.05x (+4.6%)
diff/htr/libssz/bytes32 2.91 ns 2.89 ns 0.99x (-0.7%)
diff/htr/libssz/u64 2.45 ns 2.44 ns 1.00x (-0.3%)
diff/htr/lighthouse/bool 2.18 ns 2.28 ns 1.05x (+4.8%)
diff/htr/lighthouse/bytes32 2.91 ns 2.89 ns 1.00x (-0.5%)
diff/htr/lighthouse/u64 2.19 ns 2.18 ns 1.00x (-0.3%)
diff/htr/ssz_rs/bool 2.32 ns 2.32 ns 1.00x (-0.0%)
diff/htr/ssz_rs/bytes32 57.19 ns 56.60 ns 0.99x (-1.0%)
diff/htr/ssz_rs/u64 32.86 ns 32.83 ns 1.00x (-0.1%)
hash_tree_root/beacon_state/100000 62.81 ms 210.31 ms 3.35x (+234.8%)
hash_tree_root/beacon_state/16384 14.56 ms 49.32 ms 3.39x (+238.8%)
hash_tree_root/beacon_state/300000 175.51 ms 592.77 ms 3.38x (+237.7%)
hash_tree_root/bitfields/bitlist/2048 490.71 ns 1.66 µs 3.37x (+237.5%)
hash_tree_root/bitfields/bitlist/4096 932.17 ns 3.30 µs 3.54x (+254.1%)
hash_tree_root/bitfields/bitlist/64 83.75 ns 223.41 ns 2.67x (+166.8%)
hash_tree_root/bitfields/bitvector/4096 865.08 ns 3.09 µs 3.58x (+257.6%)
hash_tree_root/bitfields/bitvector/512 85.27 ns 225.48 ns 2.64x (+164.4%)
hash_tree_root/bitfields/bitvector/64 20.84 ns 20.71 ns 0.99x (-0.6%)
hash_tree_root/consensus_containers/attestation_data 506.30 ns 1.67 µs 3.31x (+230.5%)
hash_tree_root/consensus_containers/checkpoint 77.21 ns 219.64 ns 2.84x (+184.5%)
hash_tree_root/consensus_containers/eth1_data 196.42 ns 630.51 ns 3.21x (+221.0%)
hash_tree_root/consensus_containers/fork 197.97 ns 628.56 ns 3.18x (+217.5%)
hash_tree_root/consensus_containers/pending_attestation 1.20 µs 3.95 µs 3.30x (+230.3%)
hash_tree_root/containers/beacon_block_header 358.85 ns 1.24 µs 3.46x (+245.8%)
hash_tree_root/containers/validator 502.43 ns 1.66 µs 3.31x (+230.6%)
hash_tree_root/list_u64/100 2.63 µs 8.64 µs 3.29x (+228.9%)
hash_tree_root/list_u64/1000 15.32 µs 54.39 µs 3.55x (+255.0%)
hash_tree_root/list_u64/100000 1.62 ms 5.36 ms 3.32x (+231.6%)
hash_tree_root/list_validator/100 56.01 µs 197.92 µs 3.53x (+253.4%)
hash_tree_root/list_validator/1000 549.31 µs 1.95 ms 3.55x (+254.5%)
hash_tree_root/list_validator/10000 5.49 ms 19.49 ms 3.55x (+255.1%)
hash_tree_root/nested_container/10 6.20 µs 20.94 µs 3.38x (+237.9%)
hash_tree_root/nested_container/100 55.72 µs 189.35 µs 3.40x (+239.8%)
hash_tree_root/nested_container/1000 549.83 µs 1.87 ms 3.40x (+239.7%)
hash_tree_root/primitives/bool 2.35 ns 2.27 ns 0.96x (-3.6%)
hash_tree_root/primitives/bytes32 8.13 ns 8.10 ns 1.00x (-0.3%)
hash_tree_root/primitives/u128 2.39 ns 2.38 ns 0.99x (-0.6%)
hash_tree_root/primitives/u16 2.60 ns 2.55 ns 0.98x (-2.2%)
hash_tree_root/primitives/u32 2.48 ns 2.44 ns 0.98x (-1.6%)
hash_tree_root/primitives/u64 2.49 ns 2.44 ns 0.98x (-1.9%)
hash_tree_root/primitives/u8 2.56 ns 2.51 ns 0.98x (-2.1%)
hash_tree_root/unions/0 67.65 ns 206.15 ns 3.05x (+204.7%)
hash_tree_root/unions/1 67.29 ns 206.73 ns 3.07x (+207.2%)
hash_tree_root/unions/2 423.05 ns 1.44 µs 3.41x (+241.3%)
hash_tree_root/unions/3 777.85 ns 2.08 µs 2.67x (+167.3%)
hash_tree_root/variable_container/100 2.41 µs 7.56 µs 3.14x (+213.7%)
hash_tree_root/variable_container/1000 17.28 µs 60.08 µs 3.48x (+247.7%)
hash_tree_root/variable_container/10000 164.67 µs 587.14 µs 3.57x (+256.6%)
hash_tree_root/vec_u64/100 1.76 µs 5.96 µs 3.39x (+238.6%)
hash_tree_root/vec_u64/1000 14.61 µs 52.44 µs 3.59x (+258.9%)
hash_tree_root/vec_u64/100000 1.58 ms 5.33 ms 3.37x (+237.4%)
hash_tree_root/vector_bytes32/100 5.98 µs 21.28 µs 3.56x (+255.7%)
hash_tree_root/vector_bytes32/1000 56.66 µs 206.12 µs 3.64x (+263.8%)
merkle/hash_nodes/single 53.29 ns 206.87 ns 3.88x (+288.2%)
merkle/merkleize/1 12.51 ns 12.59 ns 1.01x (+0.6%)
merkle/merkleize/1024 54.74 µs 207.30 µs 3.79x (+278.7%)
merkle/merkleize/16 833.91 ns 3.06 µs 3.67x (+266.6%)
merkle/merkleize/256 13.66 µs 51.72 µs 3.79x (+278.6%)
merkle/merkleize/4 195.87 ns 621.62 ns 3.17x (+217.4%)
merkle/merkleize/4096 219.09 µs 829.01 µs 3.78x (+278.4%)
merkle/merkleize/64 3.42 µs 12.78 µs 3.74x (+273.6%)
merkle/merkleize_with_limit/chunks/1024_of_1048576 55.40 µs 209.36 µs 3.78x (+277.9%)
merkle/merkleize_with_limit/chunks/256_of_4096 13.94 µs 52.46 µs 3.76x (+276.3%)
merkle/merkleize_with_limit/chunks/64_of_1024 3.69 µs 13.60 µs 3.69x (+268.6%)
merkle/merkleize_with_limit/chunks/65536_of_262144 3.52 ms 13.29 ms 3.78x (+278.0%)
merkle/merkleize_with_limit/chunks/65537_of_262144 3.52 ms 13.31 ms 3.78x (+277.9%)
merkle/mix_in_length/single 67.13 ns 215.02 ns 3.20x (+220.3%)
merkle/pack/1024 61.77 ns 62.09 ns 1.01x (+0.5%)
merkle/pack/256 18.39 ns 18.05 ns 0.98x (-1.9%)
merkle/pack/32 10.80 ns 10.96 ns 1.02x (+1.5%)
merkle/pack/32768 1.60 µs 1.61 µs 1.00x (+0.4%)
merkle/pack_bits/2048 18.59 ns 18.37 ns 0.99x (-1.2%)
merkle/pack_bits/512 12.02 ns 12.41 ns 1.03x (+3.3%)
merkle/pack_bits/64 11.13 ns 10.93 ns 0.98x (-1.8%)
merkle/pack_bits/8192 62.12 ns 62.48 ns 1.01x (+0.6%)

MegaRedHand and others added 6 commits November 6, 2026 13:23
@MegaRedHand @claude
feat!: use Keccak-256 instead of SHA-256 for hash_tree_root
58d16bd 
Swap the `sha2` backend for `sha3::Keccak256` (legacy Keccak padding, as
used by Ethereum's execution layer), including the build-time ZERO_HASHES
table so zero-subtree elision stays consistent with `hash_nodes`.

Breaking renames:
- trait `Sha256Hasher` -> `MerkleHasher`
- struct `Sha2Hasher` -> `KeccakHasher`
- feature `sha2-backend` -> `keccak-backend`

Roots are no longer consensus-compatible: the two unit tests pinned to
SHA-256 reference roots are ignored, and spec-tests / differential fuzz
targets against Lighthouse are expected to fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011vghNkypyMGfhziz7iorub
@MegaRedHand
perf: batch merkleize levels and speed up SHA-256 pair hashing
72b2889 
- `Sha256Hasher::hash_pairs` hashes many 64-byte pairs per call. It has a
  default implementation, so existing hashers keep working unchanged.
  `merkleize` hands each tree level to it in one batch, and `hash_nodes`
  routes single pairs through it.
- `merkleize` hashes level 0 straight from the input chunks instead of
  copying them first.
- `Sha2Hasher` moves to `sha2` 0.11, which selects ARMv8 SHA2 instructions
  at runtime (0.10 only used them with its `asm` feature), and hashes pairs
  by calling the compression function directly on the message plus a
  constant padding block.
- New opt-in `hashtree` feature: pairs are hashed by the hashtree assembly
  library (SHA-NI, AVX-512/AVX2 multi-buffer, ARMv8 SHA2, NEON). Requires a
  C toolchain; CI tests it as a separate matrix entry.
@MegaRedHand
Merge branch 'perf/batched-sha256-htr' into feat/keccak-htr
0d00bce 
Keeps this branch's tree: Keccak-256 replaces the SHA-256 backend, and the next commit ports the level batching.
@MegaRedHand
perf: batch merkleize levels and hash Keccak pairs with SIMD
88557bb 
Port the level batching from perf/batched-sha256-htr to `MerkleHasher`:
`merkleize` hands every full pair of a level to `hash_pairs` in one call,
and `hash_nodes` goes through the same method.

`KeccakHasher` moves to its own module and skips the general-purpose
sponge: each 64-byte pair is one Keccak-f[1600] permutation of a
pre-padded state. `hash_pairs` runs several permutations at once, chosen at
runtime with `cpufeatures`: AVX-512 (8 pairs), AVX2 (4) or the ARMv8 SHA3
extension (2). Leftover pairs and other CPUs use the scalar permutation
from the `keccak` crate. Each backend is tested against `sha3::Keccak256`.
@MegaRedHand
perf: build AVX2 Keccak rows in registers, use AVX-512 for single pairs
bcb6a86 
AVX2: assemble each state row from 64-bit message words in registers
instead of transposing through a stack buffer, whose 64-bit stores read
back as one vector stalled on store forwarding. On Zen 5 a full 4-way
batch drops from about 378 to 307 ns (94 to 77 ns per hash).

AVX-512: one 8-way permutation (about 143 ns) already beats one scalar
permutation (about 192 ns), so a leftover single pair uses it too.
@MegaRedHand
docs: correct Keccak leftover-pair and AVX2 notes
135e403 
With AVX-512 a single leftover pair now runs in SIMD, so the docs no
longer say leftovers always use the scalar permutation.

The AVX2 row assembly measured faster on Zen 5, but the cause is not
established: the previous commit attributed it to store-forwarding
stalls, while timings also shifted with unrelated code layout changes.
@MegaRedHand

MegaRedHand commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Updated x86_64 results: faster SHA-256 and SIMD Keccak

The description above compares main against the first version of this PR. Since then both sides got faster:

  • SHA-256 new (perf/batched-sha256-htr @ 72b2889, no PR yet). merkleize hashes each tree level as one batch, and pairs go straight to the sha2 0.11 compression function, skipping the Digest API. An optional libssz-merkle/hashtree feature hashes the batches with hashtree instead. All consensus spec tests pass with both backends, so roots are unchanged.
  • Keccak SIMD (this PR @ bcb6a86; 135e403 after it only changes comments and docs). Same level batching, plus a Keccak-f[1600] that hashes 8 pairs at once with AVX-512, 4 with AVX2 and 2 with the ARMv8 SHA3 instructions, picked at runtime. Without those, and for a single leftover pair on AVX2 or ARM, it uses the scalar permutation from the keccak crate.

This PR is now built on top of perf/batched-sha256-htr (merged in) and reuses its level batching, so until that branch lands, the diff against main also includes those batching changes.

Setup: same machine and method as the x86_64 section above: Ryzen 9 9950X3D (Zen 5, SHA-NI, AVX-512), Linux 6.12, rustc 1.95.0, bench profile, criterion medians, every bench pinned with taskset -c 4. Commits: main = 958a90f, Keccak old = 58d16bd. The full suite ran once per variant, then interleaved rounds repeated a subset; those agreed within about 2% on every row above 100 ns. The Keccak SIMD column comes from a later session than the others. In that session the previous SIMD build (88557bb) reproduced its earlier numbers within about 3%. Control rows above 1 µs (Lighthouse, ssz_rs, pack) stay within 2% across all columns, except pack/32768 at +7% in the old Keccak column. Rows under 100 ns are noisier: diff/htr/ssz_rs/bytes32, which doesn't touch libssz, moved about −11% in both SHA-256 new columns.

Summary

Brackets: SHA-256 columns vs SHA-256 main, Keccak SIMD vs Keccak old (higher is faster).

benchmark SHA-256 main SHA-256 new SHA-256 + hashtree Keccak old Keccak SIMD
merkle/hash_nodes/single 53.23 ns 64.27 ns (0.83x) 57.81 ns (0.92x) 207.04 ns 146.51 ns (1.41x)
merkle/mix_in_length/single 67.26 ns 63.65 ns (1.06x) 57.37 ns (1.17x) 206.88 ns 144.51 ns (1.43x)
hash_tree_root/consensus_containers/checkpoint 77.21 ns 65.30 ns (1.18x) 58.56 ns (1.32x) 220.10 ns 155.31 ns (1.42x)
hash_tree_root/containers/validator 496.06 ns 393.15 ns (1.26x) 298.56 ns (1.66x) 1.66 µs 626.83 ns (2.65x)
hash_tree_root/list_validator/10000 5.51 ms 4.29 ms (1.28x) 3.28 ms (1.68x) 18.72 ms 6.39 ms (2.93x)
hash_tree_root/beacon_state/16384 14.50 ms 11.20 ms (1.29x) 8.63 ms (1.68x) 49.42 ms 13.00 ms (3.80x)
hash_tree_root/beacon_state/300000 174.02 ms 135.91 ms (1.28x) 104.04 ms (1.67x) 591.53 ms 193.79 ms (3.05x)
diff/htr/beacon_state/libssz/1000000 569.14 ms 445.94 ms (1.28x) 340.35 ms (1.67x) 1.94 s 646.62 ms (3.00x)
hash_tree_root/list_u64/100000 1.62 ms 1.05 ms (1.54x) 792.71 µs (2.04x) 5.37 ms 736.28 µs (7.30x)
merkle/merkleize/4096 218.85 µs 157.61 µs (1.39x) 115.82 µs (1.89x) 828.14 µs 79.38 µs (10.43x)
merkle/merkleize_with_limit/chunks/65536_of_262144 3.52 ms 2.53 ms (1.39x) 1.86 ms (1.89x) 13.28 ms 1.28 ms (10.39x)
control: diff/htr/beacon_state/lighthouse/1000000 842.09 ms 843.77 ms 843.83 ms 843.15 ms 854.76 ms
control: diff/htr/beacon_state/ssz_rs/1000000 960.19 ms 959.13 ms 964.57 ms 953.03 ms 954.99 ms
no hashing: merkle/pack/32768 1.59 µs 1.59 µs 1.60 µs 1.70 µs 1.59 µs

Keccak SIMD time divided by SHA-256 time (bold = Keccak is faster):

Keccak SIMD time ÷ SHA-256 main SHA-256 new SHA-256 + hashtree
merkle/hash_nodes/single 2.75 2.28 2.53
merkle/mix_in_length/single 2.15 2.27 2.52
hash_tree_root/consensus_containers/checkpoint 2.01 2.38 2.65
hash_tree_root/containers/validator 1.26 1.59 2.10
hash_tree_root/list_validator/10000 1.16 1.49 1.95
hash_tree_root/beacon_state/16384 0.90 1.16 1.51
hash_tree_root/beacon_state/300000 1.11 1.43 1.86
diff/htr/beacon_state/libssz/1000000 1.14 1.45 1.90
hash_tree_root/list_u64/100000 0.46 0.70 0.93
merkle/merkleize/4096 0.36 0.50 0.69
merkle/merkleize_with_limit/chunks/65536_of_262144 0.36 0.51 0.69

Takeaways

  • SHA-256 new vs main: 1.05–1.54x faster on every benchmark that hashes, apart from a lone hash_nodes (see caveats). About 1.3x on BeaconState and validators, up to 1.54x on large u64 lists, and least on small objects that only hash a few pairs (1.05x on unions/0). With hashtree the range is 1.16–2.04x, about 1.7x on BeaconState.
  • Keccak SIMD vs Keccak old: 1.36–10.4x faster. Objects that only hash a few pairs gain least, 1.36–1.43x (Checkpoint, Fork, Eth1Data, the smallest unions and bitfields, single pairs). BeaconState and validators gain 2.65–3.89x, and merkleize up to 10.4x.
  • Keccak vs SHA-256: SHA-256 still wins on consensus-shaped data. On BeaconState, Keccak SIMD is 1.16–1.45x slower than SHA-256 new and 1.51–1.90x slower than hashtree; on Validator and validator lists, 1.49–1.59x and 1.95–2.10x. Small containers are worse, up to 2.7x and 3.3x (Fork, BeaconBlockHeader).
  • Keccak wins only on wide trees, where every level fills the 8 lanes: 19 of the 57 benchmarks beat SHA-256 new and 15 beat hashtree, all of them merkleize, u64 and Bytes32 lists and vectors, variable_container or the 4096-bit bitvector. The best case is merkleize at 2x faster than SHA-256 new and 1.45x faster than hashtree.
  • Why BeaconState stays slower: most of its hashing happens inside small containers. A Validator hashes its pubkey (1 pair), then its fields in levels of 4, 2 and 1 pairs. With Keccak that is 4 permutations at about 145 ns each, even when the 8 lanes are mostly empty. SHA-256 does 8 hashes at about 50 ns each instead. Closing this gap would mean hashing many containers side by side in the SIMD lanes, which is a bigger rewrite.

Caveats

  • merkle/hash_nodes/single got slower on SHA-256 new (53 → 64 ns), and I haven't found why. It reproduced in three passes. A standalone probe on the same machine does not show it: the new pair hash takes 40 ns for independent hashes and 63 ns for chained ones, against 52–69 ns and 66–70 ns for sha2 0.10's Digest. The other one-hash benchmarks got faster (mix_in_length/single 67 → 64 ns, checkpoint 77 → 65 ns).
  • AVX2 Keccak is not benchmarked on a CPU without AVX-512. The only AVX2 numbers come from a timing probe on this Zen 5 with the AVX2 path forced: a full 4-pair batch went from about 94 to 77 ns per pair. Those timings moved when unrelated code changed, so treat them as rough.
  • The new code is not benchmarked on ARM. The 2-way SHA3 path is tested on an Apple M3, but the M3 numbers in the description are for the old versions.

CI on this PR

Build, tests, clippy, no_std and docs pass. Spec Tests and smoke / diff_htr_container fail as expected: every failure is a root mismatch against SHA-256 vectors or ssz_rs, not a panic. Cargo Audit fails on crypto-bigint (RUSTSEC-2026-0387), which also fails on main's scheduled runs.

All 91 benchmarks (x86_64)

The first three ratio columns are speedups (higher is faster). The last one is Keccak SIMD time divided by SHA-256 new time (below 1 means Keccak is faster). merkle/merkleize/1 doesn't hash, but it runs about 4x faster in the batched builds, so it no longer works as a control.

benchmark SHA-256 main SHA-256 new SHA-256 + hashtree Keccak old Keccak SIMD new vs main hashtree vs main Keccak SIMD vs old Keccak SIMD / SHA-256 new
diff/htr/beacon_state/libssz/100000 61.49 ms 47.72 ms 36.28 ms 209.03 ms 66.08 ms 1.29x 1.69x 3.16x 1.38x
diff/htr/beacon_state/libssz/1000000 569.14 ms 445.94 ms 340.35 ms 1.94 s 646.62 ms 1.28x 1.67x 3.00x 1.45x
diff/htr/beacon_state/libssz/16384 14.08 ms 10.80 ms 8.18 ms 48.87 ms 12.55 ms 1.30x 1.72x 3.89x 1.16x
diff/htr/beacon_state/libssz/300000 173.81 ms 136.30 ms 103.59 ms 592.59 ms 196.41 ms 1.28x 1.68x 3.02x 1.44x
diff/htr/beacon_state/lighthouse/100000 91.33 ms 91.22 ms 91.47 ms 91.55 ms 92.68 ms 1.00x 1.00x 0.99x 1.02x
diff/htr/beacon_state/lighthouse/1000000 842.09 ms 843.77 ms 843.83 ms 843.15 ms 854.76 ms 1.00x 1.00x 0.99x 1.01x
diff/htr/beacon_state/lighthouse/16384 21.66 ms 21.57 ms 21.67 ms 21.88 ms 21.91 ms 1.00x 1.00x 1.00x 1.02x
diff/htr/beacon_state/lighthouse/300000 259.06 ms 258.09 ms 258.11 ms 258.87 ms 262.03 ms 1.00x 1.00x 0.99x 1.02x
diff/htr/beacon_state/ssz_rs/100000 139.27 ms 141.35 ms 141.61 ms 138.22 ms 140.25 ms 0.99x 0.98x 0.99x 0.99x
diff/htr/beacon_state/ssz_rs/1000000 960.19 ms 959.13 ms 964.57 ms 953.03 ms 954.99 ms 1.00x 1.00x 1.00x 1.00x
diff/htr/beacon_state/ssz_rs/16384 63.78 ms 63.41 ms 63.23 ms 63.89 ms 63.20 ms 1.01x 1.01x 1.01x 1.00x
diff/htr/beacon_state/ssz_rs/300000 318.77 ms 321.28 ms 320.73 ms 316.85 ms 320.04 ms 0.99x 0.99x 0.99x 1.00x
diff/htr/consensus_containers/libssz/attestation_data 505.20 ns 411.88 ns 340.71 ns 1.67 µs 1.05 µs 1.23x 1.48x 1.59x 2.55x
diff/htr/consensus_containers/libssz/checkpoint 76.98 ns 65.57 ns 58.92 ns 220.08 ns 155.61 ns 1.17x 1.31x 1.41x 2.37x
diff/htr/consensus_containers/libssz/eth1_data 196.46 ns 167.63 ns 145.68 ns 624.88 ns 448.79 ns 1.17x 1.35x 1.39x 2.68x
diff/htr/consensus_containers/libssz/fork 197.25 ns 168.91 ns 145.71 ns 625.82 ns 447.71 ns 1.17x 1.35x 1.40x 2.65x
diff/htr/libssz/bool 2.26 ns 2.26 ns 2.26 ns 2.26 ns 2.25 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/libssz/bytes32 2.89 ns 2.88 ns 2.88 ns 2.88 ns 2.89 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/libssz/u64 2.44 ns 2.44 ns 2.44 ns 2.44 ns 2.44 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/lighthouse/bool 2.17 ns 2.17 ns 2.18 ns 2.17 ns 2.17 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/lighthouse/bytes32 2.88 ns 2.89 ns 2.88 ns 2.88 ns 2.89 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/lighthouse/u64 2.18 ns 2.17 ns 2.18 ns 2.17 ns 2.18 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/ssz_rs/bool 2.31 ns 2.32 ns 2.32 ns 2.31 ns 2.31 ns 1.00x 1.00x 1.00x 1.00x
diff/htr/ssz_rs/bytes32 57.31 ns 51.31 ns 51.00 ns 57.95 ns 57.00 ns 1.12x 1.12x 1.02x 1.11x
diff/htr/ssz_rs/u64 32.78 ns 32.88 ns 32.42 ns 32.76 ns 32.77 ns 1.00x 1.01x 1.00x 1.00x
hash_tree_root/beacon_state/100000 62.80 ms 47.65 ms 36.47 ms 209.96 ms 65.91 ms 1.32x 1.72x 3.19x 1.38x
hash_tree_root/beacon_state/16384 14.50 ms 11.20 ms 8.63 ms 49.42 ms 13.00 ms 1.29x 1.68x 3.80x 1.16x
hash_tree_root/beacon_state/300000 174.02 ms 135.91 ms 104.04 ms 591.53 ms 193.79 ms 1.28x 1.67x 3.05x 1.43x
hash_tree_root/bitfields/bitlist/2048 489.43 ns 397.90 ns 302.78 ns 1.65 µs 625.45 ns 1.23x 1.62x 2.64x 1.57x
hash_tree_root/bitfields/bitlist/4096 928.81 ns 720.37 ns 542.77 ns 3.29 µs 784.62 ns 1.29x 1.71x 4.19x 1.09x
hash_tree_root/bitfields/bitlist/64 83.72 ns 73.19 ns 67.35 ns 223.41 ns 161.05 ns 1.14x 1.24x 1.39x 2.20x
hash_tree_root/bitfields/bitvector/4096 864.59 ns 659.92 ns 488.63 ns 3.09 µs 639.33 ns 1.31x 1.77x 4.84x 0.97x
hash_tree_root/bitfields/bitvector/512 83.80 ns 74.06 ns 64.99 ns 225.66 ns 162.28 ns 1.13x 1.29x 1.39x 2.19x
hash_tree_root/bitfields/bitvector/64 22.35 ns 12.75 ns 12.98 ns 20.38 ns 12.87 ns 1.75x 1.72x 1.58x 1.01x
hash_tree_root/consensus_containers/attestation_data 509.21 ns 409.44 ns 339.26 ns 1.69 µs 1.05 µs 1.24x 1.50x 1.60x 2.57x
hash_tree_root/consensus_containers/checkpoint 77.21 ns 65.30 ns 58.56 ns 220.10 ns 155.31 ns 1.18x 1.32x 1.42x 2.38x
hash_tree_root/consensus_containers/eth1_data 197.99 ns 167.93 ns 146.59 ns 626.92 ns 446.98 ns 1.18x 1.35x 1.40x 2.66x
hash_tree_root/consensus_containers/fork 198.37 ns 168.90 ns 147.20 ns 627.33 ns 453.18 ns 1.17x 1.35x 1.38x 2.68x
hash_tree_root/consensus_containers/pending_attestation 1.19 µs 967.26 ns 762.28 ns 3.95 µs 1.97 µs 1.23x 1.56x 2.00x 2.04x
hash_tree_root/containers/beacon_block_header 356.47 ns 289.90 ns 227.09 ns 1.24 µs 742.67 ns 1.23x 1.57x 1.67x 2.56x
hash_tree_root/containers/validator 496.06 ns 393.15 ns 298.56 ns 1.66 µs 626.83 ns 1.26x 1.66x 2.65x 1.59x
hash_tree_root/list_u64/100 2.70 µs 2.15 µs 1.87 µs 8.58 µs 3.72 µs 1.25x 1.45x 2.31x 1.73x
hash_tree_root/list_u64/1000 15.31 µs 11.54 µs 8.89 µs 54.41 µs 8.27 µs 1.33x 1.72x 6.58x 0.72x
hash_tree_root/list_u64/100000 1.62 ms 1.05 ms 792.71 µs 5.37 ms 736.28 µs 1.54x 2.04x 7.30x 0.70x
hash_tree_root/list_validator/100 56.18 µs 43.93 µs 33.58 µs 190.51 µs 66.91 µs 1.28x 1.67x 2.85x 1.52x
hash_tree_root/list_validator/1000 552.33 µs 430.44 µs 327.21 µs 1.87 ms 644.77 µs 1.28x 1.69x 2.90x 1.50x
hash_tree_root/list_validator/10000 5.51 ms 4.29 ms 3.28 ms 18.72 ms 6.39 ms 1.28x 1.68x 2.93x 1.49x
hash_tree_root/nested_container/10 6.22 µs 4.91 µs 3.79 µs 20.92 µs 8.41 µs 1.27x 1.64x 2.49x 1.71x
hash_tree_root/nested_container/100 55.59 µs 43.81 µs 33.43 µs 189.40 µs 66.10 µs 1.27x 1.66x 2.87x 1.51x
hash_tree_root/nested_container/1000 548.98 µs 431.38 µs 328.62 µs 1.87 ms 638.50 µs 1.27x 1.67x 2.92x 1.48x
hash_tree_root/primitives/bool 2.27 ns 2.27 ns 2.26 ns 2.27 ns 2.27 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/bytes32 8.10 ns 8.11 ns 8.09 ns 8.10 ns 8.10 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/u128 2.38 ns 2.38 ns 2.38 ns 2.38 ns 2.38 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/u16 2.55 ns 2.55 ns 2.55 ns 2.55 ns 2.55 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/u32 2.44 ns 2.43 ns 2.43 ns 2.44 ns 2.43 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/u64 2.44 ns 2.44 ns 2.44 ns 2.44 ns 2.44 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/primitives/u8 2.52 ns 2.51 ns 2.51 ns 2.51 ns 2.51 ns 1.00x 1.00x 1.00x 1.00x
hash_tree_root/unions/0 67.58 ns 64.20 ns 57.42 ns 207.55 ns 152.20 ns 1.05x 1.18x 1.36x 2.37x
hash_tree_root/unions/1 67.24 ns 63.89 ns 57.75 ns 207.00 ns 151.74 ns 1.05x 1.16x 1.36x 2.38x
hash_tree_root/unions/2 424.92 ns 351.66 ns 283.64 ns 1.44 µs 891.45 ns 1.21x 1.50x 1.61x 2.53x
hash_tree_root/unions/3 777.97 ns 670.78 ns 579.00 ns 2.06 µs 971.74 ns 1.16x 1.34x 2.12x 1.45x
hash_tree_root/variable_container/100 2.37 µs 1.89 µs 1.57 µs 7.54 µs 2.66 µs 1.26x 1.51x 2.84x 1.41x
hash_tree_root/variable_container/1000 17.20 µs 13.13 µs 10.11 µs 59.96 µs 8.72 µs 1.31x 1.70x 6.88x 0.66x
hash_tree_root/variable_container/10000 164.65 µs 122.02 µs 93.35 µs 586.31 µs 70.14 µs 1.35x 1.76x 8.36x 0.57x
hash_tree_root/vec_u64/100 1.78 µs 1.37 µs 1.10 µs 5.93 µs 1.72 µs 1.30x 1.62x 3.46x 1.25x
hash_tree_root/vec_u64/1000 14.61 µs 10.88 µs 8.32 µs 52.21 µs 6.71 µs 1.34x 1.76x 7.78x 0.62x
hash_tree_root/vec_u64/100000 1.58 ms 1.05 ms 789.53 µs 5.31 ms 557.71 µs 1.50x 2.00x 9.52x 0.53x
hash_tree_root/vector_bytes32/100 5.97 µs 4.42 µs 3.41 µs 21.19 µs 3.49 µs 1.35x 1.75x 6.07x 0.79x
hash_tree_root/vector_bytes32/1000 56.60 µs 41.48 µs 31.22 µs 205.70 µs 22.58 µs 1.36x 1.81x 9.11x 0.54x
merkle/hash_nodes/single 53.23 ns 64.27 ns 57.81 ns 207.04 ns 146.51 ns 0.83x 0.92x 1.41x 2.28x
merkle/merkleize/1 12.64 ns 2.95 ns 3.12 ns 12.74 ns 3.04 ns 4.29x 4.05x 4.19x 1.03x
merkle/merkleize/1024 54.70 µs 39.50 µs 28.97 µs 206.95 µs 20.13 µs 1.38x 1.89x 10.28x 0.51x
merkle/merkleize/16 832.76 ns 624.71 ns 456.50 ns 3.05 µs 612.70 ns 1.33x 1.82x 4.98x 0.98x
merkle/merkleize/256 13.65 µs 9.94 µs 7.28 µs 51.57 µs 5.29 µs 1.37x 1.88x 9.75x 0.53x
merkle/merkleize/4 194.53 ns 155.60 ns 110.65 ns 620.99 ns 301.38 ns 1.25x 1.76x 2.06x 1.94x
merkle/merkleize/4096 218.85 µs 157.61 µs 115.82 µs 828.14 µs 79.38 µs 1.39x 1.89x 10.43x 0.50x
merkle/merkleize/64 3.42 µs 2.48 µs 1.81 µs 12.77 µs 1.53 µs 1.38x 1.89x 8.33x 0.62x
merkle/merkleize_with_limit/chunks/1024_of_1048576 55.39 µs 40.13 µs 29.47 µs 209.19 µs 21.60 µs 1.38x 1.88x 9.68x 0.54x
merkle/merkleize_with_limit/chunks/256_of_4096 13.92 µs 10.17 µs 7.48 µs 52.43 µs 5.87 µs 1.37x 1.86x 8.93x 0.58x
merkle/merkleize_with_limit/chunks/64_of_1024 3.68 µs 2.72 µs 2.02 µs 13.59 µs 2.13 µs 1.35x 1.83x 6.39x 0.78x
merkle/merkleize_with_limit/chunks/65536_of_262144 3.52 ms 2.53 ms 1.86 ms 13.28 ms 1.28 ms 1.39x 1.89x 10.39x 0.51x
merkle/merkleize_with_limit/chunks/65537_of_262144 3.52 ms 2.53 ms 1.86 ms 13.28 ms 1.27 ms 1.39x 1.89x 10.42x 0.50x
merkle/mix_in_length/single 67.26 ns 63.65 ns 57.37 ns 206.88 ns 144.51 ns 1.06x 1.17x 1.43x 2.27x
merkle/pack/1024 61.56 ns 61.43 ns 61.95 ns 61.85 ns 61.27 ns 1.00x 0.99x 1.01x 1.00x
merkle/pack/256 18.34 ns 18.30 ns 18.37 ns 18.34 ns 18.40 ns 1.00x 1.00x 1.00x 1.01x
merkle/pack/32 10.98 ns 10.96 ns 10.95 ns 10.94 ns 10.96 ns 1.00x 1.00x 1.00x 1.00x
merkle/pack/32768 1.59 µs 1.59 µs 1.60 µs 1.70 µs 1.59 µs 1.00x 1.00x 1.07x 1.00x
merkle/pack_bits/2048 18.07 ns 18.36 ns 18.12 ns 18.04 ns 18.50 ns 0.98x 1.00x 0.98x 1.01x
merkle/pack_bits/512 12.31 ns 12.33 ns 12.26 ns 12.11 ns 12.29 ns 1.00x 1.00x 0.99x 1.00x
merkle/pack_bits/64 11.33 ns 11.27 ns 11.28 ns 11.35 ns 11.22 ns 1.01x 1.00x 1.01x 1.00x
merkle/pack_bits/8192 61.82 ns 61.77 ns 61.71 ns 61.81 ns 61.74 ns 1.00x 1.00x 1.00x 1.00x

@MegaRedHand MegaRedHand mentioned this pull request Nov 12, 2026
Draft

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MegaRedHand