eth/consensus : implement eccpow consensus engine - #10
Open
mmingyeomm wants to merge 3663 commits into
Open
Conversation
…#35025) The changes here enable us to fill tests with Amsterdam using geth EVM bin. This will be useful for block builder tests using `testing_buildBlockV1` endpoint and for filling benchmarking compute and stateful tests as Python is too slow for benchmark tests. Tested in [ethereum/execution-specs](https://github.com/ethereum/execution-specs) with: ``` uv run fill --clean --fork=Amsterdam tests/amsterdam/eip7928_block_level_access_lists/test_block_access_lists.py --evm-bin=$GETH_EVM_PATH ```
Return empty raw bytes when the GraphQL `Block.raw` resolver cannot load the block body. This matches the nil handling used by the other block-body-backed resolvers and avoids exposing RLP empty-list bytes as raw block data.
…34951) Fixes the issue discovered in NethermindEth/nethermind#11412.
The first NODES response sets total = min(int(response.RespCount), totalNodesResponseLimit), With RespCount=0, total=0 but receive become 1; receive == count is never satisfied.
Passing `--state.size-tracking=false` currently cannot disable state size tracking when it was enabled by the config file because the CLI path only turns the config value on. --------- Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com>
Each commit on a PR kicks off a CI run. Those CI jobs run to the finish regardless, even when new commits have been pushed which make them stale and useless. This change attempts to cancel any previously running job for the same PR.
``` ● Global JUMPDEST Cache - engine_newPayload benchmark ============================================================ Commit before: a065580 (master) Commit after: faef245 (core/vm: global cache for jumpdest bitmaps) Blocks: 1k mainnet (24950066 → 24951065) Runs: 3 each, clean ZFS clone per run Before (avg) With Falcon (avg) Δ Throughput 176.0 MGas/s 190.7 MGas/s +8.3% Mean NP 172.3ms 159.0ms -7.7% p50 162.8ms 150.7ms -7.4% p95 282.4ms 259.8ms -8.0% p99 391.0ms 371.6ms -5.0% Machine: Intel Ultra 7 255H, 62GB DDR5, NVMe (ZFS), governor=performance, turbo=off ``` --------- Co-authored-by: Felix Lange <fjl@twurst.com>
…#33880) Archive nodes store the full history of transactions in the index. This PR fixes a bug for users who provided the NoPruning field in a YAML config file. Now geth correctly stores full transaction history if archive is configured via YAML. --------- Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
There is currently no way for JSON-RPC clients to discover which historical data a node can serve without probing with trial-and-error calls and interpreting opaque error messages (`pruned history unavailable`). This makes it hard to build robust tooling on top of nodes that prune their history, for example nodes started with `--history.chain postmerge` or with reduced `TransactionHistory`, `LogHistory`, or `StateHistory` windows. This PR implements `eth_capabilities` as defined in ethereum/execution-apis#755. The method takes no parameters and returns the current head plus six per-resource capability records: - `state` - `tx` - `logs` - `receipts` - `blocks` - `stateproofs` Closes #33828
Fixes #35033 ## Problem The GraphQL HTTP handler decoded request bodies directly before executing the query. Unlike the JSON-RPC HTTP path, `/graphql` did not have an explicit request body limit before JSON decoding. A single `Decode` also stops after the first JSON value, so the handler now requires EOF after the GraphQL request object to ensure oversized trailing request data is not ignored. ## Changes - Limit GraphQL request bodies to 5 MiB, matching the existing JSON-RPC default body limit. - Return `413 Request Entity Too Large` when the limit is exceeded. - Require EOF after the request JSON object. - Add regression coverage for oversized query bodies and oversized trailing request data. - Fix an existing GraphQL test fixture that had an unintended trailing quote after the JSON object. ## Validation - `gofmt -w graphql/service.go graphql/graphql_test.go` - `go run golang.org/x/tools/cmd/goimports@latest -w graphql/service.go graphql/graphql_test.go` - `go test ./graphql -run TestGraphQLHTTPBodyLimit -count=1` - `go test ./graphql -count=1`
…atus (#32673) fixes #32672 This is kind of a band aid solution since it fixes the issue by bypassing the snap sync expectations of an empty db and attempting to import the new payload if we're at block 1. The next FCU will set the status to synced. Will continue looking to better understand how the above issue arises and find a more thorough solution. --------- Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
### Summary Closes #34621. `github.com/pion/dtls/v2` is affected by [CVE-2026-26014](https://nvd.nist.gov/vuln/detail/CVE-2026-26014); the fix lives in `github.com/pion/dtls/v3`. In this tree, dtls/v2 is pulled in indirectly via `github.com/pion/stun/v2 v2.0.0` (declared at `go.mod:53`), which is the only direct consumer — `p2p/nat/stun.go` is the sole call site. `github.com/pion/stun/v3` already uses dtls/v3, so bumping `stun` upgrades the vulnerable dependency without touching `pion/dtls` directly. ### API check The v3 surface used by `p2p/nat/stun.go` is byte-identical in shape to v2: | Symbol | v2 | v3 | |---|---|---| | `Dial` | `func Dial(network, address string) (*Client, error)` | same | | `Build` | `func Build(setters ...Setter) (*Message, error)` | same | | `TransactionID` | `var TransactionID Setter` | same | | `BindingRequest` | `var BindingRequest = NewType(MethodBinding, ClassRequest)` | same | | `Event` | `type Event struct` | same | | `XORMappedAddress` | `type XORMappedAddress struct { … GetFrom(*Message) error }` | same | | `DefaultPort` | `const DefaultPort = 3478` | same | So the code change is just the import rename plus an alias rename to keep the local label honest (`stunV2` → `stunV3`). ### Change `go.mod` / `go.sum`: - Replace direct `github.com/pion/stun/v2 v2.0.0` with `github.com/pion/stun/v3 v3.0.1`. - `go mod tidy` drops every `pion/dtls/v2` and `pion/stun/v2` entry from `go.sum` and pulls `pion/dtls/v3 v3.0.7`, `pion/stun/v3 v3.0.1`, `pion/transport/v3 v3.0.8` as the new indirect set. `p2p/nat/stun.go`: - Update the import path and rename the alias from `stunV2` to `stunV3`. ### Verification - `go build ./p2p/nat/` clean. - `go test ./p2p/nat/ -count=1` passes (26s). - `grep 'pion/dtls/v2\|pion/stun/v2' go.sum` returns zero matches. ### Notes - `pion/dtls` is not imported directly anywhere in the tree, so no other code needs touching. - `pion/transport/v3` was already in the dependency graph (the `stun/v3` upgrade just bumps the patch from v3.0.1 → v3.0.8); the v2 transport drops out cleanly.
…an a year (#35021) This is another one of my slop-PRs, aimed at reducing the amount of future slop PRs by doing it all in one go. All of the deprecated cli flags have been in that state for over a year. It's time to remove them, especially since they are ineffective. Note that I kept the code to report and manage deprecated cli flags, as I assume we will be deprecating more flags in the future.
Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Propagate slog Handle failures when the underlying io.Writer rejects output.
Add a disableGzip parameter to NewHTTPHandlerStack and httpConfig. initAuth sets it true so compression is disabled in the engine api. Public HTTP RPC behavior is unchanged.
Track the transaction only after it was sent out
Updates dtls to newest version
The server already rejects empty batches with -32600. On the client side, calling BatchCallContext with a zero-length slice on inproc/WS/IPC transports registers no request IDs but the server still replies with an error message whose id is null. The dispatch loop has no requestOp to match it to, so op.resp is never written and op.wait blocks until ctx deadline. Short-circuit on len(b) == 0 with the same invalidRequestError the server uses, so all transports return immediately with -32600.
Because the UBT doesn't differentiate slots from accounts, the content of the tree can not be exported as a `GenesisAlloc`, which means that `evm t8n` can not intergrate it. We have tried integrating the new format into execution-specs, but this is very hard to maintain because the team doesn't see it as a priority and their own repository is seeing a lot of churn. This PR adds the ability to capture the structure of what is being inserted in the tree, so that the information isn't lost and it can be dumped in the t8n context. --------- Co-authored-by: felipe <fselmo2@gmail.com>
This PR implements the BAL downloader. Once the Amsterdam fork is enabled, BALs are scheduled for download for BAL-eligible blocks. Unlike mandatory components such as block bodies, BALs are optional and are downloaded on a best-effort basis. If a block's essential components are ready for delivery before its BAL has been retrieved, the block will be delivered without the BAL.
This PR updates some descriptions of snap sync v2, attaching two unit tests.
This PR introduces the functionalities to persist the local-built bad blocks alongside additional execution details.
…5501) `testing_buildBlockV1` decodes `targetGasLimit` from the payload attributes but does not pass it to `miner.BuildPayloadArgs`. The miner then falls back to its configured gas ceiling, so the field is silently ignored. The engine API path (`forkchoiceUpdated`) already passes it through. This matters for fixture generation in ethereum/execution-apis, where `testing_buildBlockV1` builds Amsterdam test blocks and the gas limit must honor the CL-provided target (see ethereum/execution-apis#857 and ethereum/execution-apis#862). The new test builds an Amsterdam block with a target inside the per-block adjustment bound and checks the payload hits it exactly.
…r cancellation (#35515) Close the started signal only after the fetchers are registered on cancelWg, so suspend's Cancel actually waits for them instead of racing the registration.
…35473) #35388 keys the cache on `keccak256(address, input)`. Hashing the input is most of the cost of a lookup, and it scales with input size, so a large input is expensive to look up even when the result is trivial. That is why several precompiles had to opt out to stay ahead of it. @ayamiyaguchi spotted this and wrote the first commit here, which excluded the precompiles the key cost was hurting most. This takes the same finding in a different direction by making the key cheap in the first place, so those precompiles do not need excluding. The solution here is to key on the input bytes directly, plus three changes that follow from it: 1) Entries used to share one LRU, so a high-volume precompile could evict the pairing and modexp results the cache exists to hold. Each precompile now gets its own. 2) Now that the key is the input, entries run from tens of bytes to kilobytes, and a count means very different memory depending on the mix. Each precompile gets 1 MiB of keys and values. 3) A precompile is cached only if it says so, and can narrow an input to the bytes that determine the result via `NormalizeInput` so padding lands on the entry the unpadded call already made. ### Benchmarks Medians of five runs. #### Lookup versus running the precompile, at real vector sizes | precompile | input | #35388 | this PR | run | |----------------------|--------|------------|------------|-----------| | ECREC | 128 | 480 ns | 55 ns | 18.2 µs | | SHA256 | 128 | not cached | 58 ns | 79 ns | | RIPEMD160 | 128 | not cached | 55 ns | 454 ns | | ID | 128 | not cached | not cached | 16 ns | | MODEXP | 609 | 1.1 µs | 128 ns | 2.9 µs | | BN254_ADD | 128 | 478 ns | 54 ns | 1.0 µs | | BN254_MUL | 96 | 279 ns | 52 ns | 13.4 µs | | BN254_PAIRING | 384 | 698 ns | 85 ns | 490.1 µs | | BLAKE2F | 213 | 486 ns | 67 ns | 118 ns | | KZG_POINT_EVALUATION | 192 | 488 ns | 61 ns | 785.8 µs | | BLS12_G1ADD | 256 | 697 ns | 68 ns | 1.9 µs | | BLS12_G1MSM | 160 | 488 ns | 60 ns | 95.4 µs | | BLS12_G2ADD | 512 | 911 ns | 103 ns | 2.9 µs | | BLS12_G2MSM | 288 | 708 ns | 74 ns | 181.3 µs | | BLS12_PAIRING_CHECK | 384 | 713 ns | 85 ns | 445.6 µs | | BLS12_MAP_FP_TO_G1 | 64 | 282 ns | 45 ns | 30.7 µs | | BLS12_MAP_FP2_TO_G2 | 128 | 490 ns | 55 ns | 135.0 µs | | P256VERIFY | 160 | 494 ns | 60 ns | 33.6 µs | RIPEMD160 wasn't cached before and a lookup now costs 55 ns against 454 ns to run it. BN254_ADD was cached previously, but at 478 ns to look up against a 1 µs run the cache was barely paying for itself. With this change, the lookup is now 54 ns and much more worth it. #### The same measurement at the largest eligible input, 8 KiB | precompile | #35388 | this PR | run | |----------------------|------------|------------------|-----------| | ECREC | 12.9 µs | 74 ns | 131 ns | | SHA256 | not cached | 1.1 µs | 2.6 µs | | RIPEMD160 | not cached | 1.2 µs | 17.9 µs | | ID | not cached | not cached | 969 ns | | MODEXP | 13.0 µs | 56 ns | 50 ns | | BN254_ADD | 12.9 µs | 67 ns | 60 ns | | BN254_MUL | 12.9 µs | 56 ns | 6.4 µs | | BN254_PAIRING | 12.8 µs | 5 ns, not cached | 2 ns | | BLAKE2F | 12.9 µs | 5 ns, not cached | 1 ns | | KZG_POINT_EVALUATION | 12.9 µs | 4 ns, not cached | 1 ns | | BLS12_G1ADD | 12.8 µs | 5 ns, not cached | 1 ns | | BLS12_G1MSM | 13.0 µs | 4 ns, not cached | 1 ns | | BLS12_G2ADD | 13.0 µs | 4 ns, not cached | 1 ns | | BLS12_G2MSM | 12.9 µs | 4 ns, not cached | 1 ns | | BLS12_PAIRING_CHECK | 13.0 µs | 5 ns, not cached | 1 ns | | BLS12_MAP_FP_TO_G1 | 13.0 µs | 4 ns, not cached | 1 ns | | BLS12_MAP_FP2_TO_G2 | 12.8 µs | 4 ns, not cached | 1 ns | | P256VERIFY | 13.0 µs | 4 ns, not cached | 1 ns | The `#35388` column is flat at ~13 µs because the key is a hash of the input and does not depend on which precompile receives it. In the `this PR` column, the number beside "not cached" is the cost of deciding not to build a key: those precompiles require an exact length, so an 8 KiB input is one they will reject, and there is no result worth keying. The `run` column here is mostly a length rejection rather than work, so it is context for the other two columns rather than a comparison. --------- Co-authored-by: aya <aya@ethereum.org> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
StateSize is a development tool for calculating and tracking the size of the state. It was requested by ethPandaOps a while ago, but it has since been superseded by another approach that captures state changes via a live tracer. Therefore, this tool can be retired to reduce maintenance overhead.
…ync target (#35433) Fixes #35418. After a restart, the syncer sets the finalized marker to head−64 and the safe marker to head−32 on every downloader sync event, ignoring epoch boundaries. Since this service is registered unconditionally and `api_backend` serves `CurrentFinalBlock()` verbatim, `eth_getBlockByNumber("finalized")` can return a mid-epoch block that was never finalized and can still be reorged. The synthesized markers exist to make `finalized`/`safe` usable when no consensus client is attached. This change scopes them to exactly that case: they are only set when an explicit sync target has been specified (`--synctarget`), which is the only mode where no CL supplies real checkpoints. Normal nodes now never override consensus-client checkpoints. `TestSyncerDoesNotInventFinalityMarkers` syncs a node to head with no CL finality input and asserts the chain's finalized/safe markers stay unset; it fails on master (finalized invented at head−64) and passes with this change. --------- Co-authored-by: SillyZir <269283839+SillyZir@users.noreply.github.com> Co-authored-by: Jonny Rhea <5555162+jrhea@users.noreply.github.com> Co-authored-by: rjl493456442 <garyrong0905@gmail.com>
See #35435 for more information
This PR explicitly clears the referenced block access list at the end of each transaction. This is essential because accounts are shadow-copied into the global block access list, and it prevents accidental mutations through stale references in StateDB.
`BlobPool.Init` builds the lookup, index and store without holding
`p.lock`, and `eth/backend.go` publishes the pool to `blobpool.NewCache`
before `txpool.New` runs Init. Cache.update spawns a goroutine calling
`getByVhash`, so it can read the pool mid-construction:
```
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x28]
blobpool.(*BlobPool).getByVhash(...) blobpool.go:1303
blobpool.(*Cache).update.func1() cache.go:429
```
`Init` fills the lookup inside `billy.Open`, whose index callback runs
`parseTransaction` -> `trackTransaction`, and assigns `p.store` only
once `billy.Open` returns. For that window a lookup hit names a
transaction whose store does not exist yet, and `p.store.Get`
dereferences nil.
Build the cache after `txpool.New` so no reader exists while `Init` runs
-- `getByVhash` has exactly one caller -- and have `getByVhash` read the
store under the same lock as the lookup, returning nil when it is not
set.
A benchmark harness that restarts the client once per test hit this on
1-4% of starts; a node that starts once rarely will.
…ocol version (#35524) eth/72 announces blob transactions without the blob payload while eth/71 includes it, so the same transaction is announced with different sizes. Compare each announcement against the size expected for that peer's version instead of a single size so that honest peers on either version are no longer dropped on delivery.
…get (#35526) #35473 keys the precompile cache on the input, but SizeConstrainedCache only counts the value’s bytes, so a cacheable empty output (e.g. a failed ECRECOVER) is never evicted and grows the cache without bound. Skipping empty outputs alone would still leave sha256/ripemd160 ~256x over budget, and would only hold as long as every failing output happens to be empty.
…on (#35528) When the full data domain (cell indices 0..DataPerBlob-1) is present, a blob is the concatenation of its data cells, so it can be reconstructed without any KZG work. Add BlobsFromDataCells, which returns the blobs by concatenation when the data cells are present in canonical order and declines otherwise (so callers fall back to RecoverBlobs). Its accepted inputs are a strict subset of those accepted by RecoverBlobs, so a successful result is byte-identical; being pure byte copying it is independent of the selected KZG backend. Shared primitive: used both to skip KZG recovery when serving blobs and by the cell-recovery path (RecoverCells).
A few updates for glamsterdam devnet, taking the original commits from @spencer-tb - making the TargetGasLimit as the optional parameters - ensuring the system call's code exists (EIP-7002/7251/8282) - rework the fork selection rules in the engine API --------- Co-authored-by: spencer-tb <spencer.tb@ethereum.org>
Add RecoverCells, which returns all CellsPerBlob cells per blob from a sufficient subset. RecoverBlobs only exposes recovered blobs; serving or persisting the extension cells of a sparse-blobpool transaction needs the full 128-cell set. Add RecoverCells, which returns all CellsPerBlob cells per blob from a sufficient subset. When the full data domain (cell indices 0..DataPerBlob-1) is present (the common case for pooled transactions) the blobs are a free concatenation of the data cells and every cell follows from a systematic extension via ComputeCells (~2-6ms/blob), skipping the KZG erasure solve. Otherwise it falls back to the erasure recovery path (~15-20ms/blob), which now surfaces the library's full extended cell set instead of discarding it down to blobs as RecoverBlobs does. Both paths return byte-identical cells in canonical order. Cells only; cell proofs are never recomputed (callers retain the proofs shipped with the transaction). Tested on both the gokzg and ckzg backends, fast and slow paths, against the original cells. Note: the same fast-path optimization could also be pushed down to gokzg and ckzg. There are both arguments for (the primitive becomes faster in some cases, same for the rest of cases) and against (it is not the role of the crypto lib to be intelligent here), so I went with the wrapper for now.
…35542) The test set the local UDP port via ln.Set(enr.UDP(...)), but the "udp" ENR entry is owned by the endpoint predictor: as soon as the first node received a PONG, updateEndpoints recomputed the endpoint with no fallback port set and deleted the entry. Later nodes bootstrapping from that node's record then failed with "bad bootstrap node: missing UDP port". Use SetFallbackUDP like startLocalhostV4 already does, so the port survives endpoint statements. fixes CI errors like in https://github.com/ethereum/go-ethereum/actions/runs/31859911877/job/94953524792?pr=35529
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network