fix(node): one stale-nonce transaction could halt the chain - #16
Merged
Merged
Conversation
Stage stopped at block 84 on 2026-09-14 and stayed there. The cause was a single mint whose nonce the chain had already consumed, sitting in the transaction pool: Verification failed: Incorrect nonce for transaction from '0x662c5f11ed534ae93f29ed5e02a928081056f5f9'. Expected: 2, got: 1. author_new_block puts the whole pool into its candidate block, so that one transaction failed validation and took every block with it. And a pooled transaction is deleted only by the import of a block carrying it -- so it could never leave, and the chain could never produce again. Restarting the nodes did not help and could not: the poison is in the database. Only the provably-permanent case is evicted. An account's nonce never decreases, so nonce <= last can never satisfy nonce == last + 1 again. A nonce ABOVE the next one is a gap that the transaction filling it may still close, and those are left alone; a nonce that cannot be read is not evidence of anything and is also left alone. Two tests: a consumed nonce is evicted, a gap is kept. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage stopped at block 84 on 2026-09-14 and stayed there. The cause was a single mint whose nonce the chain had already consumed, sitting in the transaction pool:
author_new_blockputs the whole pool into its candidate block, so that one transaction failed validation and took every block with it. And a pooled transaction is deleted only by the import of a block carrying it — so it could never leave, and the chain could never produce again.Restarting the nodes did not help and could not: the poison is in the database, not in process state. That is why two redeploys changed nothing.
Only the provably-permanent case is evicted
An account's nonce never decreases, so
nonce <= lastcan never satisfynonce == last + 1again. A nonce above the next one is a gap that the transaction filling it may still close — those are left alone, or a busy pool would lose good work. A nonce that cannot be read is not evidence of anything and is also left alone.Two tests: a consumed nonce is evicted, a gap is kept.
Related
The stall was invisible until #15 — the authoring error is logged at
debug!, correctly for a per-tick failure, so three nodes sat at the same height for hours saying nothing. That change is what produced the message above.This fixes the deadlock. It does not explain how the treasury came to submit a transaction with an already-consumed nonce, which is worth its own look.
🤖 Generated with Claude Code