fix(block-producer): poison shared mempool lock#2029
Open
namedfarouk wants to merge 4 commits into0xMiden:nextfrom
Open
fix(block-producer): poison shared mempool lock#2029namedfarouk wants to merge 4 commits into0xMiden:nextfrom
namedfarouk wants to merge 4 commits into0xMiden:nextfrom
Conversation
namedfarouk
force-pushed
the
fix/2016-poison-mempool-lock
branch
from
87c618c to
f8282a2
Compare
namedfarouk
force-pushed
the
fix/2016-poison-mempool-lock
branch
from
f8282a2 to
08b87b8
Compare
Comment on lines
+155
to
+159
| pub fn lock(&self) -> MutexGuard<'_, Mempool> { | ||
| self.0.lock().unwrap_or_else(|err| { | ||
| tracing::error!(message = %err, "Mempool lock poisoned"); | ||
| std::process::abort(); | ||
| }) |
Collaborator
There was a problem hiding this comment.
This should return a poison result. The callers should then decide what to do with that information.
- submit transaction/batch should convert that into an internal error to return to the user
- build batch/block should error out, which should reach the top-level binary and be handled appropriately already
Mirko-von-Leipzig
requested changes
May 4, 2026
Mirko-von-Leipzig
requested changes
May 4, 2026
Comment on lines
+155
to
+159
| pub fn lock(&self) -> MutexGuard<'_, Mempool> { | ||
| self.0.lock().unwrap_or_else(|err| { | ||
| tracing::error!(message = %err, "Mempool lock poisoned"); | ||
| std::process::abort(); | ||
| }) |
Collaborator
There was a problem hiding this comment.
This should return a poison result. The callers should then decide what to do with that information.
- submit transaction/batch should convert that into an internal error to return to the user
- build batch/block should error out, which should reach the top-level binary and be handled appropriately already
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
std::sync::MutexRoot cause
SharedMempoolpreviously wrapped the mempool intokio::sync::Mutex, which does not poison after a panic. The first version of this fix switched to a poisonable mutex, but still surfaced poison as a panic. In the block producer that is not strong enough because some paths can catch panics or run in detached tasks.Impact
Once a mempool operation panics while holding the lock, any later mempool access now aborts the block-producer process immediately instead of continuing with potentially corrupted in-memory state or converting the failure into a recoverable gRPC error.
Closes #2016.
Testing
cargo test -p miden-node-block-producer mempool::tests::shared_mempool_lock_aborts_after_poisoning -- --exact --nocapturecargo test -p miden-node-block-producer mempool::tests::shared_mempool_lock_aborts_after_poisoning_helper -- --exact --nocapturecargo test -p miden-node-block-producer -- --skip add_transaction_traces_are_correct