In rare occasions, when a block is locally built, the leader may receive the payload in 1.2-1.3s into slot start. This creates scenarios where the leader manages to propagate the payload only to a subset of peers in the 0.3-0.2s left.
That results in a segregation of the peer set. The ones that receive the payload increase their timeout by 1s, while the others do not receive enough round change messages.
We'd like to better monitor those scenarios and understand the blast radius of this issue. What we can do for a start is add a log and a metric when there are not enough round change messages, as usually that happens in the scenario explained above.
It would be nice if we do some research as well on how we can better mitigate the above issue.
In rare occasions, when a block is locally built, the leader may receive the payload in 1.2-1.3s into slot start. This creates scenarios where the leader manages to propagate the payload only to a subset of peers in the 0.3-0.2s left.
That results in a segregation of the peer set. The ones that receive the payload increase their timeout by 1s, while the others do not receive enough round change messages.
We'd like to better monitor those scenarios and understand the blast radius of this issue. What we can do for a start is add a log and a metric when there are not enough round change messages, as usually that happens in the scenario explained above.
It would be nice if we do some research as well on how we can better mitigate the above issue.