Skip to content

net: Disallow invalid HeadersSyncState due to lagging clock - #35351

Open
hodlinator wants to merge 2 commits into
bitcoin:masterfrom
hodlinator:pr/35208_alt
Open

net: Disallow invalid HeadersSyncState due to lagging clock#35351
hodlinator wants to merge 2 commits into
bitcoin:masterfrom
hodlinator:pr/35208_alt

Conversation

@hodlinator

@hodlinator hodlinator commented May 21, 2026

Copy link
Copy Markdown
Contributor

Problem

Headers presync computes m_max_commitments from the elapsed time since the chain-start MTP plus MAX_FUTURE_BLOCK_TIME. When the local system clock is more than MAX_FUTURE_BLOCK_TIME behind the chain-start MTP, that elapsed value is negative, but it is used in arithmetic assigned to the unsigned commitment cap. This can turn the intended zero bound into a large cap, letting low-work headers presync continue instead of aborting when a reasonable commitment cap would have been exceeded.

Fix

Instead of allowing an invalid HeadersSyncState object to be created, emit an error and abort the node process.

Typically, the node will detect that the system clock is set too far in the past when comparing it to the chain tip during chain state loading and shut down before we start syncing headers. So in practice this is very unlikely to make a difference (might be possible if the system clock jumps backwards after we loaded the chain state).

Commits


Replaces #35208 which was clamping m_max_commitments to zero and then letting the HeadersSyncState consume headers until the block height either reached the the next commitment_period point and aborted, or reached the minimum work threshold and succeeded (possible when having been offline for >144 blocks).

@DrahtBot DrahtBot added the P2P label May 21, 2026
@DrahtBot

DrahtBot commented May 21, 2026

Copy link
Copy Markdown
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/35351.

Reviews

See the guideline and AI policy for information on the review process.

Type Reviewers
ACK l0rinc, sedited
Concept ACK w0xlt, dergoegge

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #36087 (util: Add and use AssertUnreachable by maflcko)
  • #36074 (scripted-diff: [test] replace assert with Assert by maflcko)
  • #35820 (refactor: keep duration calculations typed by l0rinc)
  • #35642 (headersync: do parameter search at runtime by sipa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

LLM Linter (✨ experimental)

Possible places where comparison-specific test macros should replace generic comparisons:

  • [src/test/headers_sync_chainwork_tests.cpp] BOOST_CHECK_THROW(CreateState(), HeadersSyncState::SystemClockError); -> consider BOOST_CHECK_EXCEPTION with a predicate that verifies the error message if the exact failure text matters, rather than checking only that this exception type was thrown.

2026-08-19 11:04:18

@w0xlt

w0xlt commented May 22, 2026

Copy link
Copy Markdown
Contributor

Concept ACK

@hodlinator

Copy link
Copy Markdown
Contributor Author

Been thinking more about this during the weekend.

One could posit that Bitcoin nodes should be more shelf stable in a post-apocalyptic future where NTP servers are unreachable. Re-establishing a shared accurate definition of current UTC time just from observing the sun's position could be challenging.

In that kind of scenario, preventing the node from continuing, as this PR does, could be problematic.

So should we do like #35208 and just set m_max_commitments to 0 instead? It would still be pretty unforgiving if start-height is very close to a multiple of the commitment_period - maybe allowing at least 1 commitment could be argued? (commitment_period is currently 641, giving an average of 961 blocks before aborting if we start at a random height).

Although, if clocks in the network disagree too much about UTC time, block propagation would also be suffering due to the rule to not accept blocks from too far into the future (MAX_FUTURE_BLOCK_TIME/2h). The current check failing would also mean that the we somehow ended up with a starting block read from disk that is dated more than 2h in the future, meaning our clock is not just out of sync with UTC but has also jumped backwards since we accepted the starting block. So I think the current approach in the PR is consistent with the rest of the node.

@l0rinc l0rinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is going in the right direction, seems weird to discover time drift in the constructor and catch on use site - instead of sanitizing the value before call.
Please see https://github.com/l0rinc/bitcoin/pull/186/commits for how I imagined simplifying this a lot more.

Comment thread src/headerssync.cpp Outdated
Comment thread src/test/fuzz/headerssync.cpp
Comment thread src/headerssync.h Outdated
Comment thread src/test/fuzz/headerssync.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp
Comment thread src/net_processing.cpp Outdated

@hodlinator hodlinator left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your review & suggestions @l0rinc!
Pushed the straightforward ones for now.

I'm not sure this is going in the right direction, seems weird to discover time drift in the constructor and catch on use site - instead of sanitizing the value before call.

Curious what you think about sending unsigned max_commitments into HeadersSyncState() (see inline comment).

Comment thread src/test/fuzz/headerssync.cpp
Comment thread src/headerssync.cpp Outdated
Comment thread src/net_processing.cpp Outdated
Comment thread src/test/fuzz/headerssync.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp
@hodlinator

Copy link
Copy Markdown
Contributor Author

(Rebased against master in latest push to resolve conflicts).

Comment thread src/test/fuzz/headerssync.cpp
Comment thread src/net_processing.cpp Outdated

@l0rinc l0rinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, please see my remaining suggestion to reduce side-effects and simplify testing

Comment thread src/headerssync.cpp Outdated
Comment thread src/net_processing.cpp Outdated
Comment thread src/headerssync.h Outdated

@hodlinator hodlinator left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Latest few pushes after 1c3ad92d0bdcc66f252f647deb58f8becfce2772 attempt to reconcile differing viewpoints which me & @l0rinc partially discussed out-of-band last week).

Comment thread src/headerssync.h Outdated
Comment thread src/net_processing.cpp Outdated
Comment thread src/headerssync.cpp Outdated
@l0rinc

l0rinc commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

ACK 8b680b2

@optout21, @vasild, @dergoegge, this is an alternative to the original acked PR, your re-review here would be welcome.

@dergoegge

Copy link
Copy Markdown
Member

Concept ACK

I thought the original PR was also fine. Code here also looks fine, but I won't prioritize giving this a full review.

@l0rinc

l0rinc commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@hodlinator, can you please rebase the change?

@l0rinc l0rinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 70ec860

The approach looks correct: reject the invalid elapsed-time calculation before constructing HeadersSyncState, then trigger fatal shutdown.

I left a few non-blocking suggestions and some prose is also stale:

  • The PR description still says the constructor throws an exception.
  • The first commit message says commitment_period=1, although the test uses COMMITMENT_PERIOD (600), and says the fix flips the test although it replaces it with direct boundary checks.
  • The final commit subject says "local tip", but the comparison uses the chain-start header MTP, which is not necessarily the tip. Its body could also briefly explain why this condition warrants shutdown.

Comment thread src/test/fuzz/headerssync.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp Outdated
Comment thread src/net_processing.cpp

@hodlinator hodlinator left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your latest review @l0rinc, incorporated all of it in the latest push, with one exception:
You disagree that the chain start header is the same as the local tip - when would that not be the case?

Beyond your feedback I also added/corrected some comments in net_processing.cpp in the last commit.

Comment thread src/test/fuzz/headerssync.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp Outdated
Comment thread src/net_processing.cpp

@l0rinc l0rinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 0799f3b

This correctly rejects a negative elapsed interval before it can become a large unsigned presync commitment cap, prevents constructing an invalid HeadersSyncState, and routes the unexpected clock condition through fatal shutdown. The unit test covers the exact boundary, while the functional test covers the P2P-to-shutdown path. The focused unit and functional tests pass.

I left some suggestions to make the review easier by separating risky changes from pure refactors (which also help with making the characterization test changes minimal) - I applied these suggestions in https://github.com/l0rinc/bitcoin/pull/276/commits during local review.
I'm also okay with merging this as is and happy to rereview if any of the suggestion are taken.

You disagree that the chain start header is the same as the local tip - when would that not be the case?

My understanding is that the relevant presync path receives a peer-supplied full headers message, then looks up chain_start_header from its first header.
As far as I can tell it may be any known block (we only require valid PoW, continuity, and a connection to our block index), for example an older active-chain block or a stale-fork block. I’d call it the chain-start header rather than the local tip.

Comment thread test/functional/p2p_headers_sync_with_minchainwork.py
Comment thread src/test/headers_sync_chainwork_tests.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp
Comment thread src/test/headers_sync_chainwork_tests.cpp Outdated
Comment thread src/test/headers_sync_chainwork_tests.cpp Outdated
@hodlinator

Copy link
Copy Markdown
Contributor Author

Thanks for the solid feedback again @l0rinc, agree it should make it easier to review. Took the changes from your branch with some variation, hope it's acceptable to you. The largest deviation from your l0rinc#276 end state is that I decided to const m_max_commitments.

@l0rinc l0rinc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 1695d30

The commits are very easy to understand now, thanks for considering my suggestions! The remaining nits aren't important.

Comment thread src/test/fuzz/headerssync.cpp Outdated
/*chain_start=*/start_index,
/*minimum_required_work=*/min_work);
/*minimum_required_work=*/min_work,
/*max_commitments=*/*max_commitments);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8593ec4 refactor(p2p): Pass max commitments into HeadersSyncState():

nit, if you need to touch again:

Suggested change
/*max_commitments=*/*max_commitments);
*max_commitments);

Comment thread src/net_processing.cpp Outdated
@sedited

sedited commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Looked at the patch set a few days ago and it looks correct and fine to me, but I'm not completely sure yet of the approach. Could it be better to just crash immediately with an abort if the system clock is that far out of sync, akin to maflcko's patch here: https://github.com/bitcoin/bitcoin/pull/35676/changes#diff-c24fd5f39e56e513a81d84b494862608c1620c194e5a7efbded93953b6ef43e2R106 ? Externalizing the max commitments calculation does feel like bit of a code smell to me.

@mzumsande mzumsande left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked at the patch set a few days ago and it looks correct and fine to me, but I'm not completely sure yet of the approach. Could it be better to just crash immediately with an abort if the system clock is that far out of sync, akin to maflcko's patch here: https://github.com/bitcoin/bitcoin/pull/35676/changes#diff-c24fd5f39e56e513a81d84b494862608c1620c194e5a7efbded93953b6ef43e2R106 ?

with "immediately", do you mean with that to check the system time vs best header in init and crash there without ever starting up network operations?

Externalizing the max commitments calculation does feel like bit of a code smell to me.

Another option could be to only do the time check with chain_start_header.GetMedianTimePast() in TryLowWorkHeadersSync(), and leave the max commitments calculation inside HeadersSync- so that we can change the internal check to an Assume(max_seconds_since_start >= 0);. Having to repeat the time check seems irrelevant performance-wise.

The node currently continues low-work headers presync and requests more headers when its clock is more than `MAX_FUTURE_BLOCK_TIME` behind the chain-start MTP.

Record this behavior before the follow-up rejects the invalid elapsed-time calculation.

The unit test covers HeadersSyncState() behavior while the functional test covers net_processing.cpp behavior.

Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
@hodlinator

Copy link
Copy Markdown
Contributor Author

Latest push (4cf06b1) switches back to std::abort() and tries to handle the code smell1 of externalizing the max_commitments calculation in the previous push (1695d30).

It reverts back to throwing an exception from HeaderSyncState(). This avoids duplicating the error condition logic inside TryLowWorkHeadersSync()2. Using exceptions was previously pushed back against3, but it is how C++ constructors are supposed to report errors and keeps the unit test clean.

Footnotes

  1. https://github.com/bitcoin/bitcoin/pull/35351#issuecomment-5293052515

  2. https://github.com/bitcoin/bitcoin/pull/35351#pullrequestreview-4953169514

  3. https://github.com/bitcoin/bitcoin/pull/35351#discussion_r3367476303

We should not proceed syncing headers from peers when the local system clock is incorrectly set.

A node with a system clock set too far back will typically fail early during startup when the chainstate detects the tip to be too far in the future. This means that in practice we don't expect the failure to ever happen in net_processing.cpp.

An exception is thrown from HeadersSyncState() in order to only compute the error condition once. An alternative would be to compute it a second time in TryLowWorkHeadersSync() to guard against calling HeadersSyncState(), and have an assert inside HeadersSyncState(). We shut down the process so possible resource leaks due to the exception should not be an issue, although none have been spotted. Throwing an exception also keeps the unit test straightforward.

Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
@l0rinc

l0rinc commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

diff and code review ACK ff3e2e4

nit: the PR description still describes the old graceful-shutdown

@sedited sedited left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK ff3e2e4

@sedited
sedited requested a review from mzumsande August 29, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants