Skip to content

[Security] History catchup spawns concurrent fetch_history_events tasks before Syncing is set, racing the Gateway event cursor (state desync) #466

Description

@byfor8

Summary

The history catchup spawner starts concurrent fetch_history_events tasks before the Syncing status is set. The watcher ticks every 5 seconds, and until the first in-task DB commit every tick passes the respawn gate and spawns another full history loop from the same cursor snapshot. Concurrent fetchers race upsert_watch_contract on overlapping height ranges, so the Gateway event cursor can regress or replay and event ranges are skipped or double-processed.

Security impact: incorrect state transition (event cursor desync vs chain reality) plus denial of service on protocol maintenance scheduling. No fund theft path identified.

Affected component

  • Repository: GOATNetwork/bitvm-node
  • Commit reviewed: 1ff9a4aeced45309a097eeba5955e26164302ab3 (current HEAD at time of review)
  • Files: node/src/scheduled_tasks/event_watch_task.rs, node/src/env.rs, node/src/main.rs
  • Component class: Implementation (event watcher, Gateway catchup)

Description

monitor_events_item (node/src/scheduled_tasks/event_watch_task.rs:1249-1275) gates a respawn only when status is already Syncing AND updated_at is within LOAD_HISTORY_EVENT_NO_WOKING_MAX_SECS = 600 seconds (node/src/env.rs:145). But Syncing is written only inside fetch_history_events, after the first successful block range commits (event_watch_task.rs:1146-1158). The watcher task runs on a 5 second interval (node/src/main.rs:326-332). Until that first write, every tick sees status UnSync, passes the gate, and tokio::spawns another full history loop from the same cursor snapshot. There is no JoinHandle, mutex, or CAS around the spawn.

Two or more concurrent fetchers then race upsert_watch_contract on overlapping height ranges; a slower fetcher can commit an older from_height after a faster one already advanced the cursor, so ranges are skipped or replayed. Replayed InitWithdraw and BridgeInRequest events re-upsert goat_tx_records where the sticky Processed merge (localdb.rs:3569-3583) then fights the replay. is_processing_gateway_history_events also keys off Syncing, so overlapping fetchers keep protocol maintenance deferred for an unpredictable window.

No 600 second stall is required for the race; the spawn overlap alone is sufficient.

How it can be exploited (no attacker action needed)

Triggers on any node doing history catchup: fresh node, or a node coming back after downtime, with from_height + gap below the finalized tip. The 5 second tick cadence versus the subgraph fetch duration makes overlapping spawns the common case, not an edge case.

Steps to reproduce

  1. Start a node with a watch contract whose from_height + gap is below the finalized tip (fresh node or after downtime).
  2. Observe run_watch_event_task ticking at 5 second intervals (main.rs:326-332) while the first fetch_history_events run is still before its first DB commit: multiple tokio::spawn calls occur from monitor_events_item (event_watch_task.rs:1249-1275).
  3. Inspect the watch_contract table: multiple concurrent loops upsert conflicting from_height values; a later writer can rewind the cursor after a faster one advanced it, skipping or replaying Gateway event ranges.

Proof of concept (source level)

No PoC transaction was performed and no live system was contacted; source-only review at 1ff9a4a:

// event_watch_task.rs:1249-1261, the gate and the unguarded spawn
    if watch_contract.status == WatchContractStatus::Syncing.to_string()
        && watch_contract.updated_at + LOAD_HISTORY_EVENT_NO_WOKING_MAX_SECS > current_time_secs()
    {
        info!("Event sync not finished ");
        return Ok(());
    }
    if watch_contract.from_height + watch_contract.gap < current_finalized {
        ...
        tokio::spawn(async move {
            if let Err(error) = fetch_history_events(
// event_watch_task.rs:1146-1158, Syncing written only after the first batch commits
            watch_contract.from_height = to_height + 1;
            watch_contract.status = WatchContractStatus::Syncing.to_string();
            watch_contract.updated_at = current_time_secs();
// main.rs:326-332, the 5 second tick cadence
            match run_watch_event_task(
                actor_clone2,
                local_db_clone2,
                btc_client,
                goat_client,
                5,
                cancel_token_clone,
// env.rs:145
pub const LOAD_HISTORY_EVENT_NO_WOKING_MAX_SECS: i64 = 600;

Provenance

This is the second adjacent defect explicitly left open in the repository's own TLA+ audit (audit/TLAPlus-20260710.md, Finding 8 followups). No existing issue or PR covers the spawn-before-Syncing race (checked all 41 issues and 420 PRs as of 2026-09-14).

Security impact

The Gateway event cursor can regress or replay, leaving the local state machine behind chain reality (missed withdraw, bridge-in, and take-path events) until operator repair; maintenance deferral windows become unpredictable. Freeze / liveness class, not theft.

Source only review; no PoC transactions were performed; no live system was contacted.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions