Skip to content

fix: prevent DualStore from clobbering VSS backups and mask-failing restores - #3

Merged
kaloudis merged 1 commit into
zeusfrom
fix/dual-store-vss-clobber
Jul 30, 2026
Merged

kaloudis merged 1 commit into
zeusfrom
fix/dual-store-vss-clobber

Conversation

@kaloudis

@kaloudis kaloudis commented Jul 30, 2026

Copy link
Copy Markdown

Background

ZEUS users reported restore-from-seed coming up with no channels. Investigation traced it to the app-side VSS build timeout silently falling back to an empty local store (fixed separately in the ZEUS app), which exposed two independent defects in DualStore:

Problem 1: bulk sync clobbers VSS backups

DualStore::new unconditionally spawns bulk_sync_to_vss, which pushes every local key to VSS using unconditional puts (version: -1 skips the server's conflict check). If the local store contains a fresh node built over an existing backup — e.g. a restore that fell back to an empty local store — the sync overwrites the real manager key in VSS within seconds of the next launch. No user activity is required. The old channel monitors survive only because the fresh node has no monitor keys and the sync never deletes; recovery is then limited to orphan-monitor force-closes.

Problem 2: restore mode masks VSS failures as success

In restore mode, read() collapsed any VSS error into NotFound and list() into an empty namespace. A fast-failing VSS error — auth rejection from device clock skew (InvalidRequestError skips retries entirely), DNS failure — therefore made the node build fresh with no channels while reporting success, indistinguishable from a real restore at the API surface.

Changes

  • Push safety gate (vss_push_allowed): all VSS pushes (bulk sync, per-key writes, removes) first verify the poison signature is absent: local has no channel monitors (active or archived) while VSS holds at least one. On detection, VSS writes are disabled for the session and a CRITICAL message is logged directing the user to re-restore. If VSS is unreachable during the check, the verdict stays undetermined: the current push is skipped (harmless — local retains the data and a later bulk sync catches up) and the check reruns on the next push. Healthy nodes short-circuit on the local monitor listing with no network call; restore-mode sessions pre-arm the gate open since their state derives from VSS.
  • Restore-mode error propagation: non-NotFound VSS errors in read()/list() now propagate, failing the build so callers surface the error instead of silently producing a fresh node.
  • Bulk sync skipped in restore mode: local started empty this session; there is nothing to catch up.

Notes

  • Protects already-poisoned devices in the field the moment they run a build with this change, before their next launch clobbers the backup.
  • Companion ZEUS app PR (restore hard-fail + startup handling) to follow; app changes are independent and safe to ship first.
  • cargo check clean. Rebuilding LDKNodeFFI binaries + ZEUS fetch-libraries.sh checksum bump required to ship.

Known limitations of the push gate

  • Defeated by new channel activity on a poisoned device: if a pre-fix victim opens a new channel from the fresh node, the local monitor listing becomes non-empty and pushes resume, overwriting the backed-up manager. Closing that hole would require content-level reconciliation; the gate protects the common case of an idle poisoned device.
  • A confirmed-unsafe session runs with no VSS backup at all, including for new channels opened during it. The CRITICAL log's recovery instruction (restore from seed into a new wallet) is the intended path; a poisoned wallet should not be operated long-term.

…estores

Two related fixes to the dual-store (VSS + local SQLite) backend:

1. Gate all VSS pushes on a safety check. The background bulk sync ran
   unconditionally at construction and pushed every local key to VSS
   with unconditional puts (version -1). If the local store held a
   fresh node built over an existing backup (e.g. a restore that fell
   back to an empty local store), the sync overwrote the real channel
   manager in VSS within seconds. All pushes (bulk sync, writes,
   removes) now first verify the poison signature is absent: local has
   no channel monitors (active or archived) while VSS holds at least
   one. On detection, VSS writes are disabled for the session and a
   CRITICAL message is logged. If VSS is unreachable during the check,
   the verdict stays undetermined and the push is skipped; the next
   push re-checks.

2. Propagate VSS errors during restore instead of masking them. In
   restore mode, read() collapsed any VSS failure into NotFound and
   list() into an empty namespace, so a fast-failing VSS error (auth
   rejection from clock skew, DNS failure) made the node come up fresh
   with no channels while reporting a successful build. Non-NotFound
   errors now fail the read/list — and thereby the build — so callers
   can surface the failure and retry.

Also skips the bulk sync entirely in restore mode: local started empty
this session, so there is nothing to catch up.
@kaloudis
kaloudis force-pushed the fix/dual-store-vss-clobber branch from ec6d976 to 7ea48de Compare July 30, 2026 01:05
@kaloudis
kaloudis merged commit db3f8bc into zeus Jul 30, 2026
4 of 36 checks passed
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
Pulls in ZeusLN/ldk-node#3: DualStore push-safety gate (prevents a
fresh local store from overwriting an existing VSS backup) and
restore-mode VSS error propagation (a failed VSS fetch now fails the
build instead of silently producing a fresh node with no channels).

Also includes the additive BOLT11 route-hints receive APIs from
ZeusLN/ldk-node#2; the uniffi interface change is purely additive and
the uniffi version is unchanged, so the vendored bindings remain
compatible without regeneration.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
…ding an empty wallet

Users on poor connections reported restore-from-seed coming up with no
channels. Root cause: the native VSS dual-store build had a 30/60s
timeout and on timeout or failure silently fell back to building a
fresh node with an empty local SQLite store. The restore flows ignored
the returned vssError, so the wallet config was saved and the restore
looked successful with zero channels. Worse, the fallback local DB
shadows VSS on all later dual-store builds (local-first reads), so
channels never appeared even after connectivity recovered — and the
fork's background bulk sync would then overwrite the VSS backup with
the fresh state (fixed separately in ZeusLN/ldk-node#3).

Changes:

- Add setVssFailOnError to the native LDK Node modules (Android + iOS).
  When set, a VSS build timeout or failure rejects with code vss_error
  instead of falling back to an empty local store. Reset with each
  createBuilder, so existing-node startups keep fallback behavior.
- Thread failOnVssError through initializeNode/initNode/
  createLdkNodeWallet. Hard-fail builds get a 180s timeout (restore
  time scales with node history and connection quality; failing now
  produces a visible error rather than a silently wrong wallet).
- createLdkNodeWallet deletes the just-created node dir on failed init
  so a partially-written local DB cannot shadow VSS state on retry.
- SeedRecovery passes failOnVssError: true and shows a localized error
  (new key views.Settings.SeedRecovery.vssRestoreFailed) so the user
  knows channels were NOT restored and can retry.
- WalletConfiguration passes failOnVssError when an existing seed is
  imported; new-wallet creation keeps the non-fatal warning path.
- startLdkNodeWallet treats a startup with a saved config but no local
  DB (e.g. config restored to a new device via cloud keychain sync) as
  a de-facto restore and hard-fails on VSS errors. Wallet.tsx catches
  the failure, surfaces it via SettingsStore.error/errorMsg, and
  releases fetchLock so the user can retry without restarting the app.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
Pulls in ZeusLN/ldk-node#3: DualStore push-safety gate (prevents a
fresh local store from overwriting an existing VSS backup) and
restore-mode VSS error propagation (a failed VSS fetch now fails the
build instead of silently producing a fresh node with no channels).

Also includes the additive BOLT11 route-hints receive APIs from
ZeusLN/ldk-node#2; the uniffi interface change is purely additive and
the uniffi version is unchanged, so the vendored bindings remain
compatible without regeneration.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
…ding an empty wallet

Users on poor connections reported restore-from-seed coming up with no
channels. Root cause: the native VSS dual-store build had a 30/60s
timeout and on timeout or failure silently fell back to building a
fresh node with an empty local SQLite store. The restore flows ignored
the returned vssError, so the wallet config was saved and the restore
looked successful with zero channels. Worse, the fallback local DB
shadows VSS on all later dual-store builds (local-first reads), so
channels never appeared even after connectivity recovered — and the
fork's background bulk sync would then overwrite the VSS backup with
the fresh state (fixed separately in ZeusLN/ldk-node#3).

Changes:

- Add setVssFailOnError to the native LDK Node modules (Android + iOS).
  When set, a VSS build timeout or failure rejects with code vss_error
  instead of falling back to an empty local store. Reset with each
  createBuilder, so existing-node startups keep fallback behavior.
- Thread failOnVssError through initializeNode/initNode/
  createLdkNodeWallet. Hard-fail builds get a 180s timeout (restore
  time scales with node history and connection quality; failing now
  produces a visible error rather than a silently wrong wallet).
- createLdkNodeWallet deletes the just-created node dir on failed init
  so a partially-written local DB cannot shadow VSS state on retry.
- SeedRecovery passes failOnVssError: true and shows a localized error
  (new key views.Settings.SeedRecovery.vssRestoreFailed) so the user
  knows channels were NOT restored and can retry.
- WalletConfiguration passes failOnVssError when an existing seed is
  imported; new-wallet creation keeps the non-fatal warning path.
- startLdkNodeWallet treats a startup with a saved config but no local
  DB (e.g. config restored to a new device via cloud keychain sync) as
  a de-facto restore and hard-fails on VSS errors. Wallet.tsx catches
  the failure, surfaces it via SettingsStore.error/errorMsg, and
  releases fetchLock so the user can retry without restarting the app.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
Pulls in ZeusLN/ldk-node#3: DualStore push-safety gate (prevents a
fresh local store from overwriting an existing VSS backup) and
restore-mode VSS error propagation (a failed VSS fetch now fails the
build instead of silently producing a fresh node with no channels).

Also includes the additive BOLT11 route-hints receive APIs from
ZeusLN/ldk-node#2; the uniffi interface change is purely additive and
the uniffi version is unchanged, so the vendored bindings remain
compatible without regeneration.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 31, 2026
…ding an empty wallet

Users on poor connections reported restore-from-seed coming up with no
channels. Root cause: the native VSS dual-store build had a 30/60s
timeout and on timeout or failure silently fell back to building a
fresh node with an empty local SQLite store. The restore flows ignored
the returned vssError, so the wallet config was saved and the restore
looked successful with zero channels. Worse, the fallback local DB
shadows VSS on all later dual-store builds (local-first reads), so
channels never appeared even after connectivity recovered — and the
fork's background bulk sync would then overwrite the VSS backup with
the fresh state (fixed separately in ZeusLN/ldk-node#3).

Changes:

- Add setVssFailOnError to the native LDK Node modules (Android + iOS).
  When set, a VSS build timeout or failure rejects with code vss_error
  instead of falling back to an empty local store. Reset with each
  createBuilder, so existing-node startups keep fallback behavior.
- Thread failOnVssError through initializeNode/initNode/
  createLdkNodeWallet. Hard-fail builds get a 180s timeout (restore
  time scales with node history and connection quality; failing now
  produces a visible error rather than a silently wrong wallet).
- createLdkNodeWallet deletes the just-created node dir on failed init
  so a partially-written local DB cannot shadow VSS state on retry.
- SeedRecovery passes failOnVssError: true and shows a localized error
  (new key views.Settings.SeedRecovery.vssRestoreFailed) so the user
  knows channels were NOT restored and can retry.
- WalletConfiguration passes failOnVssError when an existing seed is
  imported; new-wallet creation keeps the non-fatal warning path.
- startLdkNodeWallet treats a startup with a saved config but no local
  DB (e.g. config restored to a new device via cloud keychain sync) as
  a de-facto restore and hard-fails on VSS errors. Wallet.tsx catches
  the failure, surfaces it via SettingsStore.error/errorMsg, and
  releases fetchLock so the user can retry without restarting the app.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 31, 2026
Pulls in ZeusLN/ldk-node#3: DualStore push-safety gate (prevents a
fresh local store from overwriting an existing VSS backup) and
restore-mode VSS error propagation (a failed VSS fetch now fails the
build instead of silently producing a fresh node with no channels).

Also includes the additive BOLT11 route-hints receive APIs from
ZeusLN/ldk-node#2; the uniffi interface change is purely additive and
the uniffi version is unchanged, so the vendored bindings remain
compatible without regeneration.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant