Skip to content

fix(vss): bound retry budget so builds fail fast when VSS is unreachable - #4

Merged
kaloudis merged 2 commits into
zeusfrom
fix/vss-retry-budget
Jul 30, 2026
Merged

kaloudis merged 2 commits into
zeusfrom
fix/vss-retry-budget

Conversation

@kaloudis

@kaloudis kaloudis commented Jul 30, 2026

Copy link
Copy Markdown

Each HTTP request is already capped at 10s by vss-client's client-level timeout, but retry_policy() allowed 100 attempts with up to 180s of cumulative backoff — a single VSS operation could keep build_with_dual_store* blocked for several minutes. On mobile, that outlives the caller's watchdog timeout (180s on restore), which then abandons the build in a still-live thread over the shared storage dir — the root enabler of the timeout/cleanup race flagged in ZeusLN/zeus#4292 review.

Changes:

  • Retry budget: 6 attempts / 30s total delay, base backoff 100ms (worst case ≈ 1 minute per operation, comfortably inside the mobile watchdog)
  • Stop retrying AuthError — auth rejections (e.g. clock skew) cannot succeed on retry without new credentials

cargo check clean. Follow-up release + binary bump handled on the ZEUS side.

🤖 Generated with Claude Code

Trade-offs

  • Backup-staleness window: a failed background VSS push is only logged and reconciled by the next session's bulk sync. Per-push outage tolerance drops from ~3+ minutes to ~1 minute. Acceptable under the local-first dual-store design; noting it for visibility.
  • AuthError skip assumes fixed-header auth (deterministic server rejection, e.g. clock skew). Header-provider failures also map to AuthError, so an lnurl-auth deployment — where token fetches can fail transiently and succeed on retry — should not adopt this skip. Documented in the code comment.
  • Standalone build_with_vss_store (not used by ZEUS): exhausted persist retries there can be fatal to LDK, so that deployment would want a larger budget. Also documented in the code comment.
  • A slow-but-alive server (full 10s per request, eventually succeeding) can still stretch a multi-operation restore past the mobile watchdog — the ZEUS-side watchdog remains the backstop for that case.

kaloudis added 2 commits July 30, 2026 12:07
Each HTTP request is already capped at 10s by vss-client's client-level
timeout, but the retry policy allowed 100 attempts with up to 180s of
cumulative backoff, so a single VSS operation could keep
build_with_dual_store blocked for several minutes — long past the mobile
caller's watchdog timeout, which then abandons the build in a still-live
thread over the shared storage dir.

Cap retries at 6 attempts / 30s total delay (worst case roughly one
minute per operation) and stop retrying AuthError, which cannot succeed
without new credentials.
…ptions

Note that the tuning assumes ZEUS's dual-store fixed-header deployment,
that the 30s max-total-delay is only a safety cap, and that the
AuthError skip is wrong for lnurl-auth deployments where header-provider
token fetches can fail transiently.
@kaloudis
kaloudis merged commit 32fc853 into zeus Jul 30, 2026
4 of 36 checks passed
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 30, 2026
Picks up ZeusLN/ldk-node#4: VSS retry budget cut from 100 attempts/180s
to 6 attempts/30s so dual-store builds against an unreachable VSS server
error out in about a minute — well inside the restore watchdog — instead
of leaving an orphaned build thread behind, and AuthError responses are
no longer retried.

Android .so files are larger than the previous release's because those
were built with an older stable rustc (toolchain was upgraded midway
through the previous release build); this release is built uniformly
with rustc 1.97.1. APK size is unaffected as Gradle strips symbol
tables at packaging.
kaloudis added a commit to kaloudis/zeus that referenced this pull request Jul 31, 2026
Picks up ZeusLN/ldk-node#4: VSS retry budget cut from 100 attempts/180s
to 6 attempts/30s so dual-store builds against an unreachable VSS server
error out in about a minute — well inside the restore watchdog — instead
of leaving an orphaned build thread behind, and AuthError responses are
no longer retried.

Android .so files are larger than the previous release's because those
were built with an older stable rustc (toolchain was upgraded midway
through the previous release build); this release is built uniformly
with rustc 1.97.1. APK size is unaffected as Gradle strips symbol
tables at packaging.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant