Skip to content

Recent easy corruption of nodes on latest/nightly/testing tag for sync & bucket corruption log for fresh sync #911

@jun0tpyrc

Description

@jun0tpyrc

What version are you using?

multiple of
stellar/quickstart:nightly
stellar/quickstart:latest
stellar/quickstart:testing
at the moment being "Id": "sha256:e7f05fac9ce971eccaabfd528297380c682f39f0d687c02bc7547d273a360145",

What did you do?

nodes broken , reworked multiple times with different tags

What did you expect to see?

working

What did you see instead?

always have postgres broken

2026-03-02T06:18:44.085 [default INFO] Assigning calculated value of 2 to FAILURE_SAFETY
current exception: std::runtime_error("Bucket directory is corrupt")
backtrace:
   0: stellar::printCurrentBacktrace()
             at /wd/src/util/Backtrace.cpp:27:22
   1: stellar::printBacktraceAndAbort()
             at /wd/src/main/main.cpp:140:5
   2: <unknown>
   3: <unknown>
   4: __cxa_throw
   5: loadLastKnownLedgerInternal
             at /wd/src/ledger/LedgerManagerImpl.cpp:550:9
   6: start
             at /wd/src/main/ApplicationImpl.cpp:805:21
   7: runApp
             at /wd/src/main/ApplicationUtils.cpp:117:10
   8: operator()
             at /wd/src/main/CommandLine.cpp:1638:20
      __invoke<(lambda at main/CommandLine.cpp:1576:9) &>
             at /usr/lib/llvm-20/bin/../include/c++/v1/__type_traits/invoke.h:179:25
      __call<(lambda at main/CommandLine.cpp:1576:9) &>
             at /usr/lib/llvm-20/bin/../include/c++/v1/__type_traits/invoke.h:243:12
      __invoke_r<int, (lambda at main/CommandLine.cpp:1576:9) &>
             at /usr/lib/llvm-20/bin/../include/c++/v1/__type_traits/invoke.h:273:10
      operator()
             at /usr/lib/llvm-20/bin/../include/c++/v1/__functional/function.h:167:12
      operator()
             at /usr/lib/llvm-20/bin/../include/c++/v1/__functional/function.h:319:10
   9: operator()
             at /usr/lib/llvm-20/bin/../include/c++/v1/__functional/function.h:436:12
      operator()
             at /usr/lib/llvm-20/bin/../include/c++/v1/__functional/function.h:995:10
      runWithHelp
             at /wd/src/main/CommandLine.cpp:479:12
  10: run
             at /wd/src/main/CommandLine.cpp:1570:12
2026-03-02T06:14:18.407 [default INFO] Assigning calculated value of 2 to FAILURE_SAFETY
current exception: std::runtime_error("sqlite3_statement_backend::loadOne: database disk image is malformed")
2026-03-02 06:15:19 UTC [120668-1607] stellar@horizon ERROR:  could not read block 31756 in file "base/16384/16401": read only 0 of 8192 bytes
2026-03-02 06:15:19 UTC [120668-1608] stellar@horizon STATEMENT:  SELECT COALESCE(MAX(sequence), 0) FROM history_ledgers

even re-sync with diff image tag still seeing this

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    Backlog (Not Ready)

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions