Skip to content

Add tiled tree recovery API - #77

Merged
Amaury Chamayou (achamayou) merged 8 commits into
mainfrom
achamayou-tiled-tree-recovery-api
Sep 3, 2026
Merged

Add tiled tree recovery API#77
Amaury Chamayou (achamayou) merged 8 commits into
mainfrom
achamayou-tiled-tree-recovery-api

Conversation

@achamayou

@achamayou Amaury Chamayou (achamayou) commented Aug 21, 2026

Copy link
Copy Markdown
Member

Summary

  • add TiledTreeT::resume for an existing namespace, serialized tree state, and a caller-validated full-tile boundary
  • add a dual-boundary resume overload and immutable_size() so interrupted flush recovery distinguishes the last verified prefix from the rollback seal
  • add TiledTreeT::from_frontier so logical recovery can complete before any tile namespace is ready
  • add TileWriterT::repair for detached/background namespace population or repair after an explicit trusted prefix
  • add monotonic adopt_tile_prefix after the detached writer is quiesced
  • add flush_up_to to publish and seal only a committed full-tile prefix while preserving a speculative suffix
  • preserve trusted tiles while replacing stale or untrusted suffix files at every tile level
  • preserve existing fresh-tree and ordinary TileWriter scan/repair behavior
  • document ownership, concurrency, checkpoint, repair, adoption, committed-prefix, and failure contracts

APIs

Resume a complete, durable tile prefix:

auto resumed = TiledTree::resume(
  config,
  hash_algorithm_short_name,
  serialised_tree,
  full_tile_boundary);

After an interrupted flush, restore the last fully successful prefix separately from the possibly larger rollback seal:

auto resumed = TiledTree::resume(
  config,
  hash_algorithm_short_name,
  serialised_tree,
  flushed_tile_boundary,
  immutable_boundary);

The caller establishes namespace ownership. Recovery verifies every required tile and higher-level roll-up against the serialized tree. Proof reads are capped at flushed_tile_boundary; tiles through immutable_boundary remain sealed against rollback, while the unverified suffix is replaced when flushing resumes.

Restore logical state before tiles are ready, repair independently, then adopt the completed prefix:

auto recovering = TiledTree::from_frontier(
  config,
  hash_algorithm_short_name,
  serialised_tree);

TileStore repair_store(config.prefix, hash_algorithm_short_name);
auto repair = TileWriter::repair(repair_store, trusted_boundary);
repair.write_up_to(target_size, authoritative_leaf_at);

// After the repair writer is quiesced:
recovering.adopt_tile_prefix(target_boundary);

from_frontier performs no namespace I/O and trusts no tile files. Root computation and appends are available immediately. If old leaves are non-resident, tile-dependent proofs and live flushing fail until a complete repaired prefix overlaps the resident frontier.

For a tree with a speculative suffix, publish only its committed prefix:

log.flush_up_to(committed_leaf_count);

Only complete tiles wholly within that leaf count are written and sealed. A failed write may advance immutable_size() without advancing flushed_size(); applications should checkpoint the serialized frontier and both boundaries together.

The detached writer may run on a background thread; merklecpp does not create or manage threads. The application must serialize writers sharing the namespace and quiesce repair before adoption. Adoption exhaustively verifies the prefix, updates flushed and immutable boundaries together, and continues to distrust the suffix.

Validation

  • exact CI reproduction in WSL: Debug Clang, OpenSSL enabled, integrated clang-tidy, all three public-header checks, and 22/22 tests passed
  • WSL GCC full short suite: 20/20 tests passed
  • WSL targeted clang-tidy build: tiles_resume, tiles_writer, and tiles_tree
  • WSL trace-enabled build and tests: tiles_resume and tiles_writer
  • lifecycle coverage for frontier-only restore, no-I/O construction, guarded proof/flush behavior, detached repair, incremental repair, adoption, monotonicity, resumed growth, interrupted flush recovery, committed-prefix flushing, and stale suffix replacement
  • final read-only code review found no remaining high-confidence defects

Restore TiledTree from serialized tree state and a caller-validated tile boundary. Harden tree deserialization and replace untrusted suffix tiles during resumed growth.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 455a0fc3-10c9-4831-bf66-f06ff952f607
Restore logical tree state before tiles are ready, populate or repair the namespace with a detached writer, and adopt a quiesced durable prefix without trusting stale suffix files.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 455a0fc3-10c9-4831-bf66-f06ff952f607
Make the traversal child invariant explicit and satisfy the public-header lint rules for deserialization and writer rebinding.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 455a0fc3-10c9-4831-bf66-f06ff952f607
Validate every required tile and roll-up against the serialized frontier, and restore distinct flushed and immutable boundaries after interrupted flushes.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve deserialization conflicts by retaining the newer exception-safe bounds checks and fuzz coverage from main.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Allow consensus-driven callers to write and seal only complete tiles within a committed leaf-count prefix while retaining a rollbackable logical suffix.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@achamayou
Amaury Chamayou (achamayou) marked this pull request as ready for review September 2, 2026 11:05
@achamayou
Amaury Chamayou (achamayou) requested a review from a team as a code owner September 2, 2026 11:05
Copilot AI balanced review requested due to automatic review settings September 2, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Durable recovery and crash-consistency behavior warrant final human review despite comprehensive coverage.

Pull request overview

Adds durable tiled-tree recovery, detached repair/adoption, and committed-prefix flushing.

Changes:

  • Adds resume, frontier restoration, repair, adoption, and dual-boundary recovery APIs.
  • Adds flush_up_to() and immutable-boundary tracking.
  • Expands lifecycle tests and recovery documentation.
File summaries
File Description
merklecpp_tiles.h Implements recovery and committed-prefix APIs.
test/tiles_resume.cpp Tests restoration, repair, adoption, and interrupted flushes.
test/tiles_writer.cpp Tests trusted-prefix repair behavior.
test/tiles_geometry.cpp Tests committed-prefix flushing.
test/CMakeLists.txt Registers the recovery test.
README.md Introduces recovery workflows.
doc/tiles-guide.rst Documents lifecycle and durability contracts.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@achamayou
Amaury Chamayou (achamayou) merged commit 9d8a751 into main Sep 3, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants