Repository navigation
Conversation
Two tests of the paged restore: - once every page is restored, queries, writes, syncs and snapshots read nothing from the replica; - the bytes read before restorePaged() resolves barely grow with the database: 112 KB for 8.6 MB, 176 KB for 33 MB (the page indexes grow, the rest does not), under 2% of the database. bench/bench-paged-restore.ts (pnpm bench:paged) measures the time to the first response of a paged restore against a full restore as the database grows, through a simulated network: 335 ms to 528 ms from 16 MiB to 1 GiB, where a full restore takes 1.3 s to 64 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Sfj4nDS4Mc5fAjmzSLYG7k
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The two items left on PRD-2215 after #37.
Once filled, nothing is read from the replica
New test: after
filledresolves, a worker thread runsPRAGMA integrity_checkand a full scan through the VFS, writes a row, and the replica syncs and snapshots. The client records everyopenLtxFile(): none.Acceptance: time to first response as the database grows
Test (deterministic, runs in CI): the bytes read before
restorePaged()resolves, for a database of about 8.6 MB and one four times larger (33 MB). Before the database can be opened, the restore reads two LTX headers, the page indexes of the plan, which take a few bytes per page, and the schema with its read-ahead:It asserts that 4× the data reads less than 2× the bytes, and under 2% of the database.
Benchmark:
bench/bench-paged-restore.ts(pnpm bench:paged), which needs the paged VFS built. The replica is read through a simulated network: 20 ms per request, and one 100 MB/s link shared by concurrent reads. A worker thread reads a random row as soon as the paged restore resolves. Apple M4 Max, Node 24.18:The results are in the README, under Benchmarks.
Two findings, not addressed here
calcRestorePlan()lists the ten levels of the replica one after another, one request each. Listing them concurrently would cut the time to first response by about a third with S3-like latency, and would speed up full restores too.🤖 Generated with Claude Code
https://claude.ai/code/session_01Sfj4nDS4Mc5fAjmzSLYG7k