[mcollina]

Follow-up to #37: the Postgres counterpart of its S3 change. A paged restore reads each page a query waits for with one ranged read, so the latency of openLtxFile() is time the query waits.

Before

PgReplicaClient.openLtxFile({ offset, size }):

1. SELECT size from the files table, for the bounds and to detect a missing file;
2. a cursor over the chunks covering the range, each a whole 1 MiB chunk, cut on the client.

A read of 32 pages (≤ 128 KiB) took two round trips and transferred 1–2 MiB.

After

One query, through the same server-side cursor:

SELECT c.seq, substring(c.data FROM … FOR …) AS data
FROM files f
LEFT JOIN chunks c ON <same file> AND c.seq BETWEEN first AND last
WHERE <file>
ORDER BY c.seq

• substring() cuts each chunk to the bytes of the range on the server. Chunks are stored uncompressed (pg_column_compression(data) is null, since LTX pages are already LZ4-compressed), so Postgres reads only the slice it returns.
• No row: the file does not exist, and the call rejects as before, since the first batch is read before it returns. One row without a chunk: the range starts past the end, and the stream is empty.
• Offsets are computed as bigint, so files larger than 2 GiB work; the length passed to substring() is capped at the chunk size before its int cast.
• A stream destroyed early closes the cursor, releasing the connection.

Numbers

200 reads of 128 KiB at random offsets of a 16 MiB file, local Postgres 18: 9.7 ms → 1.4 ms per read.

Tests

New openLtxFile reads ranges across chunks, in one query each: 64-byte chunks, 40 of them (more than one cursor batch), ranges inside a chunk, across chunks, from a boundary, to the end, past the end and from the end; each read counts the queries with the debug hook of postgres.js (the old code fails it with 2). Then a stream destroyed early, with a pool of one connection, followed by another read.

Also ran Lavoratore's replicator tests on Postgres with this build, including its paged restore tests: 13/13.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sfj4nDS4Mc5fAjmzSLYG7k