Repository navigation
perf: skip unchanged lifecycle reads with SQLite data_version - #219
Merged
Merged
Conversation
On Linux, the lifecycle check keeps one read-only connection open and uses PRAGMA data_version as the engine's commit check. When the value and the agent thread set match the last successful read, it skips the source read. Windows and every other platform keep the full read. A replaced database is detected by the dev:ino:ctime identity and forces a reopen and full read. A skip inside a tool's outer transaction writes nothing and reuses the last committed version; a full read inside one never records a new version, so a rollback cannot leave a newer version behind restored rows. Copy-server: idle 0.1378 -> 0.0567 cores; 1 commit/s 0.1378 -> 0.0647.
Owner
Author
|
@coderabbitai full review |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
|
Owner
Author
|
CodeRabbit had no quota for this head: its free OSS review limit was reached and its replies say "Review rate limited". Under the standing review exception through 2026-11-10 the review is skipped, with later findings as a follow-up PR. The repository's bompus-ai-review app and the local Sol xhigh review cover the gate. |
![bompus-ai-review[bot]](https://avatars.githubusercontent.com/u/253635?s=60&v=4){kind=small}
The gate fingerprinted only the database file, so revoking read permission on the -wal while the writer stayed open left the identity unchanged and the already-open connection returned ready. The fingerprint now covers the WAL sidecar's device, inode, mode and owner, and a missing or unstattable WAL forces the full read. WAL content timestamps stay out of the identity because a commit changes them; data_version reports the commit. Regression: warm a WAL source, chmod the -wal 000 while the writer is open, reconcile returns unavailable.
The gate now resolves the database filename before locating the WAL, so a filename symlink cannot point the fingerprint at a different WAL, and it includes the -shm sidecar's identity and access state. A permission change on either sidecar closes the persistent connection and forces the full read; a missing sidecar is normal and an unstattable one forces the read. Tests: a symlinked database with the target's WAL read permission revoked returns unavailable, and an external writer holding the shared memory plus a revoked -shm returns unavailable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #218 (closed, not landed): the stat-fingerprint gate could cache a pre-commit WAL snapshot and could survive an outer transaction rollback.
On Linux,
Lifecycle.reconcileopens one persistent read-only connection to the source and usesPRAGMA data_versionas the commit check. When the value and the agent thread set match the last successful read, it skips the source read and returns the saved result. A changed value does the full read.The database filename is resolved before its sidecars are located. The connection is reopened when the database,
-walor-shmsidecar changes identity or access mode, so a replaced file, a filename symlink retarget or a sidecar permission change forces a full read. Any stat or query error, a dead connection, a missing or unreadable source, or an unexpected platform falls back to the full read; no path can admit on an unverified cache. Inside a tool's outer transaction a skip writes nothing and reuses the last committed version; a full read there never records a new version, so a transaction rollback cannot leave a newer version behind the restored rows.Windows does not open a persistent connection and keeps today's full read; the handle behaviour there is untested. The ready/unavailable answers, invalid and unverified row handling, and delivery admission are unchanged; only an unchanged read is skipped.
Evidence
unavailable, a replaced inode is read again, a dead gate connection falls back to the full read, revoking read permission on the-walor the-shm(with an external writer holding the shared memory) after the cache is warm returnsunavailable, a symlinked database with the target WAL permission revoked returnsunavailable, and a non-Linux instance never opens the gate and reads every time. Focused file 82 pass;bun run checkpass; full suite 1,468 pass / 0 fail / 7 skip.Limits