Repository navigation
sqlitedb: catch a database file replaced after a transaction's first write - #94
Merged
Merged
Conversation
…write SQLite checks that its open file is still the one at the path (SQLITE_READONLY_DBMOVED) only when a write transaction starts. A file replaced after that, by a sync client, a restore or a copy-then-rename, isn't noticed. COMMIT succeeds into the old, now nameless file and the write is lost without an error. _open_db now records the (st_dev, st_ino) of the file it opened. Inside a journaled write operation, _commit checks that the path still names that file before committing; a deleted file counts as replaced. If it doesn't, the handle is treated as poisoned and the transaction goes through #93's recovery: reopen, replay against the file now at the path, verify, commit. Only a window of microseconds between the check and the COMMIT remains. The check is one stat per commit (about 1 us locally, against about 1.8 ms for the commit). A volume with unstable inode numbers can trip it falsely, which costs one replay onto the same file, since nothing had been committed yet. This is a Python-only addition, recorded in the bridge entry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012Dn4gCNCU2jUGdmfPVCAGK
This was referenced Oct 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the gap #93 recorded as undetectable.
The problem
SQLite checks whether its open file is still the one at the path only when a write transaction starts (
SQLITE_READONLY_DBMOVED, which #93 recovers from).If the file is replaced after that first write, the rest of the transaction and its COMMIT go to the old file. Typical causes are a sync client (Dropbox, OneDrive, iCloud), a backup restore, or a copy-then-rename script. That old file no longer has a name, and it's deleted when the connection closes. COMMIT reports success and the write is lost.
test_sqlite_alone_loses_the_writeshows this on a baresqlite3connection.The change
_open_dbrecords the(st_dev, st_ino)of the file it opened._commitchecks that the path still names that file before committing (_file_was_replaced). A deleted file counts as replaced.The data-loss window shrinks from "first write to commit" (the whole operation) to the microseconds between the check and COMMIT. This is the same comparison SQLite's unix VFS makes for DBMOVED, applied at the one moment SQLite doesn't make it.
Cost
One
os.statper commit: about 0.9 µs locally, against about 1,800 µs for an insert and commit. On network drives a stat costs more, but so does the commit.Edge cases
do_run_sql_query, aren't covered. They're rare.This is a Python-only addition, recorded in the
sqlitedbbridge entry. MATLAB has the same gap, narrower because each mksqlite statement commits on its own.Tests (
tests/test_sqlitedb_transient_errors.py, 6 new)add_docsis caught and the document lands;The replaced-file and deleted-file tests fail without the change.
Locally:
black --check,ruff check(0.16.5),pytest(800 passed), andcheck_bridge_coverage.py(0 problems).It doesn't remove the advice not to keep a DID database that's in use inside a sync folder: sync clients can also overwrite in place or make conflicted copies, which no check inside DID can see.
Generated by Claude Code