sftp: remove the partially written temp file if writing it fails - #219
Merged
ThomasWaldmann merged 1 commit intoSep 6, 2026
Merged
Conversation
store() writes to a temp file and then renames it to the final name. If writing that temp file failed (e.g. server disk full, I/O error), the partially written *.tmp file was left behind on the server: invisible to .list (TMP_SUFFIX), never removed, occupying space. Only a failing rename was cleaned up, not a failing write. Now the temp file is removed on any failure of the write or the rename. The cleanup is best effort: if the unlink itself fails (e.g. the file was never created, or the connection is gone), the original exception is kept, which is the more interesting one and lets with_reconnect still do its job. This mirrors the posixfs fix (borgbackup#217). Note: if the connection is lost mid-write, the temp file cannot be removed over the dead connection and, as store() is retried from scratch with a fresh temp name, that one temp file may be orphaned on the server; that pre-existing limitation is not addressed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
store()writes to a temp file and then renames it to the final name. If writing that temp file failed (e.g. the server ran out of space, or an I/O error), the partially written*.tmpfile was left behind on the server: invisible tolist()(TMP_SUFFIX), never removed, occupying space. Only a failing rename was cleaned up, not a failing write.Now the temp file is removed on any failure of the write or the rename. The cleanup is best effort: if the
unlinkitself fails (e.g. the file was never created, or the connection is gone), the original exception is kept, which is the more interesting one and letswith_reconnectstill do its job.Tests use a fake paramiko client (like the existing
test_sftp_store_memoryview_gives_bytes_to_paramiko), so they need no real server:writeand, for pipelined writes, at close time) → the temp file that was opened gets unlinked;The two write-failure tests fail on
mainand pass with this change.Note: if the connection is lost mid-write, the temp file cannot be removed over the dead connection and, since
store()is retried from scratch with a fresh temp name, that one temp file may be orphaned on the server. That pre-existing limitation (sftp-only, due to reconnect) is not addressed here.🤖 Generated with Claude Code