feat(sftp): add downloadToRandomAccess for offset-based download#173
Merged
vicajilau merged 2 commits intoJun 30, 2026
Merged
Conversation
Adds SftpFile.downloadToRandomAccess, which downloads a file into a dart:io RandomAccessFile, writing each SFTP read reply at its own offset rather than requiring replies to arrive in order. Unlike downloadTo (which streams sequentially into a StreamSink), downloadToRandomAccess uses a completion queue that decouples the order in which pipelined read replies arrive from the order in which they are written to disk. This lets pipelined reads keep making progress when a later offset completes before an earlier one, improving throughput on high-latency or reordering links. Implementation notes: - Adaptive concurrency: starts with one outstanding read and ramps up to maxPendingRequests as replies succeed, backing off the chunk size on short reads to avoid over-requesting past EOF. - Validates chunkSize and maxPendingRequests as positive. - Throws SftpError on incomplete downloads (byte count mismatch). Add a protocol-level test that issues reads in pipelined fashion and replies out of order, asserting the final file contents are written at the correct offsets.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #173 +/- ##
==========================================
+ Coverage 53.84% 53.99% +0.14%
==========================================
Files 64 64
Lines 5198 5269 +71
==========================================
+ Hits 2799 2845 +46
- Misses 2399 2424 +25
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
SftpFile.downloadToRandomAccess, which downloads a file into adart:ioRandomAccessFile, writing each SFTP read reply at its own offset rather than requiring replies to arrive in order.Unlike
downloadTo(which streams sequentially into aStreamSink),downloadToRandomAccessuses a completion queue that decouples the order in which pipelined read replies arrive from the order in which they are written to disk. This lets pipelined reads keep making progress when a later offset completes before an earlier one, improving throughput on high-latency or reordering links.Chain note
This PR is built on top of #172 (
feat(sftp): prefer posix-rename@openssh.com). If #172 is merged first, only thedownloadToRandomAccesscommits will show in this PR's diff. If #172 is still open, both features are included here — please review and merge #172 first, then this PR will auto-rebase to show only the remaining changes.Changes
SftpFile.downloadToRandomAccess(RandomAccessFile destination, ...):maxPendingRequestsas replies succeed, backing off the chunk size on short reads to avoid over-requesting past EOF.chunkSizeandmaxPendingRequestsas positive.SftpErroron incomplete downloads (byte count mismatch)._ReadCompletionhelper class (kept SDK 2.17-compatible — no Dart 3 record syntax).Test plan
test/src/sftp/sftp_client_protocol_test.dart— all 13 tests pass (including the new out-of-order test).read3beforeread2and the output file is still in offset order.dart analyze— no new warnings introduced.Backport of ServerBox's forked dartssh2.