refactor(downloader): 重构下载器分片与系统资源分配逻辑 - #3506
Draft
Pigeon0v0 wants to merge 14 commits into
Draft
Conversation
Contributor
审阅者指南重构下载器，使用自定义的自适应分段（range-based）多连接实现，并集中管理资源（连接、缓冲区、带宽）；将原先基于“线程”的单文件限制改为基于“连接”的限制；更新 UI / 配置绑定；并对小文件与大文件 / 未知大小文件引入不同的处理方式，以改善内存占用和下载性能。 自适应下载流 vs 顺序下载流的时序图sequenceDiagram
actor User
participant LoaderDownload as LoaderDownload
participant FileDownloader as FileDownloader
participant AdaptiveRangeDownloader as AdaptiveRangeDownloader
participant DownloadResourceManager as DownloadResourceManager
participant HttpClient as HttpClient
User->>LoaderDownload: ProcessFileAsync(file)
LoaderDownload->>FileDownloader: DownloadSingleAsync(url, localPath, enableParallelChunks)
alt enableParallelChunks
FileDownloader->>AdaptiveRangeDownloader: TryDownloadAsync(url, localPath, useBrowserUA, customUA, expectedSize)
alt large_file_with_range
AdaptiveRangeDownloader->>DownloadResourceManager: AcquireConnectionAsync(url)
AdaptiveRangeDownloader->>HttpClient: SendAsync(range_request)
HttpClient-->>AdaptiveRangeDownloader: PartialContent
loop per_segment
AdaptiveRangeDownloader->>DownloadResourceManager: ReserveBufferAsync(BufferSize)
AdaptiveRangeDownloader->>DownloadResourceManager: ThrottleAsync(bytes)
AdaptiveRangeDownloader->>DownloadResourceManager: RecordDownloadedBytes(bytes)
end
AdaptiveRangeDownloader-->>FileDownloader: true
FileDownloader->>FileDownloader: PromoteTempFile(localPath)
FileDownloader->>FileDownloader: MarkDownloadCompleted(trackedFile)
else range_not_supported_or_small
AdaptiveRangeDownloader-->>FileDownloader: false
FileDownloader->>FileDownloader: DownloadSequentiallyAsync(...)
end
else !enableParallelChunks
FileDownloader->>FileDownloader: DownloadSequentiallyAsync(...)
end
FileDownloader->>DownloadResourceManager: AcquireConnectionAsync(url)
FileDownloader->>HttpClient: SendAsync(request)
HttpClient-->>FileDownloader: Response
loop read_stream
FileDownloader->>DownloadResourceManager: ReserveBufferAsync(bufferSize)
FileDownloader->>DownloadResourceManager: ThrottleAsync(bytes)
FileDownloader->>DownloadResourceManager: RecordDownloadedBytes(bytes)
end
FileDownloader->>FileDownloader: PromoteTempFile(localPath)
FileDownloader->>FileDownloader: MarkDownloadCompleted(trackedFile)
文件级变更
与关联 Issue 的对照评估
可能关联的 Issue
提示与命令与 Sourcery 交互
自定义使用体验前往你的 仪表盘：
获取帮助Original review guide in EnglishReviewer's GuideRefactors the downloader to use a custom adaptive range-based multi-connection implementation with centralized resource management (connections, buffers, bandwidth), replaces thread-based per-file limits with connection-based limits, updates UI/config bindings, and introduces different handling for small vs large/unknown-size files to improve memory usage and download performance. Sequence diagram for adaptive vs sequential download flowsequenceDiagram
actor User
participant LoaderDownload as LoaderDownload
participant FileDownloader as FileDownloader
participant AdaptiveRangeDownloader as AdaptiveRangeDownloader
participant DownloadResourceManager as DownloadResourceManager
participant HttpClient as HttpClient
User->>LoaderDownload: ProcessFileAsync(file)
LoaderDownload->>FileDownloader: DownloadSingleAsync(url, localPath, enableParallelChunks)
alt enableParallelChunks
FileDownloader->>AdaptiveRangeDownloader: TryDownloadAsync(url, localPath, useBrowserUA, customUA, expectedSize)
alt large_file_with_range
AdaptiveRangeDownloader->>DownloadResourceManager: AcquireConnectionAsync(url)
AdaptiveRangeDownloader->>HttpClient: SendAsync(range_request)
HttpClient-->>AdaptiveRangeDownloader: PartialContent
loop per_segment
AdaptiveRangeDownloader->>DownloadResourceManager: ReserveBufferAsync(BufferSize)
AdaptiveRangeDownloader->>DownloadResourceManager: ThrottleAsync(bytes)
AdaptiveRangeDownloader->>DownloadResourceManager: RecordDownloadedBytes(bytes)
end
AdaptiveRangeDownloader-->>FileDownloader: true
FileDownloader->>FileDownloader: PromoteTempFile(localPath)
FileDownloader->>FileDownloader: MarkDownloadCompleted(trackedFile)
else range_not_supported_or_small
AdaptiveRangeDownloader-->>FileDownloader: false
FileDownloader->>FileDownloader: DownloadSequentiallyAsync(...)
end
else !enableParallelChunks
FileDownloader->>FileDownloader: DownloadSequentiallyAsync(...)
end
FileDownloader->>DownloadResourceManager: AcquireConnectionAsync(url)
FileDownloader->>HttpClient: SendAsync(request)
HttpClient-->>FileDownloader: Response
loop read_stream
FileDownloader->>DownloadResourceManager: ReserveBufferAsync(bufferSize)
FileDownloader->>DownloadResourceManager: ThrottleAsync(bytes)
FileDownloader->>DownloadResourceManager: RecordDownloadedBytes(bytes)
end
FileDownloader->>FileDownloader: PromoteTempFile(localPath)
FileDownloader->>FileDownloader: MarkDownloadCompleted(trackedFile)
File-Level Changes
Assessment against linked issues
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
# Conflicts: # PCL.Core/App/Localization/Languages/zh-CN.xaml # Plain Craft Launcher 2/Pages/PageSetup/PageSetupGameManage.xaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
本 PR 尚未经过全面人工 review，因此暂时 Draft
本 PR 对下载器的分片策略与系统资源分配进行了重构。
修改内容：针对文件下载回退到 HTTP/1.1（至少在我这里高峰期 1.1 的速度显著快于 2.0），元数据与 API 请求仍然使用 HTTP/2.0；仅针对大于 4 MiB 的文件尝试分片；批量任务中未知大小的文件直接下载，不进行 Range 探测；大文件下载采用新的动态分片机制，分片大小更加灵活；内存缓存改为全局共享，并及时将内容写入硬盘，避免极端情况下内存过高占用。
测试时进行 Minecraft 26.2 清洁安装，下载高峰阶段相比 PCL 2.13.1.0 的相同阶段内存占用可减少约 2/3（稳定在 180 - 220 MB），同时保持下载速度基本持平甚至略微更快；与 PCL CE 2.15.0 相比内存占用差距不大（此 PR 版本占用略微减少），主要改进了部分情况下下载速度极其缓慢的问题。
同时，close #3356 。
本 PR 主要使用 AI 完成，因此需要较为细致的测试与检查。同时，回退到 HTTP/1.1 还是保留 HTTP/2.0 可能仍需要进一步调查与讨论。
Summary by Sourcery
重构下载器，使用新的自适应基于范围的分段机制并引入共享的全局资源管理，从而改进内存使用和大文件性能，同时重构配置和 UI，使其以“连接数”而非“线程数”的方式进行交互。
Enhancements:
DownloadService使用方式替换为：内部顺序下载器以及仅在文件足够大时才启用的自适应基于范围的并行下载器。DownloadResourceManager，在全局范围内协调所有下载的 HTTP 连接、共享缓冲区和带宽限速。Original summary in English
Summary by Sourcery
Refactor the downloader to use a new adaptive range-based segmentation mechanism with shared global resource management, improving memory usage and large-file performance while reworking configuration and UI to talk in terms of connections rather than threads.
Enhancements: