perf(import): read tags for a multi-file manual import preview in parallel - #220
Draft
jordanfelle wants to merge 2 commits into
Draft
jordanfelle wants to merge 2 commits into
jordanfelle wants to merge 2 commits into
Conversation
The preview read tags and duration (ffprobe or TagLib over network storage) one file at a time, which took about 340 s for a 3.5 GB, roughly 100-file audiobook on a cold cache. Warm the existing tag cache with four concurrent reads before the per-file loop; the loop then reads every file from the cache, so results are unchanged.
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 26, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 26, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 26, 2026
The tag extraction cache clears itself entirely once it passes 1,000 entries. Warming a folder with more files than that filled the cache, wiped it mid-prefetch, and the per-file loop then read every file again. Prefetch at most 800 files; the rest are read by the loop as before.
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 27, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 27, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 27, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 27, 2026
jordanfelle
added a commit
to jordanfelle/chaptarr
that referenced
this pull request
Sep 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DRAFT.
Problem
The Manual Import preview reads tags and duration (ffprobe or TagLib over network storage) one file at a time, which took about 340 s for a 3.5 GB, roughly 100-file audiobook on a cold cache (a 655-file audiobook took 577 s).
Fix
Warm the existing tag cache with four concurrent reads before the per-file loop; the loop then reads every file from the cache, so results are unchanged. Files that fail tag extraction are read twice.
Verification
SimpleImportDecisionMakertest (fails without the change).Review update:
PerFileMatchingruns a full staged match with FTS recall for every file; see the linked issue), not by tag reads. This PR stays a draft until a cold folder has been timed before and after.multi_file_preview_should_not_prefetch_more_files_than_the_tag_cache_can_hold, failed with 887 reads against a cap of 800) (3,022 core tests pass).