Skip to content

Sort a wide directory's chunks while it is read - #1547

Merged
thekevinbot merged 4 commits into
mainfrom
bot/1461-step3-1020
Nov 2, 2026
Merged

thekevinbot merged 4 commits into
mainfrom
bot/1461-step3-1020

Conversation

@thekevinbot

Copy link
Copy Markdown
Collaborator

Fixes #1546
Refs #1461

A directory wider than one 8192-entry chunk was read whole, then sorted on one thread. Now, once a second chunk arrives, a thread of its own sorts each chunk while the next is read; the sorted chunks are then split at names sampled from the first chunk and merged on every core (a stable sort over each run's already-sorted shares). A directory of one chunk is sorted as before, with no thread. Only listing.rs changes; scanner.rs and tree_walk.rs are untouched.

Measurements

1.2M-entry flat tree (./docs and the root, every 50th file a dotfile), release builds, paired runs on a loaded host. Output md5 identical to main for ./docs, ./*, ./, ./docs/*, ./**/*.md, and a 74k-file deep tree.

PENDING

E2E Verification

PENDING

Changelog / Migrations

  • Changelog fragment added under <root>/<pkg>/changelog.d/ for each changed package (or: skip-changelog trailer on a commit with reason)
  • Migration fragment with the same slug added under <root>/<pkg>/migrations.d/ for each changed package (every subsection but Summary _None._ when nothing breaks)

@thekevinbot
thekevinbot marked this pull request as ready for review October 20, 2026 12:28
@thekevinbot
thekevinbot merged commit 9c74e5c into main Nov 2, 2026
89 checks passed
@thekevinbot
thekevinbot deleted the bot/1461-step3-1020 branch November 2, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed: sort a wide directory's chunks while it is read

1 participant