perf(fast): word-registry inverted index + score-bucketed pick queue (5000-merge round)#59
Merged
Conversation
Windowed phase profiling at 5000 merges (full wikitext-2) showed the two costs that grow or stay fixed with merge depth: - wordres: the per-step registry precompute scanned ~74k words x 5000 steps (half of Boundless's loop) while the registry barely shrank. A WordRegistry inverted index (candidate -> containing words), maintained incrementally from each changed word's old/new candidate sets, makes changed-word discovery an index lookup. - pick: the full-map best scan grew 25 -> 474ms per 500-step window as the global candidate map grew 37k -> 313k. PickQueue score-buckets candidates (BTreeMap score -> id set) fed by the same op stream that updates the sharded counts, with per-shard candidate interning; pick is a tail lookup, and the max bucket is exactly the tied set, resolved by the unchanged earliest-entry logic, so tie-breaks stay byte-identical. The SuperBPE premerge replay reuses the same index (no per-call scans), and pick is now flat at 1-10ms per window in every case. Full scale, 5000 merges, cold subprocess per case, digests unchanged: - BPE: 3.96s -> 2.71s (1.5x) - BNE n=4: 7.41s -> 4.07s (1.8x) - Boundless: 20.76s -> 11.41s (1.8x) - SuperBPE: 23.42s -> 15.70s (1.5x) 500-merge and 2000-row digests unchanged. Tried and reverted: per-step net-transition dedup before bucket updates (noise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
10× merge depth round: full wikitext-2, 5000 merges, with windowed phase profiling (per-500-step windows) to see which costs grow, shrink, or stay fixed as training deepens.
What the evolution profile showed
Two costs don't scale with actual work:
wordresis flat and huge: the per-step registry precompute scanned ~74k words × 5000 steps (~9.5s, 50% of Boundless's loop) — the registry barely shrinks (73,985→72,080; words rarely collapse fully), so this never gets cheaper.pickgrows: the full-map best-scan went 25→474ms per window as the global candidate map grew 37k→313k over 5000 merges (~3.9s of BNE's 6.7s loop — n-ary windows inflate its candidate space fastest).Everything else (apply/diff/shard) tracks the affected-doc count, which collapses 4,866→112 docs/step — that part was already right.
Changes
WordRegistryinverted index (candidate → containing words), maintained incrementally from each changed word's old/new candidate key sets — changed-word discovery becomes an index lookup instead of a 74ktry_mergescan.PickQueue— gigatoken's priority-queue idea, adapted to our exact tie contract: candidates score-bucketed (BTreeMapscore → id set), fed by the same op stream that updates the sharded counts, with per-shard candidate interning. Pick = tail lookup; the max bucket is the tied set, resolved by the unchanged earliest-entry logic — tie-breaks stay byte-identical (all digests prove it).Tried and reverted: net-transition dedup per step before bucket updates (11.07→11.02s — noise).
Results (cold subprocess per case, full split, 5000 merges)
d770bfd660✓eb47975d9a✓7b51919f33✓5ed8191cb8✓After:
pickis flat at 1–10ms/window in every case; all remaining per-window cost is work-proportional. 500-merge full-scale digests (2af4c7edca/967201e51c/10318b5e1c/30929f0c51) and 2000-row defaults unchanged; both suites (properly aliased) 145 passed, 2 skipped; clippy unchanged.Next round (already scoped)
The candidate interning PickQueue introduced is half of full u32-id keying; extending ids through ops/global/index kills the remaining
Vec<GraphV>clone+hash traffic — the early-window mass (steps 0–500 are ~half of Boundless's remaining loop). That's the gigatokenShortPretokenCache-style round, next.🤖 Generated with Claude Code