perf(fast): batch merge replay for pre-existing merges#61
Merged
Conversation
Replaying N pre-existing merges (SuperBPE phase 2, add_merges) applied them one at a time, each with its own doc pass. Trainer.apply_merges replays the whole list in one call: word-level replay composes each original word pointer to its final graph via the WordRegistry (recording each word's exact collapse step), doc-top events simulate exactly the top-level matches sequential replay performs (both children Node-collapsed before step k, bytes match, left-to-right non-overlapping, cascades), and docs are rewritten once at the end. Byte-identical by construction; a direct probe (sequential x250 vs one batch call) differs in 0 of 1,290 docs, and all digests at all scales are unchanged. Non-Unconn graphs and non-pair merges fall back to the sequential path. SuperBPE full scale: 5000 merges 14.44s -> 8.25s (1.75x), 500 merges 6.48s -> 5.96s. Other cases unchanged. Trained-state reuse (to_connected) was built, measured, and rejected: the phase-1 -> phase-2 handoff is lossy by design (merges travel as lossy UTF-8 strings; non-UTF8-aligned tokens round-trip through U+FFFD and are silently dropped from phase-2 state, in both implementations), so reusing true trained state can never be byte-identical. Documented in issue. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #60 (merge order: #59 → #60 → this, normal merges).
What
Trainer.apply_mergesreplays a whole pre-existing merge list in one call instead of N sequential per-merge doc passes (SuperBPE's phase-2 replay is 2,500 merges at the 5000-merge scale): word-level replay via the existingWordRegistrycomposing original pointer → final graph with exact collapse steps, doc-top events simulated precisely as sequential replay performs them (Node-collapse ordering, byte match, left-to-right non-overlap, cascades), one doc rewrite at the end. Non-Unconn/non-pair falls back to the sequential path.Byte-identical by construction and by probe: sequential
apply_merge×250 vs oneapply_mergeson the same corpus → 0 of 1,290 docs differ; all digests unchanged at all three scales.Why not trained-state reuse (the bigger idea — built, measured, rejected)
Carrying phase-1's trained state into phase 2 broke full-scale digests, and the diff exposed why: the phase handoff is lossy by design. Merges travel as strings (
get_merges/add_merges); non-UTF8-aligned tokens (e.g. the(0xE2, 0x80)pair inside an en-dash) round-trip through U+FFFD and the replayed merge matches nothing — silently dropped from phase-2's starting state. The reference does exactly the same, so this is canonical semantics; true-state reuse can never match it. Filed as a spec question in the linked issue — an output-changing decision, not taken here.Results (cold, full scale)
5ed8191cb8✓30929f0c51✓BPE/BNE/Boundless unchanged at both depths (digests exact); 2000-row defaults all match; both suites 145 passed, 2 skipped; clippy unchanged.
🤖 Generated with Claude Code