Skip to content

fast: training mutates the global cluster cache, so results depend on process history #56

Description

@AmitMY

Found while benchmarking at full scale (wikitext-2, 500 merges): the fast implementation's training mutates the global cluster cache (apply_merge_to_cluster_cache in fast/src/units.rs), so the second training run in the same process starts from mutated multi-byte-cluster graphs and can produce different merges than the first (e.g. full-scale BPE cold 2af4c7edca vs warm 2a0889a340). Deterministic per cache state, invisible at small scales/merge counts.

The reference implementation has no such statefulness — training the same tokenizer twice in one process yields identical results — so this is a divergence from the source-of-truth contract, and it silently affects any benchmark or user code that trains more than once per process (e.g. SuperBPE's two phases are only correct because they intend to share the cache).

Likely fix direction: make the cluster cache immutable during training (snapshot per trainer, as the streaming path already does with snapshot_word_cache/replace_word_cache), or key it per-trainer rather than global.

All current benchmark methodology (bench_tokenizers.py subprocess-per-case, and the PR #55 numbers) uses cold processes, so published numbers are unaffected.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions