Repository navigation

tokenizer: Reuse buffers in peeking reads - #2729

Closed

mvzink wants to merge 3 commits into

apache:mainfrom

readysettech:perf-tokenize-word-buffer

Closed

tokenizer: Reuse buffers in peeking reads#2729
mvzink wants to merge 3 commits into
apache:mainfrom
readysettech:perf-tokenize-word-buffer

Commits on Oct 26, 2026
- bench: add mixed SQL benchmarks and identifier length sweep
  mvzink
  committed
  bc1c106
- tokenizer: move peeking_take_while onto State
  mvzink
  committed
  9b6e281
- tokenizer: avoid temporary strings when building tokens
  mvzink
  committed
  dafd04e