Commit 4f601d2
committed
perf: reuse cached tokens when probing PostgreSQL typed literals
The typed-literal lookahead copied the tokenizer and scanned the token after
an identifier, then scanned it again on the normal expression path. Ordinary
column references and function names paid this cost even when the following
token could not form a type-prefixed literal.
Consume the leading name once and reuse the real tokenizer's cached next
token. Only copy tokenizer state when a following string, modifier list,
qualification or multiword type prefix warrants a type probe. Let the type
parser accept an already-consumed leading token, retaining the original
source span without rescanning it. Failed probes resume the ordinary name
or function parser with the same token boundary and quoting information.
This changes allocation-free lookahead, not grammar, AST shape or ownership.
It reduces the observed simple SELECT, JOIN and complex SELECT costs relative
to the first expression implementation. The broader grammar still has a
measurable cost relative to the earlier selective parser; performance claims
remain scoped to measured workloads and allocation modes.
Validation: the forced full C++ build passes 1,370 active tests with 37
backend-dependent skips and builds the corpus harness. All 42 focused
expression/AST tests pass under AddressSanitizer and UndefinedBehaviorSanitizer.
The direct Rust API benchmark validates complete AST parsing and canonical
SQL equivalence before timing. Benchmark sources, results and build artifacts
remain uncommitted.1 parent 24cd1f3 commit 4f601d2
2 files changed
Lines changed: 11 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
181 | 181 | | |
182 | 182 | | |
183 | 183 | | |
184 | | - | |
185 | | - | |
186 | | - | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
187 | 188 | | |
188 | 189 | | |
189 | 190 | | |
190 | 191 | | |
191 | 192 | | |
192 | | - | |
193 | | - | |
| 193 | + | |
| 194 | + | |
194 | 195 | | |
195 | 196 | | |
196 | 197 | | |
197 | 198 | | |
198 | 199 | | |
199 | 200 | | |
200 | 201 | | |
| 202 | + | |
201 | 203 | | |
202 | 204 | | |
203 | 205 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22 | 22 | | |
23 | 23 | | |
24 | 24 | | |
25 | | - | |
26 | | - | |
| 25 | + | |
| 26 | + | |
27 | 27 | | |
28 | | - | |
| 28 | + | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
0 commit comments