[INJECT-03] Implement the allocation-free SQL tokenizer and reusable state

[chrisbbreuer]

Objective

Implement SQL token kinds, positions, delimiters, comments, numbers, variables, operators, dialect flags, streaming state, reset, statistics, and custom lookup callbacks using caller-owned or fixed storage.

Deliverables

• Implement the complete scoped behavior without Git submodules.
• Keep public APIs explicit about ownership, resource limits, and errors.
• Add focused unit, integration, compatibility, fuzz, and benchmark coverage as applicable.
• Update documentation and machine-readable compatibility status.

Acceptance criteria

• The scoped implementation and tests are merged on main.
• Zig formatting, focused tests, and the repository CI suite pass.
• No unexplained compatibility skips, silent fallbacks, leaks, or unbounded hostile-input paths remain.
• Benchmarks or security evidence is attached when this affects a hot or untrusted path.

Relationships

• Parent: INJECT-00

Blocked by

• INJECT-01
• INJECT-02

Blocks

• INJECT-04
• INJECT-05

Baselines

• Zig 0.17-dev resolved and locked through Pantry.
• ModSecurity 3.0.16, Coraza 3.7.0, CRS 4.28.0, and libinjection 4.0.0 where applicable.

[chrisbbreuer]

Initial implementation slice: 6bc894f adds reusable fixed-capacity SQL lexical state, byte-offset tokens, dialect quoting/comments/numbers/variables, and explicit truncation/unterminated reporting. CI passed: https://github.com/zig-utils/zig-injection/actions/runs/29892766547. The issue remains open for full upstream differential parity.