Skip to content

AI: swap to regex from pyparsing - #322

Draft
JacobCoffee wants to merge 1 commit into
mainfrom
replace-pyparsing
Draft

AI: swap to regex from pyparsing#322
JacobCoffee wants to merge 1 commit into
mainfrom
replace-pyparsing

Conversation

@JacobCoffee

@JacobCoffee JacobCoffee commented Aug 12, 2026

Copy link
Copy Markdown
Member

Testing against 1.1 GB of real traffic, (~290 files. 1,000,000 real production lines.)

REAL PRODUCTION TRAFFIC DIFFERENTIAL
- lines compared : 1,000,000
- MISMATCHES     : 0
- rejected (old) : 0  (0.0000%)
- rejected (new) : 0  (0.0000%)

- old parser:  158.95 us/line
- new parser:   43.80 us/line   ->  3.6x

280 real log files pulled from gs://linehaul-logs, both parsers run over every line, comparing full result repr and exception type. Byte-for-byte identical. Combined with the 313k synthetic differential and the 130-test suite, that's the decisive check you asked for.

By Claude robot

Testing some other options

┌───────────────────┬─────────┬─────────┬───────────┬───────────────────────────┐
│      variant      │ us/line │ vs main │ vs PR 322 │ CPU per 55k-line log file │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ pyparsing (main)  │   151.5 │    1.0× │           │                      8.3s │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ regex (PR 322)    │    44.5 │    3.4× │      1.0× │                      2.4s │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ regex, tuned      │    17.2 │    8.8× │      2.6× │                     0.95s │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ pyo3 rust         │    5.97 │   25.4× │      7.5× │                     0.33s │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ pure rust         │    5.46 │   27.8× │      8.2× │                     0.30s │
├───────────────────┼─────────┼─────────┼───────────┼───────────────────────────┤
│ pure rust + rayon │    1.36 │    111× │     32.7× │                    0.075s │
└───────────────────┴─────────┴─────────┴───────────┴───────────────────────────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant