Repository navigation
Conversation
…ammars Resolves part of highlightjs#4529: rewrite three super-linear regexes so a single match is one deterministic pass. - livescript: the regex-literal body (\.|[^\\\n])*? is ambiguous on backslashes, so both alternatives can consume the same input and the engine backtracks per partition. Unrolled to (?:[^\\/\n]|\\.)*? which matches the same strings deterministically. - csharp: the raw-string body (.|\n)*? rescans the closing delimiter and lacks a guard for longer quote runs. Replaced with a tempered loop that rejects the terminator at every char plus a (?!") delimiter check. - bash: the function-name rule is greedy, so on failure the engine tries every \w partition before re-checking the () suffix. The lazy variant finds the same match without the partition search. Verification: markup output is byte-identical to main on regression probes and all 1650 tests pass. Median highlight() on 20 KB adversarial inputs: bash ~24% faster, livescript ~18% faster. Not done: the stan/perl/scilab payloads in highlightjs#4529 scale quadratically even with the flagged regexes removed - the residual cost is dominated by identifier-scanning rules that match O(n) per position, which a regex rewrite cannot fix without changing tokenization.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Relates to #4529 — fixes the three of the six live-confirmed patterns where a regex-level rewrite eliminates the super-linear behavior without changing semantics.
Changes
/(?![ *])(\\.|[^\\\n])*?/[gim]*(?=\W)— the body alternation\\.|[^\\\n]is ambiguous on backslashes (both alternatives can consume\), so failure requires backtracking through every partition. Unrolled to(?:[^\\/\n]|\\.)*?, which matches the same strings in a single deterministic pass."""("*)(?!")(.|\n)*?"""\1— the(.\|\n)*?body rescans toward the terminator at every position, and the original had no guard preventing a longer quote run from being treated as the delimiter. Replaced with a tempered loop(?:(?!"""\1(?!"))[^])*?plus(?!")after the closing delimiter.\w[\w\d_]*\s*\(\s*\)\s*\{— greedy\w*means failure tries every partition before rechecking the() {suffix. The lazy*?variant finds the identical match without the partition search (verified: match spans identical onfoo() {,fn_name ( ) {,x = bar() {},foo(1){).Verification
npm run build+npm run test: 1650 passing (was 1650 on clean main before the change).livescriptandcsharpis byte-identical tomainon regression probes (regex literals, divisions, raw strings with""""delimiters and interior quotes).highlight()on the issue's 20 KB adversarial inputs: bash ≈ 24% faster (256 ms → 194 ms), livescript ≈ 18% faster (3.2 s → 2.6 s). Numbers are noisy on this box but consistent across runs.What this PR does not do — honest scope note
stan,perl,scilabpayloads still scale quadratically even with the flagged regexes reverted or rewritten. Profiling shows the residual cost is dominated by unflagged identifier-scanning rules (e.g.stan's\w*(bernoulli|beta|…)distribution matcher,livescript'sJS_IDENT_RE) that match O(remaining-input) per position. That is linear work at the regex level — no backtracking to remove — so a regex rewrite cannot fix it without changing how the tokenizer tries modes per position. Alternatives (atomic groups, tempered loops, validation lookaheads) made things worse in benchmarks here. Flagging rather than shipping a semantic change; happy to take a second pass if a maintainer sees a route I missed.Checklist
string-rawfor csharp.