Skip to content

Fix wrong helper selection when a keyword is followed by ( - fixes #701 - #1221

Open
askalf wants to merge 3 commits into
porsager:masterfrom
sprayberry-code:fix/builder-keyword-last-occurrence
Open

askalf wants to merge 3 commits into
porsager:masterfrom
sprayberry-code:fix/builder-keyword-last-occurrence

Conversation

@askalf

@askalf askalf commented Sep 15, 2026

Copy link
Copy Markdown

Fixes #701

Summary

  • src/types.js compiles one regex per helper keyword (values, in, select, as, returning, \(, update, insert). Each was ((?:^|[\s(])KW(?:$|[\s(]))(?![\s\S]*\1) — a capture group plus a backreference \1 in the trailing "no later occurrence" lookahead.
  • A backreference matches the literal captured text, delimiters included. " in(" and " in " are different strings, so a later " in " did not cancel an earlier " in(". The stale earlier hit survived, and Builder.build (which picks the highest index) then ranked the "(select " hit above it.
  • Result: the dynamic value array was handed to the select helper → escapeIdentifiers → escape() → str.replace on a number, throwing TypeError: str.replace is not a function.
  • Fix is 4 lines in src/types.js: hoist the keyword pattern into a const and repeat it inside the lookahead instead of backreferencing the captured text, so any later occurrence counts regardless of surrounding delimiters.
  • Adds 13 regression tests beside the existing Last keyword used even with duplicate keywords test — 9 discriminate (fail on base), 3 are declared controls (pass on both arms), and 1 is the pre-existing test re-run as a baseline.
######## HEAD (fix applied) ########
PASS  0  Last keyword used even with duplicate keywords (pre-existing)
PASS  1  Last keyword used even when an earlier keyword is followed by (
PASS  2  Last keyword used even when it ends the string
PASS  3  Last keyword used when nested keywords are all spaced
PASS  4  Single keyword followed by ( is still used
PASS  5  Last keyword used for keywords other than in
PASS  6  Last keyword used when the spaced keyword comes first
PASS  7  Last keyword used with three occurrences and three delimiters
PASS  8  Last keyword used case insensitively
PASS  V1 Last keyword used when the earlier keyword is in a nested fragment
PASS  V3 Repeated keyword with identical ( delimiters (control)
PASS  V4 Last keyword used when an earlier keyword is newline delimited
PASS  V2 Last keyword used with an empty array
🎉 all passed

######## BASE (fix NOT applied) ########
PASS  0  Last keyword used even with duplicate keywords (pre-existing)
FAIL  1  Last keyword used even when an earlier keyword is followed by (  :: threw str.replace is not a function
FAIL  2  Last keyword used even when it ends the string  :: threw str.replace is not a function
PASS  3  Last keyword used when nested keywords are all spaced
PASS  4  Single keyword followed by ( is still used
FAIL  5  Last keyword used for keywords other than in  :: expected 3 != got undefined
FAIL  6  Last keyword used when the spaced keyword comes first  :: threw str.replace is not a function
FAIL  7  Last keyword used with three occurrences and three delimiters  :: threw str.replace is not a function
FAIL  8  Last keyword used case insensitively  :: threw str.replace is not a function
FAIL  V1 Last keyword used when the earlier keyword is in a nested fragment  :: threw str.replace is not a function
PASS  V3 Repeated keyword with identical ( delimiters (control)
FAIL  V4 Last keyword used when an earlier keyword is newline delimited  :: threw str.replace is not a function
FAIL  V2 Last keyword used with an empty array  :: threw syntax error at or near ")"
⚠️ 9 failed

The three rows that pass on both arms are deliberate controls: the same-delimiter spaced path, the single-keyword in( path, and the case the backreference already handled correctly. Two base failures are notably not exceptions: one returns a wrong answer (expected 3 != got undefined) and one is rejected by the server (syntax error at or near ")", from the emitted where x in )) — both stronger arguments for the fix than a build-time TypeError.

Decisions

src/types.js, the .map() that compiles builders:

-}).map(([x, fn]) => ([new RegExp('((?:^|[\\s(])' + x + '(?:$|[\\s(]))(?![\\s\\S]*\\1)', 'i'), fn]))
+}).map(([x, fn]) => {
+  const keyword = '(?:^|[\\s(])' + x + '(?:$|[\\s(])'
+  return [new RegExp(keyword + '(?![\\s\\S]*' + keyword + ')', 'i'), fn]
+})

The intent of the regex was "use the last occurrence of the keyword". Repeating the keyword pattern in the lookahead expresses that directly; the backreference expressed the narrower and unintended "no later occurrence with identical delimiters". The capture group is dropped because nothing else read it.

Minimality: one expression, no behaviour added, no new dependency, no version bump, no changelog. It preserves the existing structure — still one regex per keyword, still consumed unchanged by Builder.build.

Alternatives rejected:

  • Make Builder.build re-scan for the last match instead of relying on the lookahead — larger change to the hot path, and the lookahead is already the designed mechanism.
  • Use a global regex with lastIndexOf-style scanning — changes search() semantics for all eight keywords and risks lastIndex statefulness on shared regex objects.
  • Special-case in — the defect is generic to every keyword (values shows the same index shift); special-casing would leave the others wrong.
  • Also fix (a,b,c) IN (${...}) — rejected as out of scope: different root cause, a separate '\\(' : select builder legitimately out-ranks in there. That is a separate bug and belongs in its own PR.

Not run: npm run test:esm/test:cjs/test:deno as a whole (the bootstrap needs createdb/psql to provision roles, unavailable in my environment); the thirteen tests were instead run end-to-end against a real PostgreSQL-wire-protocol backend directly, so the t() harness wrapper itself and the transpiled cjs/deno copies are unverified here — the changed expression is plain ES2020 so no transpile-specific risk is expected, but CI should confirm.

AI assistance: this bug was found and the fix and tests were drafted with AI tooling in my workflow; the tests and checks above were executed as pasted. I'm responsible for the change and will handle review feedback.

…rsager#701

The builder keyword regexes used a backreference (\1) in their trailing
negative lookahead, so the 'no later occurrence' assertion only rejected
a repeat that matched the SAME delimiters as the first hit. ' in(' and
' in ' are different text, so a later ' in ' did not cancel an earlier
' in('. Builder.build picks the keyword with the highest index, so
'where x in(select ... where x in ${...}' kept the leading in(, ranked
the '(select ' hit above it, and handed the array to the select helper,
which threw TypeError: str.replace is not a function.

Repeat the keyword pattern inside the lookahead instead of
backreferencing the captured text, so any later occurrence counts
regardless of which delimiters surround it.
The single-keyword test built `x in((1,2))` - the template supplies the
( and the in builder emits its own ( ) - so the operand was a row
constructor and Postgres rejected it with "operator does not exist:
integer = record" on both arms. Compare a row constructor on both sides
instead, which keeps the in( delimiter the test exists to cover.

Also cover the cases the fix changes that no test pinned: a keyword
other than in, the reverse ordering, three occurrences with three
delimiters, and uppercase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

WHERE IN throws TypeError: str.replace is not a function

1 participant