Repository navigation
Fix: parser rejects formulas with Excel's internal prefixes (#1655) - #1771
marcin-kordas-hoc wants to merge 11 commits into
Conversation
Excel marks every function added after the original OOXML specification with an
internal prefix when it serializes a workbook, so an imported file can contain
`_xlfn.IFS(...)` where the user typed `IFS(...)`. The ProcedureName token required
the name to start with a letter, so none of the prefixes matched and the formula
failed with "Parsing error. Redundant input, expecting EOF but found: (".
The prefixes are now an optional part of the ProcedureName and OffsetProcedureName
patterns, and are dropped when the token is turned into a function name. Doing this
in the lexer rather than by rewriting the formula string keeps the prefixes intact
wherever they are not a function name, such as inside a string literal.
FormulaParser and ParserWithCaching derive the name through one shared helper, so a
prefixed formula and its unprefixed equivalent also produce the same cache hash.
Closes #1655
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
hyperformula-docs | cc890aa | Commit Preview URL Branch Preview URL |
Oct 23 2026, 08:25 AM |
Performance comparison of head (cc890aa) vs base (d143456) |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a697a2a. Configure here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
computeHashFromTokens only special-cased ProcedureName tokens; OffsetProcedureName is a separate, uncategorized token type (OFFSET has its own grammar rule), so it fell through to the generic branch that hashes the raw token image — meaning _xlfn.OFFSET(...) and OFFSET(...) did NOT share a cache entry, contradicting this fix's own stated design goal. Verified empirically before and after (hashes differ pre-fix, match post-fix) using the parser's own tokenizeFormula(), not a separately built lexer — a second buildLexerConfig() call creates a distinct OffsetProcedureName token-type object, so testing this via an external lexer silently never exercises the real bug (fixed the same latent issue in the existing 'shares one cache entry' spec test). Also adds a cheap leading-underscore guard to canonicalProcedureNameFromToken/ canonicalOffsetProcedureNameFromToken before invoking the prefix-stripping regex, since every prefix starts with '_' and the overwhelming majority of tokens don't have one. Verified as a real, correctness-preserving improvement for the common case with a standalone microbenchmark; a reorder alternative suggested by review was tried and measured to NOT help (and slightly hurt the prefixed case), so it was not adopted. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- parser-consts.ts / LexerConfig.ts: the 'prefixes are matched in lower case only'
claim is true for ProcedureName but not for OffsetProcedureName, which reuses the
prefix pattern inside an already case-insensitive regex (the 'i' flag exists for
the translated OFFSET name, pre-dating this fix). Replaced 'harmless' with the
actual asymmetry and why it's not worth a dedicated case-sensitive-prefix matcher:
OFFSET predates the OOXML cutoff this prefix scheme exists for, so Excel can never
emit a prefixed OFFSET call in any case — there is no reachable real input, only
hand-typed formulas.
- docs/guide/file-import.md: corrected the claim that HyperFormula 'ignores' the
_xlpm. prefix. It only strips a prefix in front of a function call; _xlpm. never
appears there in a real .xlsx — it always prefixes a bare LAMBDA/LET parameter
name, which this fix deliberately does not touch (HF has no LAMBDA/LET, so such a
formula is #NAME? either way).
- LexerConfig.ts: added {type} JSDoc annotations to match the majority convention
already used elsewhere in src/parser/.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
GreenFlux
left a comment
There was a problem hiding this comment.
Looks good, thanks for the quick response!
Tobiadefami
left a comment
There was a problem hiding this comment.
Could we restrict the prefixed OFFSET token to function calls?
| // cutoff this whole prefix scheme exists for, so Excel can never actually emit a prefixed OFFSET call | ||
| // in any case — the asymmetry has no reachable real input, only hand-typed formulas (see the | ||
| // "an upper-cased prefix is accepted on OFFSET, unlike everywhere else" test). | ||
| const OffsetProcedureName = createToken({ name: 'OffsetProcedureName', pattern: new RegExp(`(?:${EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN})?${offsetProcedureNameLiteral}`, 'i') }) |
There was a problem hiding this comment.
This currently breaks an existing named-expression reference:
const engine = HyperFormula.buildFromArray([[null]], {
licenseKey: 'gpl-v3',
})
const address = {sheet: 0, row: 0, col: 0}
engine.addNamedExpression('_xlfn.OFFSET_RATE', 42)
engine.setCellContents(address, [['=_xlfn.OFFSET_RATE']])
engine.getCellValue(address) // Expected: 42The regression test passes on base c920375c, but fails on head a2185233 with:
Expected: 42
Received: #ERROR!
Parsing error: expected LParen, found '_RATE'
The new pattern recognizes _xlfn.OFFSET without requiring (, splitting the complete name into an OFFSET token and _RATE. Could we preserve the named-expression reference when there is no function call?
There was a problem hiding this comment.
Fixed in eaaab7c. The prefixed OFFSET pattern now looks ahead for an opening parenthesis (whitespace allowed, as for the unprefixed form), so _xlfn.OFFSET_RATE lexes as one named expression again and your repro returns 42. A bare =_xlfn.OFFSET now yields #NAME? instead of a parsing error. The unprefixed pattern is unchanged. Regression tests are in hyperformula-tests#55 (fecaded); I confirmed they fail on the previous head and pass now.
|
|
||
| | Prefix | Example | | ||
| | --- | --- | | ||
| | `_xlfn.` | `=_xlfn.IFS(A1>B1,"Pass","Fail")` | |
There was a problem hiding this comment.
Could we correct the IFS example here and in the getCellFormula() explanation on line 43?
The current formula has three arguments:
=_xlfn.IFS(A1>B1,"Pass","Fail")
IFS expects condition/result pairs, so this returns #N/A with “Wrong number of arguments.” Could we use:
=_xlfn.IFS(A1>B1,"Pass",A1<=B1,"Fail")
The corresponding getCellFormula() output would be:
=IFS(A1>B1,"Pass",A1<=B1,"Fail")
There was a problem hiding this comment.
Good catch, fixed in eaaab7c: both the table row and the getCellFormula() explanation now use =_xlfn.IFS(A1>B1,"Pass",A1<=B1,"Fail") / =IFS(A1>B1,"Pass",A1<=B1,"Fail").
The prefixed OffsetProcedureName pattern matched `_xlfn.OFFSET` without a following parenthesis, so it split a named expression such as `_xlfn.OFFSET_RATE` into an OFFSET token and `_RATE`, and turned a bare `=_xlfn.OFFSET` into a parsing error instead of #NAME?. The prefixed form now looks ahead for an opening parenthesis, allowing whitespace before it as the grammar does for unprefixed OFFSET. The unprefixed pattern is unchanged. Also correct the IFS example in the file-import guide: it had three arguments and evaluated to #N/A. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md
…xes in the parser guide The changelog rules ask for the public issue where one exists. The parser guide said OFFSET is hashed exactly as typed; a prefix is now dropped first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t import libraries A LAMBDA called in place is a parsing error, not #NAME?. Which libraries keep the prefixes was not measured, so the guide no longer names them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1771 +/- ##
========================================
Coverage 97.28% 97.28%
========================================
Files 207 207
Lines 16218 16230 +12
Branches 3556 3589 +33
========================================
+ Hits 15777 15789 +12
Misses 433 433
Partials 8 8
🚀 New features to boost your workflow:
|

Context
Excel writes an internal prefix in front of some function names when it saves an
.xlsxfile, for example_xlfn.IFS(...)for every function added after Excel 2007. The parser requires a function name to start with a letter, so it rejected these formulas with a parsing error. Whether a user meets this depends on the import library, because some libraries pass the prefixes through. Registering the prefixed name as a custom function does not help, because the lexer rejects the leading underscore before names are resolved.The fix:
ProcedureNametoken accepts an optional prefix:_xlfn.,_xlfn._xlws.,_xlws.,_xlpm.or_xludf..canonicalProcedureNameFromTokendrops it, so_xlfn.IFS(...)parses to the same function call asIFS(...). Prefixes are matched in lower case, as Excel writes them.OFFSEThas its own token. It accepts a prefix only when an opening parenthesis follows, so a named expression such as_xlfn.OFFSET_RATEkeeps working. Its pattern is case-insensitive for the translated name, so it also accepts an upper-case prefix.ParserWithCachingcomputes the cache hash through the same helpers, so a prefixed formula and its unprefixed form share one cache entry,OFFSETincluded.getCellFormula()andgetCellSerialized()return the formula without the prefix.The prefix is removed in the lexer, not by rewriting the formula string. A string replace, as sketched in the issue, would also change text inside string literals:
="_xlfn.IFS is a literal"would become="IFS is a literal".docs/guide/file-import.mddocuments the prefixes, andhyperformula/dev-docs/PARSER.mdtells contributors to read function names through the new helpers.How did you test your changes?
fix/hf-217-excel-internal-prefixesinhyperformula-tests(hyperformula-tests#55),unit/parser/excel-internal-function-prefixes.spec.ts: 31 tests pass. Withhyperformula/src/parser/reverted todevelop, 23 of them fail."Fail"for=_xlfn.IFS(A1>B1,"Pass",A1<=B1,"Fail")withA1= 3 andB1= 5,3for=_xlfn._xlws.SORT(A1:B1), and42for a named expression_xlfn.OFFSET_RATEset to 42.Compatibility with Excel and Google Sheets
Not applicable: the formulas behave as their unprefixed form, which this PR does not change. A prefix does not add a function:
=_xlfn.CONCAT("a","b")returns#NAME?, because HyperFormula has noCONCAT.Public API changes
None. Formulas that used to fail with a parsing error now parse.
Out of scope and follow-ups
_xlpm.x(aLAMBDAorLETparameter, not a call) is not stripped. HyperFormula supports neither function, with or without the prefixes.Types of changes
Related issues
Checklist
dev-docs/pages this change affects – or none needed it.🤖 Generated with Claude Code
Note
Medium Risk
Touches core lexer/parser tokenization and parse-cache hashing; behavior change is localized to prefixed function calls but mistakes could affect formula parsing broadly.
Overview
Fixes formulas imported from
.xlsxthat Excel serializes with internal function-name prefixes (e.g._xlfn.IFS,_xlws.SORT), which previously failed parsing because names had to start with a letter.The lexer now accepts optional
_xlfn.,_xlfn._xlws.,_xlws.,_xlpm., and_xludf.prefixes onProcedureNameand (with lookahead for() onOFFSET.canonicalProcedureNameFromToken/canonicalOffsetProcedureNameFromTokenstrip prefixes before name mapping, andParserWithCachinghashes the canonical names so prefixed and unprefixed formulas share cache entries. Stored/read-back formulas omit the prefix via the AST, without rewriting the raw string (so literals like"_xlfn.IFS"stay intact).Docs add an XLSX import section on these prefixes; CHANGELOG and PARSER.md note the behavior for contributors.
Reviewed by Cursor Bugbot for commit cc890aa. Bugbot is set up for automated code reviews on this repo. Configure here.