Fix: parser rejects formulas with Excel's internal prefixes (#1655)

CHANGELOG.md

@@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 ### Fixed
 
 - Fixed the `EDATE` function returning the 28th of February instead of the 29th when the shifted date falls in February of a leap year (e.g. `=EDATE(DATE(2028, 1, 31), 1)` now returns `2028-02-29` instead of `2028-02-28`). [#1781](https://github.com/handsontable/hyperformula/pull/1781)
+- Fixed the parser rejecting formulas with the function-name prefixes that Excel writes into `.xlsx` files (e.g. `_xlfn.XLOOKUP`), which are now ignored and left out of `getCellFormula()`. [#1655](https://github.com/handsontable/hyperformula/issues/1655)
 
 ## [3.5.0] - 2026-10-09
 

docs/guide/file-import.md

@@ -23,6 +23,27 @@ To import CSV files, use a third-party [CSV parser](https://www.npmjs.com/search
 
 To import XLSX files, use a third-party [XLSX parser](https://www.npmjs.com/search?q=xlsx) (e.g., [ExcelJS](https://www.npmjs.com/package/exceljs) or [xlsx](https://www.npmjs.com/package/xlsx)). Then pass the result to HyperFormula as a JavaScript array.
 
+### Excel's internal function prefixes
+
+Excel stores some function names with an internal prefix. It marks every function added after Excel 2007 this way, so a file can contain `_xlfn.IFS(...)` where the user typed `IFS(...)`. The prefixes are an artifact of how Excel saves the file, not part of the function name.
+
+Whether the prefixes reach your code depends on the parser you use: ExcelJS, for example, passes them through.
+
+HyperFormula ignores these prefixes, so you can pass the formula straight to the engine:
+
+| Prefix | Example |
+| --- | --- |
+| `_xlfn.` | `=_xlfn.IFS(A1>B1,"Pass",A1<=B1,"Fail")` |
+| `_xlfn._xlws.` | `=_xlfn._xlws.FILTER(A1:A9,B1:B9>1)` |
+| `_xlws.` | `=_xlws.SORT(A1:A9)` |
+| `_xludf.` | `=_xludf.MY_FUNCTION()` |
+
+The `_xlpm.` prefix marks a `LAMBDA`/`LET` parameter name. HyperFormula does not support `LAMBDA` or `LET`, with or without the prefixes: a `LET` formula returns `#NAME?`, and a `LAMBDA` called in place, such as `=_xlfn.LAMBDA(_xlpm.x,_xlpm.x+1)(1)`, is a parsing error.
+
+[`getCellFormula()`](../api/classes/hyperformula.md#getcellformula) returns the formula without the prefix, so `=_xlfn.IFS(A1>B1,"Pass",A1<=B1,"Fail")` reads back as `=IFS(A1>B1,"Pass",A1<=B1,"Fail")`.
+
+A prefix does not add a function. If HyperFormula does not support the function itself, the cell holds a `#NAME?` error. See the [list of supported functions](built-in-functions.md).
+
 ### Example: Import XLSX files in Node
 
 This example uses [ExcelJS](https://www.npmjs.com/package/exceljs) to import XLSX files into HyperFormula.

hyperformula/dev-docs/PARSER.md

@@ -20,7 +20,8 @@
 - **The lexer is language-dependent.** Function names, the argument separator, the decimal separator, and error literals all come from the config and the translation package. Never hard-code an English name or an English-locale separator. See [`I18N.md`](I18N.md).
 - **Parsing and unparsing move together.** A grammar change that `Unparser.ts` does not learn about breaks round-tripping: the formula parses, the engine calculates, and `getCellFormula` returns something the user never typed. Add a round-trip test for every grammar change.
 - **A function name can look like a cell reference.** `HEX2DEC` contains the cell reference `HEX2`, and `LOG10` is one. A new function with such a name needs a parse test and a round-trip test.
-- **The cache is load-bearing.** `ParserWithCaching` does not key on the formula string. `computeHashFromTokens` hashes the lexer tokens with each reference in the AST's address form — relative parts as offsets from the formula's address, `$` parts absolute — and every function name except `OFFSET` made canonical (`OFFSET` has its own token and is hashed as typed); rewritten ASTs are stored and fetched under `computeHashFromAst`. The two must give the same key for the same formula, or `fetchCachedResultForAst` throws `There is no AST with such key in the cache`. A new token kind must hash to the same text in both: `computeHashFromTokens` falls back to the raw token image, so it needs its own branch whenever that image is not canonical, and `computeHashOfAstNode` needs a case for every new AST node type. A token that carries an address must be hashed in that address form, as the existing reference branches do with `cellAddressFromString` and `hash(true)`, never by its raw text — otherwise the same text in another cell reuses an AST built for the first. Anything else the parse result depends on must be in the key too.
+- **The cache is load-bearing.** `ParserWithCaching` does not key on the formula string. `computeHashFromTokens` hashes the lexer tokens with each reference in the AST's address form — relative parts as offsets from the formula's address, `$` parts absolute — and every function name except `OFFSET` made canonical (`OFFSET` has its own token and is hashed as typed, less any prefix described below); rewritten ASTs are stored and fetched under `computeHashFromAst`. The two must give the same key for the same formula, or `fetchCachedResultForAst` throws `There is no AST with such key in the cache`. A new token kind must hash to the same text in both: `computeHashFromTokens` falls back to the raw token image, so it needs its own branch whenever that image is not canonical, and `computeHashOfAstNode` needs a case for every new AST node type. A token that carries an address must be hashed in that address form, as the existing reference branches do with `cellAddressFromString` and `hash(true)`, never by its raw text — otherwise the same text in another cell reuses an AST built for the first. Anything else the parse result depends on must be in the key too.
+- **A function name can carry an Excel internal prefix.** Excel writes `_xlfn.`, `_xlws.` and similar prefixes into `.xlsx` files (`EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN` in `parser-consts.ts`). The `ProcedureName` and `OffsetProcedureName` tokens accept one, and `canonicalProcedureNameFromToken` and `canonicalOffsetProcedureNameFromToken` drop it. Read a function name from a token through those helpers, never from the raw image.
 - **Address arithmetic lives in the `*Address.ts` files.** Do not reimplement relative-to-absolute conversion at a call site.
 - **A parse failure is a value.** It produces a `ParsingErrorVertex` in the dependency graph, not a thrown exception. One bad formula must not take down the engine.
 

hyperformula/src/parser/FormulaParser.ts

@@ -69,6 +69,7 @@ import {
   ArrayLParen,
   ArrayRParen,
   BooleanOp,
+  canonicalProcedureNameFromToken,
   CellReference,
   ColumnRange,
   ConcatenateOp,
@@ -157,8 +158,7 @@ export class FormulaParser extends EmbeddedActionsParser {
    */
   private procedureExpression: AstRule = this.RULE('procedureExpression', () => {
     const procedureNameToken = this.CONSUME(ProcedureName) as ExtendedToken
-    const procedureName = procedureNameToken.image.toUpperCase().slice(0, -1)
-    const canonicalProcedureName = this.lexerConfig.functionMapping[procedureName] ?? procedureName
+    const canonicalProcedureName = canonicalProcedureNameFromToken(procedureNameToken.image, this.lexerConfig.functionMapping)
     const args: Ast[] = []
 
     let argument = this.SUBRULE(this.booleanExpressionOrEmpty)

hyperformula/src/parser/LexerConfig.ts

@@ -9,6 +9,7 @@ import {ParserConfig} from './ParserConfig'
 import {
   ALL_WHITESPACE_PATTERN,
   COLUMN_REFERENCE_PATTERN,
+  EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN,
   NON_RESERVED_CHARACTER_PATTERN,
   ODFF_WHITESPACE_PATTERN,
   RANGE_OPERATOR,
@@ -47,7 +48,58 @@ export const RangeSeparator = createToken({ name: 'RangeSeparator', pattern: new
 export const ColumnRange = createToken({ name: 'ColumnRange', pattern: new RegExp(`${COLUMN_REFERENCE_PATTERN}${RANGE_OPERATOR}${COLUMN_REFERENCE_PATTERN}`) })
 export const RowRange = createToken({ name: 'RowRange', pattern: new RegExp(`${ROW_REFERENCE_PATTERN}${RANGE_OPERATOR}${ROW_REFERENCE_PATTERN}`) })
 
-export const ProcedureName = createToken({ name: 'ProcedureName', pattern: new RegExp(`([${UNICODE_LETTER_PATTERN}][${NON_RESERVED_CHARACTER_PATTERN}]*)\\(`) })
+export const ProcedureName = createToken({ name: 'ProcedureName', pattern: new RegExp(`(?:${EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN})?([${UNICODE_LETTER_PATTERN}][${NON_RESERVED_CHARACTER_PATTERN}]*)\\(`) })
+
+const excelInternalFunctionPrefixRegexp = new RegExp(`^(?:${EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN})`)
+const UNDERSCORE_CHAR_CODE = '_'.charCodeAt(0)
+
+/**
+ * Strips one of Excel's internal function-name prefixes (see EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN)
+ * off the front of a token image, if present.
+ *
+ * Every prefix starts with `_`, and this function is called on every ProcedureName and
+ * OffsetProcedureName token in every parse, so the common case — a name with no prefix at all — takes
+ * a plain character check instead of always paying for the regex match-and-fail.
+ *
+ * @param {string} nameWithoutTrailingParen - a token image with any trailing `(` already removed
+ */
+function stripExcelInternalFunctionPrefix(nameWithoutTrailingParen: string): string {
+  if (nameWithoutTrailingParen.charCodeAt(0) !== UNDERSCORE_CHAR_CODE) {
+    return nameWithoutTrailingParen
+  }
+  return nameWithoutTrailingParen.replace(excelInternalFunctionPrefixRegexp, '')
+}
+
+/**
+ * Reads the canonical function name out of a ProcedureName token.
+ *
+ * The token image spans the whole match, so it carries the trailing opening parenthesis and, for a
+ * formula imported from an .xlsx file, one of the prefixes Excel prepends when it serializes a
+ * workbook. Dropping both here is what lets `_xlfn.IFS(A1)` resolve to the same function as `IFS(A1)`.
+ *
+ * The prefix is removed before the name is upper-cased, because the prefixes are matched in lower
+ * case only.
+ *
+ * @param {string} image - image of the ProcedureName token, for example `_xlfn.IFS(`
+ * @param {Record<string, string>} functionMapping - maps a translated function name to its canonical English name
+ */
+export function canonicalProcedureNameFromToken(image: string, functionMapping: Record<string, string>): string {
+  const procedureName = stripExcelInternalFunctionPrefix(image.slice(0, -1)).toUpperCase()
+  return functionMapping[procedureName] ?? procedureName
+}
+
+/**
+ * Strips an Excel internal function-name prefix off an OffsetProcedureName token's image.
+ *
+ * OffsetProcedureName has no trailing parenthesis to remove (unlike ProcedureName — OFFSET's grammar
+ * rule consumes `(` separately) and no translation lookup (the token pattern already embeds the
+ * localized OFFSET name), so it needs only the prefix stripped, not the full canonicalization above.
+ *
+ * @param {string} image - image of the OffsetProcedureName token, for example `_xlfn.OFFSET`
+ */
+export function canonicalOffsetProcedureNameFromToken(image: string): string {
+  return stripExcelInternalFunctionPrefix(image)
+}
 
 const cellReferenceMatcher = new CellReferenceMatcher()
 export const CellReference = createToken({
@@ -93,7 +145,22 @@ export const buildLexerConfig = (config: ParserConfig): LexerConfig => {
   const ArrayRowSeparator = createToken({name: 'ArrayRowSep', pattern: config.arrayRowSeparator})
   const ArrayColSeparator = createToken({name: 'ArrayColSep', pattern: config.arrayColumnSeparator})
   const NumberLiteral = createToken({ name: 'NumberLiteral', pattern: new RegExp(`(([${config.decimalSeparator}]\\d+)|(\\d+([${config.decimalSeparator}]\\d*)?))(e[+-]?\\d+)?`) })
-  const OffsetProcedureName = createToken({ name: 'OffsetProcedureName', pattern: new RegExp(offsetProcedureNameLiteral, 'i') })
+  // OFFSET has its own token because it has its own grammar rule, so it needs the prefix handling of
+  // ProcedureName repeated here. The 'i' flag exists for the translated OFFSET name (pre-dates this
+  // prefix support) and applies to the whole pattern, so unlike ProcedureName it also makes the prefix
+  // case-insensitive: `_XLFN.OFFSET(...)` is accepted here where `_XLFN.SUM(...)` is a parsing error.
+  // Left as-is rather than given a dedicated case-sensitive-prefix matcher: OFFSET predates the OOXML
+  // cutoff this whole prefix scheme exists for, so Excel can never actually emit a prefixed OFFSET call
+  // in any case — the asymmetry has no reachable real input, only hand-typed formulas (see the
+  // "an upper-cased prefix is accepted on OFFSET, unlike everywhere else" test).
+  // Unlike ProcedureName, this token does not include the '(' itself, so the prefixed form must look
+  // ahead for one: otherwise it would claim the start of a named expression such as `_xlfn.OFFSET_RATE`,
+  // which lexes as a single NamedExpression without prefix support. The unprefixed form keeps its
+  // pre-existing pattern unchanged.
+  const OffsetProcedureName = createToken({
+    name: 'OffsetProcedureName',
+    pattern: new RegExp(`(?:${EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN})${offsetProcedureNameLiteral}(?=(?:${whitespaceTokenRegexp.source})?\\()|${offsetProcedureNameLiteral}`, 'i'),
+  })
 
   let ArgSeparator: TokenType
   let inject: TokenType[]

hyperformula/src/parser/ParserWithCaching.ts

@@ -19,6 +19,8 @@ import {Cache} from './Cache'
 import {FormulaLexer, FormulaParser, ExtendedToken} from './FormulaParser'
 import {
   buildLexerConfig,
+  canonicalOffsetProcedureNameFromToken,
+  canonicalProcedureNameFromToken,
   CellReference,
   ColumnRange,
   LexerConfig,
@@ -239,9 +241,10 @@ export class ParserWithCaching {
           hash = hash.concat(cellAddress.hash(true))
         }
       } else if (tokenMatcher(token, ProcedureName)) {
-        const procedureName = token.image.toUpperCase().slice(0, -1)
-        const canonicalProcedureName = this.lexerConfig.functionMapping[procedureName] ?? procedureName
+        const canonicalProcedureName = canonicalProcedureNameFromToken(token.image, this.lexerConfig.functionMapping)
         hash = hash.concat(canonicalProcedureName, '(')
+      } else if (tokenMatcher(token, this.lexerConfig.OffsetProcedureName)) {
+        hash = hash.concat(canonicalOffsetProcedureNameFromToken(token.image))
       } else if (tokenMatcher(token, ColumnRange)) {
         const [start, end] = token.image.split(':')
         const startAddress = columnAddressFromString(start, baseAddress, this.resolveSheetReference)

hyperformula/src/parser/parser-consts.ts

@@ -22,6 +22,31 @@ export const ROW_REFERENCE_PATTERN = `(${SHEET_NAME_PATTERN})?\\${ABSOLUTE_OPERA
 export const R1C1_CELL_REFERENCE_PATTERN = '[rR][0-9]*[cC][0-9]*'
 export const CELL_REFERENCE_WITH_NEXT_CHARACTER_PATTERN = `(${CELL_REFERENCE_PATTERN})[^${NON_RESERVED_CHARACTER_PATTERN}]`
 
+/**
+ * Prefixes that Excel prepends to a function name when it serializes a workbook.
+ *
+ * Excel marks every function added after the original OOXML specification (~Excel 2007) with one
+ * of these prefixes in the stored XML, so a file may contain `_xlfn.IFS(...)` where the user typed
+ * `IFS(...)`. They are serialization artifacts rather than a part of the function name, and other
+ * spreadsheet engines drop them on import, so HyperFormula accepts and ignores them too.
+ *
+ * | Prefix         | Meaning                                                  |
+ * |----------------|----------------------------------------------------------|
+ * | `_xlfn.`       | function newer than the OOXML specification               |
+ * | `_xlfn._xlws.` | as above, and callable only in a worksheet                |
+ * | `_xlws.`       | callable only in a worksheet                             |
+ * | `_xlpm.`       | `LAMBDA`/`LET` parameter name                            |
+ * | `_xludf.`      | user-defined function (a `LAMBDA` stored in Name Manager) |
+ *
+ * The alternatives are ordered longest-first so that `_xlfn._xlws.` is consumed whole instead of
+ * leaving `_xlws.` behind. Excel always writes them in lower case, so wherever this pattern is used
+ * to recognize a *function call* (the `ProcedureName` token in LexerConfig.ts) it is matched in lower
+ * case only: an upper-cased spelling there is not something Excel can produce. The one exception is
+ * the `OffsetProcedureName` token, which reuses this pattern inside an already case-insensitive regex
+ * (for the translated OFFSET name) and so matches an upper-cased prefix too — see its comment.
+ */
+export const EXCEL_INTERNAL_FUNCTION_PREFIX_PATTERN = '_xlfn\\._xlws\\.|_xlfn\\.|_xlws\\.|_xlpm\\.|_xludf\\.'
+
 export const NAMED_EXPRESSION_PATTERN = `[${UNICODE_LETTER_PATTERN}_][${NON_RESERVED_CHARACTER_PATTERN}]*`
 
 export const ALL_DIGITS_ARRAY = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9']