[minleejae]

Purpose

MySQL's LOAD DATA and LOAD XML failed to parse, so a script that loads CSV or XML files
stopped at its first load:

LOAD DATA LOCAL INFILE '/tmp/a.csv' REPLACE INTO TABLE db1.t PARTITION (p0, p1)
    CHARACTER SET utf8mb4
    FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
    LINES TERMINATED BY '\n'
    IGNORE 1 LINES
    (a, @b) SET c = @b + 1, d = DEFAULT;

LOAD XML LOCAL INFILE '/tmp/a.xml' INTO TABLE t ROWS IDENTIFIED BY '<row>';

LOAD DATA INFILE failed at INFILE, because LOAD DATA only read BigQuery's
LOAD DATA {INTO | OVERWRITE}. Any other LOAD x was DuckDB's LOAD extension, so
LOAD XML INFILE … failed at INFILE, after LOAD XML had been read as an extension named
xml.

SELECT … INTO OUTFILE uses the same FIELDS and LINES clauses, and they differed from
MySQL's grammar there too:

• A bare FIELDS or LINES, which MySQL rejects, was accepted, and a bare LINES was dropped
  from the output: SELECT a INTO OUTFILE 'f' LINES FROM t printed as
  SELECT a INTO OUTFILE 'f' FROM t.
• Subclauses in another order, which MySQL accepts, were rejected:
  FIELDS ESCAPED BY '\\' TERMINATED BY ','.
• CHARSET and a quoted character set name (CHARACTER SET 'utf8mb4') were rejected.

Current design (As-is)

• LoadDataStatement models BigQuery's LOAD DATA; it shares only the target table with
  MySQL's statement. Its deparse printed the OPTIONS and FROM FILES expressions with
  toString(), bypassing ExpressionDeParser.
• LOAD x went to ExtensionStatement unless the second word was DATA.
• MySqlSelectIntoClause held the FIELDS/LINES subclauses as seven flat fields, filled by
  three OUTFILE-only productions that allowed only the documented order and an empty clause.
• LoadDataStatement and ExportDataStatement had two copies of the WITH CONNECTION
  production.

Target design (To-be)

• MySqlLoadDataStatement (in statement.load, next to BigQuery's LoadDataStatement)
  models both statements of the
  MySQL 8.4 grammar:
  getFormat() (DATA or XML), getLockOption() (LOW_PRIORITY or CONCURRENT),
  isLocal(), getFileName(), getDuplicateHandling() (REPLACE or IGNORE, reusing
  CreateTable.DuplicateHandling of CREATE TABLE … SELECT), getTable(), getPartitions(),
  getCharacterSet() / isUseCharsetKeyword(), getRowsIdentifiedBy(), getFileFormat(),
  getIgnoreCount() / getIgnoreUnit() (LINES or ROWS), getColumns() (columns and user
  variables; empty for ()) and getUpdateSets() (the SET clause as UpdateSets, one column
  and one value each). As in MySQL's grammar, LOAD DATA and
  LOAD XML take the same clauses in
  the same order.
• MySqlFileFormat (in statement) is the one model of the {FIELDS | COLUMNS} and
  LINES clauses, and MySqlFileFormat() is the one production, shared by LOAD DATA and
  SELECT … INTO OUTFILE. Each clause needs at least one subclause. Like MySQL's
  field_term_list and line_term_list, the subclauses may come in any order and repeat. Like
  the server, the format keeps the last value of each and prints them in the documented order:
  FIELDS ESCAPED BY '\\' TERMINATED BY ',' prints as FIELDS TERMINATED BY ',' ESCAPED BY '\\'.
  An OPTIONALLY ENCLOSED BY stays optional after a later ENCLOSED BY, as in the server.
• MySqlSelectIntoClause gains getFileFormat(). Its existing getters and setters
  (getFieldsTerminatedBy(), setFieldsOptionallyEnclosed(), getFieldsKeyword(), …) stay, and
  read and write the shared format. MySqlSelectIntoClause.FieldsKeyword stays and maps to
  MySqlFileFormat.FieldsKeyword. CHARACTER SET also takes CHARSET (isUseCharsetKeyword())
  and a quoted name, with a production shared with LOAD DATA. The clause now records its source
  range, as every parsed object should since fix: give every parsed object a node spanning its source #2870.
• Dispatch. LOAD {DATA | XML} [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE is MySQL's
  statement, in the MySQL and MariaDB dialects and without a dialect. BigQuery's
  LOAD DATA INTO, DuckDB's LOAD httpfs, PostgreSQL's LOAD 'auto_explain' and LOAD xml
  keep their reading. The other dialects keep rejecting the MySQL forms. No token is added;
  INFILE and CONCURRENT remain usable as names.
• Behaviour changes. A bare FIELDS, COLUMNS or LINES after INTO OUTFILE is now a
  parse error, as in MySQL. IGNORE n LINES accepts at most 2147483647, the limit of MySQL's
  grammar.
• Wiring. StatementVisitor (default methods), StatementVisitorAdapter (target table,
  column list, SET clause), StatementDeParser (one appendTo shared with toString(); the
  column list and the SET clause go through ExpressionDeParser), TablesNamesFinder (the
  target table and the tables of SET subqueries), feature analysis (MODIFIES_DATA) and
  validation (the existing loadData feature, now in the MySQL and MariaDB capabilities).
• BigQuery clean-ups in the same area. LOAD DATA's OPTIONS and FROM FILES expressions
  now go through ExpressionDeParser, like EXPORT DATA's, and both statements share one
  WITH CONNECTION production.

Not included:

• The bulk-load extensions in MySQL 8.4's grammar that the LOAD DATA page does not document
  (FROM, URL, S3, COUNT n, IN PRIMARY KEY ORDER, COMPRESSION =, PARALLEL =,
  MEMORY =, ALGORITHM = BULK).
• Hexadecimal and bit-value separators (TERMINATED BY 0x2c, X'2c', b'0'): the released
  MySqlSelectIntoClause getters return StringValue.
• := in the SET clause and CHAR SET, which UPDATE and column types do not read either.
• LOAD INDEX INTO CACHE.

Validation

• 92 new test cases: MySqlLoadDataStatementTest (66: every clause with the MySQL and MariaDB
  dialects and without one, toString() and StatementDeParser output, re-parse, AST fields,
  subclauses in any order, rejected forms, other dialects and other LOAD readings, statement
  boundaries, visitors, TablesNamesFinder, feature analysis, source ranges, building in code),
  MySqlFileFormatTest (18: INTO OUTFILE forms, the delegating getters and setters, rejected
  bare clauses), LoadDataValidatorTest (3), StructuredStatementVisitorTest (+4: BigQuery and
  MySQL LOAD DATA through a custom ExpressionDeParser) and StatementValidatorTest (+1).
  ASTNodeCoverageTest also checks the source ranges of four LOAD and INTO OUTFILE
  statements.

• ./gradlew spotlessApply cleanTest check jar: 13,027 tests, 0 failures, 25 skipped; the grammar
  has no JavaCC choice conflicts; Checkstyle, PMD and SpotBugs pass.

• Compared with master (a30a67920) on the DDL/DCL survey corpora and on the non-DDL
  statement corpora (MySQL and PostgreSQL, each with its dialect and without one): the three
  MySQL LOAD DATA / LOAD XML statements are fixed with the MySQL dialect and without one, with
  no regressions and no other changed result.

• MySQL 8.4.11 accepts all 64 statements the tests accept for the MySQL dialect, 55 of which master rejects. They fail only when run, mostly with --secure-file-priv (1290) or Loading local data is disabled (3948). This includes the repeated and reordered FIELDS/LINES subclauses, ROWS IDENTIFIED BY in LOAD DATA, FIELDS/LINES in LOAD XML, IGNORE 2147483647 LINES, CHARACTER SET 'utf8mb4' and an empty column list. It reports a syntax error (1064) for all 34 statements the tests reject, including bare FIELDS/LINES after INTO OUTFILE, IGNORE 2147483648 LINES, LOAD DATA LOCAL LOW_PRIORITY … and clauses out of order.

AI assistance: prepared with Claude Code.