Issue #4189 : SQL File Output, option to skip INSERT statements and select fields by jbs-atolcd · Pull Request #8782 · apache/hop

jbs-atolcd · 2026-10-07T10:48:48Z

Adds two options to the SQL File Output transform:

"Do not add INSERT statements": only available when "Add create table statement" is checked, so you can generate a ddl-only script (CREATE TABLE) without dumping every row.
"Specify table fields" in the Content tab, with a field grid (name / rename) and a Get fields button. When enabled, only the selected fields end up in the CREATE TABLE and INSERT statements, renamed if needed.

Format and encoding are now grouped in the Content tab. Existing pipelines keep the current behaviour (both options off by default).

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004
In any other case, please file an Apache Individual Contributor License Agreement.

mattcasters · 2026-10-07T17:39:47Z

+    // Do not create insert statements
+    wlDoNotAddInsertStmt = new Label(wFileName, SWT.RIGHT);
+    wlDoNotAddInsertStmt.setText(
+        BaseMessages.getString(PKG, "SQLFileOutputDialog.DoNotAddInsertStatements.Tooltip")); // /!\


[bug] The checkbox caption is loaded from SQLFileOutputDialog.DoNotAddInsertStatements.Tooltip instead of .Label. Both bundles define a short label ("Do not add 'insert into' statements"), but the dialog puts the long tooltip in the right-aligned label, so it is clipped next to the checkbox. The // /!\ comment looks left in by mistake. That same tooltip, and the user manual, also say this can emit TRUNCATE, but activateTruncate() clears and disables truncate whenever "Add create table" is checked, and this option is only enabled in that case, so the two cannot be combined.

Suggestion: Use the .Label key for setText and keep the tooltip on setToolTipText. Drop the truncate claim from the tooltip and the manual, unless truncate is allowed to stay on together with this option.

Ah sorry , the /!\ was a reminder for myself to switch to the .Label key, and I forgot to come back to it. Fixed, and I ve removed the TRUNCATE mention from the tooltip and the docs

mattcasters · 2026-10-07T17:39:47Z

+  private List<SQLFileOutputField> sqlFileOutputFields = new ArrayList<>();
+
+  @HopMetadataProperty(key = "specifyFields")
+  private boolean specifyFields;


[bug] specifyFields and the field list are only applied in SQLFileOutput.processRow(). getSqlStatements() still builds CREATE TABLE from the full incoming row, and the dialog SQL button calls that after getInfo(), so it can show and run DDL for columns the user excluded or renamed. check() still requires every incoming field to exist in the target table under its original name, so a valid subset or rename is reported as "Fields in input stream, not found in output table". analyseImpact() likewise records a write for every incoming field.

Suggestion: When specifyFields is set, build the same selected row meta as processRow (lookup by name, apply rename) and use it in getSqlStatements(), check(), and analyseImpact(). Table Output already does this for its SQL button and check().

Right.. That was only handled at runtime. i ve added getSqlRowMeta() in the meta, used by processRow, getSqlStatements and check. analyseImpact applies the same selection and rename, but skips unknowns fields instead of failing. Added unit tests too.

mattcasters · 2026-10-07T17:39:47Z

+            r,
+            wFields,
+            1,
+            new int[] {1, 2}, // Target column indexes (1 = Name, 2 = Rename)


[suggestion] getFieldsFromPrevious(..., new int[] {1, 2}, ...) copies the incoming name into both the Name column and the Rename column. Rename is documented as optional, and a non-empty rename is always written into the generated column name, so Get fields freezes every SQL column to the current stream name. Editing Name afterwards changes which stream field is read without changing the SQL column.

Suggestion: Pass new int[] {1} so only the Name column is filled and Rename stays empty unless the user sets it.

mattcasters · 2026-10-07T17:39:47Z

      data.outputRowMeta = getInputRowMeta().clone();
      meta.getFields(data.outputRowMeta, getTransformName(), null, null, this, metadataProvider);
-      data.insertRowMeta = getInputRowMeta().clone();
+      if (meta.isSpecifyFields()


[suggestion] If "Specify table fields" is checked and the grid is empty, this condition fails and the transform silently uses every incoming field for CREATE and INSERT. That contradicts the option and the manual ("only the selected fields").

Suggestion: When specifyFields is true, honor the list even if it is empty: fail with a clear error, or generate no columns. Use the full input row only when the option is off.

Agreed, an empty selection now fails with an explicit error, at runtime and in check / the SQL button.

mattcasters · 2026-10-07T17:39:47Z

-          data.db.getSqlOutput(schemaName, tableName, data.insertRowMeta, r, meta.getDateFormat())
-              + ";";
+      String sql = "";
+      if (!meta.isDoNotAddInsertStatements()) {


[suggestion] Inserts are skipped whenever doNotAddInsertStatements is true, even if createTable is false. The dialog clears the checkbox unless "Add create table" is checked, but metadata injection or a hand-edited pipeline still takes this branch and writes an empty file (no CREATE, no INSERT) while linesOutput keeps increasing. The manual says the option is only available together with create table.

Suggestion: Skip INSERT generation only when isDoNotAddInsertStatements() and isCreateTable() are both true.

Fixed, inserts are now only skiped when create table is enabled too

mattcasters · 2026-10-07T17:39:47Z

+  @HopMetadataProperty(
+      key = "name",
+      injectionKey = "FIELD_NAME",
+      injectionKeyDescription = "SelectValues.Injection.FIELD_NAME")


[suggestion] injectionKeyDescription points at SelectValues.Injection.FIELD_NAME and SelectValues.Injection.FIELD_RENAME. Those keys are not in this plugin's bundle. Metadata injection resolves the description with BaseMessages.getString on this class, so the UI shows !SelectValues.Injection.FIELD_NAME!.

Suggestion: Add SQLFileOutput.Injection.FIELD_NAME and SQLFileOutput.Injection.FIELD_RENAME to the English and French message bundles and reference those keys. The same applies to the rename property below.

Copy paste leftover from Select Values, sorry. Added SQLFileOutput.Injection.FIELD_NAME / FIELD_RENAME in En and Fr.

mattcasters · 2026-10-07T17:39:47Z

+          new ColumnInfo(
+              BaseMessages.getString(PKG, "SQLFileOutputMeta.Content.RenameTo"),
+              ColumnInfo.COLUMN_TYPE_TEXT,
+              ValueMetaFactory.getValueMetaNames())


[nit] The Rename column is COLUMN_TYPE_TEXT, but it is constructed with ValueMetaFactory.getValueMetaNames(). That varargs constructor stores data-type names as combo values, and a text column never shows them. It looks like a copy of a type column.

Suggestion: Use the two-argument constructor: new ColumnInfo(label, ColumnInfo.COLUMN_TYPE_TEXT).

… and select fields Adds two options to the SQL File Output transform: - "Do not add INSERT statements": only available when "Add create table statement" is checked, so you can generate a ddl-only script (CREATE TABLE) without dumping every row. - "Specify table fields" in the Content tab, with a field grid (name / rename) and a Get fields button. When enabled, only the selected fields end up in the CREATE TABLE and INSERT statements, renamed if needed. Format and encoding are now grouped in the Content tab. Existing pipelines keep the current behaviour (both options off by default).

github-actions Bot added Documentation Integration Testing labels Oct 7, 2026

mattcasters requested changes Oct 7, 2026

View reviewed changes

jbs-atolcd force-pushed the main branch from a708e25 to 6a80bbc Compare October 8, 2026 09:14

Conversation

jbs-atolcd commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jbs-atolcd commented Oct 7, 2026 •

edited

Loading