Skip to content

NIFI-14869: Add MSSQL MERGE-based upsert via new DatabaseDialectService - #11442

Open
Adelwageeh96 wants to merge 1 commit into
apache:mainfrom
Adelwageeh96:feature/NIFI-14869-mssql-upsert
Open

NIFI-14869: Add MSSQL MERGE-based upsert via new DatabaseDialectService#11442
Adelwageeh96 wants to merge 1 commit into
apache:mainfrom
Adelwageeh96:feature/NIFI-14869-mssql-upsert

Conversation

@Adelwageeh96

Copy link
Copy Markdown

Implement supportsUpsert() and getUpsertStatement() in MSSQLDatabaseDialectService to produce a MERGE statement for UPSERT operations on SQL Server.

  • Added new controller service: MSSQLDatabaseDialectService Provides SELECT with paging, UPSERT using MERGE, and basic ALTER/CREATE DDL generation for SQL Server 2012+.

  • Added new controller service: MSSQL2008DatabaseDialectService Extends MSSQLDatabaseDialectService to support SQL Server 2008, including SELECT with ROW_NUMBER() paging.

  • Registered both services in META-INF/services/org.apache.nifi.controller.ControllerService Enables NiFi to discover and use the new MSSQL dialect services.

  • Added unit tests:

    • TestMSSQLDatabaseDialectService.java: Verifies SQL generation and UPSERT logic for MSSQLDatabaseDialectService.
    • TestMSSQL2008DatabaseDialectService.java: Verifies paging and SELECT logic for MSSQL2008DatabaseDialectService.

Summary

NIFI-14869

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000
  • Pull request contains commits signed with a registered key indicating Verified status

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using ./mvnw clean install -P contrib-check
    • JDK 21
    • JDK 25

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for proposing this addition @Adelwageeh96.

On initial review, I think the service implementation should be limited to current versions of Microsoft SQL Server. MS SQL Server 2008 reached end of support in 2019, so that implementation should be removed.

@Adelwageeh96
Adelwageeh96 force-pushed the feature/NIFI-14869-mssql-upsert branch from 6dfc86d to 6dddce5 Compare August 30, 2026 09:02
@Adelwageeh96

Copy link
Copy Markdown
Author

Hi @exceptionfactory , thanks for the review. I've removed MSSQL2008DatabaseDialectService and its test as requested — SQL Server 2008 reached end of support in 2019, so the dialect service now only targets SQL Server 2012+.

Changes pushed to the branch (commit 6dddce5):

Removed MSSQL2008DatabaseDialectService.java
Removed TestMSSQL2008DatabaseDialectService.java
Removed its entry from META-INF/services/org.apache.nifi.controller.ControllerService
MSSQLDatabaseDialectService (SELECT with paging, UPSERT via MERGE, ALTER/CREATE DDL) and its tests are unchanged.

@Adelwageeh96
Adelwageeh96 force-pushed the feature/NIFI-14869-mssql-upsert branch from 6dddce5 to eeaed88 Compare August 30, 2026 11:04

@exceptionfactory exceptionfactory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for scoping down the changes @Adelwageeh96.

On further review, there are several additional things to adjust:

  1. The new class should be moved to the nifi-database-dialect-service module, not placed in nifi-standard-processors
  2. The test class should have all public modifiers removed since they are not necessary
  3. Strings that span more than one line should be able to use multiline strings, instead of concatenation

@Adelwageeh96
Adelwageeh96 force-pushed the feature/NIFI-14869-mssql-upsert branch from eeaed88 to 39d308c Compare September 2, 2026 11:05
@Adelwageeh96

Copy link
Copy Markdown
Author

Hello @exceptionfactory , thanks for the detailed feedback. I've made all three changes:

  1. Moved MSSQLDatabaseDialectService (and its test) into the nifi-database-dialect-service module, alongside
    StandardDatabaseDialectService, and registered it in that module's META-INF/services instead of nifi-standard-processors.
  2. Removed the unnecessary public modifiers from TestMSSQLDatabaseDialectService.
  3. Converted the multi-line MERGE assertion strings in the test to text blocks instead of + concatenation.

Latest commit: 39d308c. Ready for another look whenever you have a chance.

@Adelwageeh96
Adelwageeh96 force-pushed the feature/NIFI-14869-mssql-upsert branch from 39d308c to 7765f2f Compare September 3, 2026 07:24
Implement supportsUpsert() and getUpsertStatement() in
MSSQLDatabaseDialectService to produce a MERGE statement for
UPSERT operations on SQL Server.

- Added new controller service: MSSQLDatabaseDialectService
  Provides SELECT with paging, UPSERT using MERGE, and basic ALTER/CREATE DDL generation for SQL Server 2012+.

- Added new controller service: MSSQL2008DatabaseDialectService
  Extends MSSQLDatabaseDialectService to support SQL Server 2008, including SELECT with ROW_NUMBER() paging.

- Registered both services in META-INF/services/org.apache.nifi.controller.ControllerService
  Enables NiFi to discover and use the new MSSQL dialect services.

- Added unit tests:
  - TestMSSQLDatabaseDialectService.java: Verifies SQL generation and UPSERT logic for MSSQLDatabaseDialectService.
  - TestMSSQL2008DatabaseDialectService.java: Verifies paging and SELECT logic for MSSQL2008DatabaseDialectService.

Signed-off-by: Adel Wageih <adelwageeh222@gmail.com>
@Adelwageeh96
Adelwageeh96 force-pushed the feature/NIFI-14869-mssql-upsert branch from 7765f2f to 3e078d6 Compare September 3, 2026 13:51
@Adelwageeh96

Copy link
Copy Markdown
Author

Hi @exceptionfactory , There's one failing check, code-compliance/Scan, but it's unrelated to this change — it failed on a network timeout downloading py4j (nifi-python-framework's download-py4j step), not anything in the MSSQL dialect service. I don't have permission to re-run it myself; happy to have it re-triggered whenever convenient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants