Skip to content

[SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNION ALL - #58506

Open
tokoko wants to merge 1 commit into
apache:masterfrom
tokoko:full-join-union-all
Open

[SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNION ALL#58506
tokoko wants to merge 1 commit into
apache:masterfrom
tokoko:full-join-union-all

Conversation

@tokoko

@tokoko tokoko commented Sep 3, 2026

Copy link
Copy Markdown

What changes were proposed in this pull request?

PropagateEmptyRelation already rewrites a join whose condition is FalseLiteral for most join types: inner/cross and left semi collapse to an empty relation for example. This PR adds FullOuter handling. With a false condition no row on either side can find a match, so the join is equivalent to a UNION ALL of both sides, each padded with nulls for the other side's columns.

Why are the changes needed?

1 = 0 is not an equi-join condition, so today this plan runs as a BroadcastNestedLoopJoin which is inefficient. After the rewrite both sides are scanned once and fuse into a single whole-stage codegen stage, with no exchange.

Users can usually write the union themselves, but not always. The motivating case is a MERGE with an intentionally non-matching condition, where the join is generated by the MERGE rewrite rather than written by hand:

MERGE INTO target_table AS t
USING source AS s ON 1 = 0
WHEN NOT MATCHED THEN INSERT *
WHEN NOT MATCHED BY SOURCE AND [replaceWhereCondition] THEN DELETE

This is the shape for a replaceWhere-style operation made possible by #52185.

Does this PR introduce any user-facing change?

No. Query results are unchanged; only the optimized plan changes.

How was this patch tested?

New and updated unit tests.

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code 2.1.259 (Claude Opus 5)

@HyukjinKwon

Copy link
Copy Markdown
Member

seems like the PR title was truncated. Can you make it complete please?

@tokoko tokoko changed the title [SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNI… [SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNION Sep 4, 2026
@tokoko tokoko changed the title [SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNION [SPARK-53618][SQL] Turn a FULL JOIN with a false condition into a UNION ALL Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants