Describe the bug
HashJoinExec and NestedLoopJoinExec report EmissionType::Incremental for JoinType::LeftSemi, but neither operator emits any LeftSemi output during the probe phase. All output rows are produced from the build-side visited bitmap after the probe side is exhausted, which is EmissionType::Final behaviour.
The comment next to the classification says the incremental group is the joins that "only need to generate matched rows from the probe side". That is not true for LeftSemi in these operators, which output build-side rows.
Where
HashJoinExec::compute_properties and NestedLoopJoinExec::compute_properties both place LeftSemi in the EmissionType::Incremental arm, alongside Inner, Right, RightSemi, RightAnti and RightMark.adjust_indices_by_join_type in joins/utils.rs returns empty index arrays for LeftSemi | LeftAnti | LeftMark during the probe phase, so nothing is emitted while probing.need_produce_result_in_final includes LeftSemi, and get_final_indices_from_shared_bitmap produces the matched build rows for it once the probe side is done.
So the operator itself already treats LeftSemi as "produce in final", and only the reported plan property disagrees.
Why it matters
- Wrong plan property. Anything that inspects
pipeline_behavior() gets a misleading answer for LeftSemi hash and nested loop joins.
- Unbounded probe side is not rejected. With a bounded build side and an unbounded, incrementally emitting probe side, the join is labelled
Incremental and passes pipeline checks, yet it can never emit a row because it waits for probe exhaustion. LeftAnti and LeftMark in the same situation are labelled Both; LeftSemi should get the same treatment at minimum.
- Inconsistent predicates.
maintains_input_order reasons about which joins emit from the bitmap at the end and includes LeftSemi, while the emission classification excludes it. Two different predicates for the same underlying fact invite future drift.
To Reproduce
// bounded left, unbounded incremental right, join_type = LeftSemi
let join = HashJoinExec::try_new(left, right, on, None, &JoinType::LeftSemi, None,
PartitionMode::CollectLeft, NullEquality::NullEqualsNothing)?;
assert_eq!(join.pipeline_behavior(), EmissionType::Incremental); // passes today
// but executing it produces no output until `right` ends, which never happensExpected behavior
LeftSemi should be reported as EmissionType::Final for HashJoinExec and NestedLoopJoinExec, since the operators produce nothing before the probe side completes. If Final turns out to have unwanted planning side effects, Both is still more accurate than Incremental and matches how LeftAnti and LeftMark are handled.
Ideally the classification and maintains_input_order derive from the same predicate (need_produce_result_in_final or a renamed equivalent) so they cannot disagree.
Additional context
SortMergeJoinExec is unaffected. It reports Incremental unconditionally and does emit LeftSemi rows as it streams.
Surfaced during review of #24957, where the comment was reworded to "everything else is emitted incrementally", which made the mismatch more visible. The classification itself predates that PR and is unchanged by it. #24957 keeps the existing Incremental label for LeftSemi to avoid a behaviour change inside a refactor; this issue tracks fixing the label separately.
Describe the bug
HashJoinExecandNestedLoopJoinExecreportEmissionType::IncrementalforJoinType::LeftSemi, but neither operator emits anyLeftSemioutput during the probe phase. All output rows are produced from the build-side visited bitmap after the probe side is exhausted, which isEmissionType::Finalbehaviour.The comment next to the classification says the incremental group is the joins that "only need to generate matched rows from the probe side". That is not true for
LeftSemiin these operators, which output build-side rows.Where
HashJoinExec::compute_propertiesandNestedLoopJoinExec::compute_propertiesboth placeLeftSemiin theEmissionType::Incrementalarm, alongsideInner,Right,RightSemi,RightAntiandRightMark.adjust_indices_by_join_typeinjoins/utils.rsreturns empty index arrays forLeftSemi | LeftAnti | LeftMarkduring the probe phase, so nothing is emitted while probing.need_produce_result_in_finalincludesLeftSemi, andget_final_indices_from_shared_bitmapproduces the matched build rows for it once the probe side is done.So the operator itself already treats
LeftSemias "produce in final", and only the reported plan property disagrees.Why it matters
pipeline_behavior()gets a misleading answer forLeftSemihash and nested loop joins.Incrementaland passes pipeline checks, yet it can never emit a row because it waits for probe exhaustion.LeftAntiandLeftMarkin the same situation are labelledBoth;LeftSemishould get the same treatment at minimum.maintains_input_orderreasons about which joins emit from the bitmap at the end and includesLeftSemi, while the emission classification excludes it. Two different predicates for the same underlying fact invite future drift.To Reproduce
Expected behavior
LeftSemishould be reported asEmissionType::FinalforHashJoinExecandNestedLoopJoinExec, since the operators produce nothing before the probe side completes. IfFinalturns out to have unwanted planning side effects,Bothis still more accurate thanIncrementaland matches howLeftAntiandLeftMarkare handled.Ideally the classification and
maintains_input_orderderive from the same predicate (need_produce_result_in_finalor a renamed equivalent) so they cannot disagree.Additional context
SortMergeJoinExecis unaffected. It reportsIncrementalunconditionally and does emitLeftSemirows as it streams.Surfaced during review of #24957, where the comment was reworded to "everything else is emitted incrementally", which made the mismatch more visible. The classification itself predates that PR and is unchanged by it. #24957 keeps the existing
Incrementallabel forLeftSemito avoid a behaviour change inside a refactor; this issue tracks fixing the label separately.