SubstraitRelVisitor.visit(org.apache.calcite.rel.core.Aggregate) builds the Substrait grouping columns from aggregate.getGroupSets() and never reads getGroupSet(). Calcite emits one output column per bit of getGroupSet() — Aggregate.deriveRowType iterates groupSet.asList() — and its constructor only asserts that each grouping set is contained in the group set, never that the union of the sets equals it. A key that appears in the group set but in no grouping set therefore produces a Calcite column the conversion does not account for.
Measured on main at 7310fc8, over a scan foo(a i64, b i64, c string):
LogicalAggregate.create(
scan, List.of(), ImmutableBitSet.of(0, 1, 2),
List.of(ImmutableBitSet.of(0, 1), ImmutableBitSet.of(1)), List.of());
That node's row type is RecordType(BIGINT a, BIGINT b, VARCHAR c) — three columns — while the union of its grouping sets is {0, 1}. Converting it fails at the plan root:
java.lang.IllegalArgumentException: Plan.Root names count (3) must match input record type
depth-first named-field count (2)
At the root the mismatch at least surfaces. Deeper in a plan there is no name count to check, so the column is dropped silently and every measure index in the emit mapping shifts by one.
The spec side is unambiguous: "Each of the grouping expressions must occur in at least one of the grouping sets" (logical relations, spec v0.101.0), so there is no Substrait aggregate this Calcite shape maps onto. The fix is to reject it explicitly — naming the group-set keys that appear in no grouping set — rather than converting to a relation that is narrower than the node it came from.
Reachable from a RelBuilder-constructed plan, an optimizer rule, or a non-Isthmus Calcite front end; Calcite's own containment assertion is disabled outside tests, so nothing upstream stops it either. Found while reviewing #1161.
SubstraitRelVisitor.visit(org.apache.calcite.rel.core.Aggregate)builds the Substrait grouping columns fromaggregate.getGroupSets()and never readsgetGroupSet(). Calcite emits one output column per bit ofgetGroupSet()—Aggregate.deriveRowTypeiteratesgroupSet.asList()— and its constructor only asserts that each grouping set is contained in the group set, never that the union of the sets equals it. A key that appears in the group set but in no grouping set therefore produces a Calcite column the conversion does not account for.Measured on
mainat 7310fc8, over a scanfoo(a i64, b i64, c string):That node's row type is
RecordType(BIGINT a, BIGINT b, VARCHAR c)— three columns — while the union of its grouping sets is{0, 1}. Converting it fails at the plan root:At the root the mismatch at least surfaces. Deeper in a plan there is no name count to check, so the column is dropped silently and every measure index in the emit mapping shifts by one.
The spec side is unambiguous: "Each of the grouping expressions must occur in at least one of the grouping sets" (logical relations, spec v0.101.0), so there is no Substrait aggregate this Calcite shape maps onto. The fix is to reject it explicitly — naming the group-set keys that appear in no grouping set — rather than converting to a relation that is narrower than the node it came from.
Reachable from a
RelBuilder-constructed plan, an optimizer rule, or a non-Isthmus Calcite front end; Calcite's own containment assertion is disabled outside tests, so nothing upstream stops it either. Found while reviewing #1161.