You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Native upsert can lose the row count when selecting non-null FixedSizeBinary(0) values, including values nested in ROW/STRUCT columns.
The native Python CI for apache/paimon#10970 built paimon-rust main at 5bb42a110eeb22229851e9e36205376bedb729f8 and failed four cases in native_update_encoded_inputs_test.py::test_scalar_input_encodings_keep_values: empty_fixed_binary with root/row inputs and batch/stream native upsert. The Python execution path passes.
Failure:
cannot build selected upsert rows: Invalid argument error: all columns in a record batch must have the same length
TableUpsert::selected_rows uses Arrow's take kernel. For zero-width fixed binary, that kernel constructs the result without an explicit length. When the selected values are all valid, the value buffer is empty and there is no null buffer to carry the row count, so a nonempty selection becomes a zero-length array. Enclosing structs can inherit the incorrect length or contain children with inconsistent lengths.
Reproduced locally with the same failing Python test and a Rust regression covering row selections, nulls and nested structs. Affected columns can preserve their row counts by concatenating selected row slices.
Activity