You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Thanks for creating this @zhjwpku , do you mind adding sub tasks around non shredding first, then shredding as well. You can look here under the go tracker Track A to have something similar for cpp
Thanks for creating this @zhjwpku , do you mind adding sub tasks around non shredding first, then shredding as well. You can look here under the go tracker Track A to have something similar for cpp
Just dug into some details about Variant Shredding [1].
@nssalian Thanks for your interest! Do you have a concrete plan for the variant type support? I'm not quite sure if we need variant encoding in the iceberg library. As we depend on arrow-cpp for parquet, I'm not sure if we want to reuse variant encoding from arrow-cpp. (cc @HuaHuaY who is adding variant encoding to arrow-cpp)
@wgtmac thanks for weighing in. Yes, the plan is to wire arrow-cpp's variant into iceberg-cpp's type system, not duplicate the codec. iceberg-go follows the same pattern (defers to arrow-go's variant).
Activity
nssalian commentedon May 21, 2026
Thanks for creating this @zhjwpku , do you mind adding sub tasks around non shredding first, then shredding as well. You can look here under the go tracker Track A to have something similar for cpp
zhjwpku commentedon May 21, 2026
Just dug into some details about Variant Shredding [1].
[1] https://parquet.apache.org/docs/file-format/types/variantshredding/
nssalian commentedon May 21, 2026
@zhjwpku I would be best to add support for Variant as a type first which is usually writing unshredded variant data. Shredded writes can follow.
wgtmac commentedon May 21, 2026
@nssalian Thanks for your interest! Do you have a concrete plan for the variant type support? I'm not quite sure if we need variant encoding in the iceberg library. As we depend on arrow-cpp for parquet, I'm not sure if we want to reuse variant encoding from arrow-cpp. (cc @HuaHuaY who is adding variant encoding to arrow-cpp)
nssalian commentedon May 21, 2026
@wgtmac thanks for weighing in. Yes, the plan is to wire arrow-cpp's variant into iceberg-cpp's type system, not duplicate the codec. iceberg-go follows the same pattern (defers to arrow-go's variant).