Apache Iceberg Rust version
main at 5266f7dd3241b6b94843e85f10771d2452c8b874 and release v0.10.1
Describe the bug
Map literals are represented as parallel JSON arrays:
{"keys":["a","b"],"values":[1]}Literal::try_from_json currently combines these arrays with Iterator::zip. If their lengths differ, zip silently discards the unmatched keys or values. When this JSON appears in a schema initial-default or write-default, deserialization therefore accepts malformed metadata and stores a different value from the one supplied. This can change values used during schema evolution or writes without reporting invalid metadata.
The schema conversion also suppresses errors while parsing defaults, so simply returning an error from the literal parser would otherwise turn an invalid default into None. Invalid defaults should make schema deserialization fail.
The Java reference implementation rejects map literals unless the key and value arrays have equal lengths.
How to reproduce
Parse the JSON above with a map<string, int> type, either directly through Literal::try_from_json or as a field default in a serialized schema.
Current result: parsing succeeds with only {"a": 1}; the unmatched key is lost. The inverse case, with more values than keys, also silently drops data.
Expected behavior
Map literal parsing should return a DataInvalid error when the key and value arrays have different lengths. Schema deserialization should propagate that error rather than discard the invalid default.
Willingness to contribute
I submitted #3110 with direct literal-parser regressions for both mismatch directions and end-to-end schema deserialization coverage for both default fields.
Additional context
This is a correctness issue, not a security report. I searched open and closed issues and pull requests for map defaults, JSON literal parsing, unequal arrays, and key/value length validation; none address this behavior.
Apache Iceberg Rust version
mainat5266f7dd3241b6b94843e85f10771d2452c8b874and releasev0.10.1Describe the bug
Map literals are represented as parallel JSON arrays:
{"keys":["a","b"],"values":[1]}Literal::try_from_jsoncurrently combines these arrays withIterator::zip. If their lengths differ,zipsilently discards the unmatched keys or values. When this JSON appears in a schemainitial-defaultorwrite-default, deserialization therefore accepts malformed metadata and stores a different value from the one supplied. This can change values used during schema evolution or writes without reporting invalid metadata.The schema conversion also suppresses errors while parsing defaults, so simply returning an error from the literal parser would otherwise turn an invalid default into
None. Invalid defaults should make schema deserialization fail.The Java reference implementation rejects map literals unless the key and value arrays have equal lengths.
How to reproduce
Parse the JSON above with a
map<string, int>type, either directly throughLiteral::try_from_jsonor as a field default in a serialized schema.Current result: parsing succeeds with only
{"a": 1}; the unmatched key is lost. The inverse case, with more values than keys, also silently drops data.Expected behavior
Map literal parsing should return a
DataInvaliderror when the key and value arrays have different lengths. Schema deserialization should propagate that error rather than discard the invalid default.Willingness to contribute
I submitted #3110 with direct literal-parser regressions for both mismatch directions and end-to-end schema deserialization coverage for both default fields.
Additional context
This is a correctness issue, not a security report. I searched open and closed issues and pull requests for map defaults, JSON literal parsing, unequal arrays, and key/value length validation; none address this behavior.