[C++] RecordBatch::MakeEmpty() may drop ordered flag and unsignedness of dictionary types when creating dictionary-encoded column

Describe the bug, including details regarding any error messages, version, and platform.

Found when working on dictionary encoding in nanoarrow in apache/arrow-nanoarrow#861 ...I have yet to confirm independently of that. The bit of code this affects for me checking a roundtrip of an empty record batch through IPC...I am not sure it actually makes sense for any practical purpose to have an empty (but ordered) dictionary.

  EXPECT_EQ(maybe_batch.ValueUnsafe()->ToString(), empty->ToString());
  if (data_type->id() != arrow::Type::DICTIONARY ||
      !std::static_pointer_cast<arrow::DictionaryType>(data_type)->ordered()) {
    EXPECT_TRUE(maybe_batch.ValueUnsafe()->Equals(*empty)) << empty->ToString();
  }

Component(s)

C++

@paleolimbot

Hi @paleolimbot, just to confirm the expected behavior here: should RecordBatch::MakeEmpty() preserve the ordered flag for DictionaryType, or is it acceptable for that information to be lost for empty batches?

I think it should be preserved...having the schema of a batch not equal the schema used to create it caused errors in my tests and I think is the intention of MakeEmpty.

@paleolimbot

Thanks @paleolimbot for confirming, I’ll try to reproduce this and look into where the ordered flag is being dropped during MakeEmpty().

I tried to reproduce this using RecordBatch::MakeEmpty() and IPC roundtrips with ordered dictionary types, but I wasn’t able to observe the ordered flag being dropped.

Is this specific to a case where the dictionary array has actual dictionary values (e.g., non-empty dictionary with empty indices), or does it require a particular nanoarrow workflow?

Thank you for looking!

It looks like the place I ran across that was here:

https://github.com/apache/arrow-nanoarrow/pull/861/changes#diff-809791cd3b01ea5de2038a046f68a5e5649126a74d9e2ac9ac18217c65f395dfR1326

...where there's quite a few nanoarrow things going on too. I will double check that from my end to make sure this is actually an Arrow C++ issue 😬

Okay, will wait for your confirmation on whether this turns out to be on the Arrow C++ side.

CoPilot via Opus 4.5 generated this (rather dramatic, if I may say so) confirmation of the bug. Basically, the schema is correct but NOT the data type of column.

// Diagnostic test to demonstrate Arrow C++ MakeEmpty() bug
// The schema has ordered=1, but the array type created by MakeEmpty() has ordered=0
TEST(NanoarrowIpcTest, ArrowMakeEmptyLosesOrderedFlag) {
  // Create an ordered dictionary type
  auto data_type = arrow::dictionary(arrow::int32(), arrow::utf8(), /*ordered=*/true);
  std::shared_ptr<arrow::Schema> schema =
      arrow::schema({arrow::field("col", data_type)});

  // MakeEmpty should create a batch with ordered=true arrays
  auto maybe_empty = arrow::RecordBatch::MakeEmpty(schema);
  ASSERT_TRUE(maybe_empty.ok()) << maybe_empty.status();
  auto empty = maybe_empty.ValueUnsafe();

  // Schema correctly has ordered=1
  auto schema_dict_type =
      std::static_pointer_cast<arrow::DictionaryType>(empty->schema()->field(0)->type());
  EXPECT_TRUE(schema_dict_type->ordered()) << "Schema should have ordered=true";

  // BUG: Array type has ordered=0
  auto array_dict_type =
      std::static_pointer_cast<arrow::DictionaryType>(empty->column(0)->type());
  // This is the bug in Arrow C++ MakeEmpty() - the array type loses ordered flag
  // https://github.com/apache/arrow/issues/49674
  EXPECT_FALSE(array_dict_type->ordered())
      << "BUG CONFIRMED: MakeEmpty() array has ordered=0 (should be 1)";
}

That makes sense, I was only checking the schema before, but I can reproduce the issue when inspecting the array type. I’ll take a closer look at how the array is constructed in MakeEmpty().

@paleolimbot

Hi @paleolimbot, I was able to reproduce this. The issue occurs because DictionaryBuilder reconstructs the DictionaryType without preserving the ordered flag (it defaults to false). Since MakeEmptyArray relies on builder->Finish(), the resulting array loses this information.

I implemented a fix in MakeEmptyArray to restore the original type after building the array, and added a test that verifies the ordered flag is preserved.

Before opening a PR, I wanted to confirm whether this scoped fix is preferred, or if you'd rather address it at the DictionaryBuilder level.

I'm not an expert on these internals, but I can see how the DictionaryBuilder might not be able to guarantee the ordering of the keys that come out of it while preserving the dictionary indices emitted for previous rows. It seems like your approach is reasonable...make a PR and we can discuss there!

@paleolimbot

Hi @paleolimbot, I have opened a PR for this issue #49798

apacheGH-41017: Add RecordBatch::MakeEmpty test for dictionary ordere…

in apache/arrow-nanoarrow#882 I also found that it may be dropping the unsigned-ness from unsigned dictionary types!

Navigation Menu

Repository navigation

[C++] `RecordBatch::MakeEmpty()` may drop ordered flag and unsignedness of dictionary types when creating dictionary-encoded column #49674

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[C++] RecordBatch::MakeEmpty() may drop ordered flag and unsignedness of dictionary types when creating dictionary-encoded column #49674

Description

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Activity

Reranko05 commented on Apr 17, 2026

paleolimbot commented on Apr 17, 2026

Reranko05 commented on Apr 17, 2026

Reranko05 commented on Apr 17, 2026

paleolimbot commented on Apr 17, 2026

Reranko05 commented on Apr 17, 2026

paleolimbot commented on Apr 17, 2026

Reranko05 commented on Apr 18, 2026

Reranko05 commented on Apr 18, 2026

paleolimbot commented on Apr 18, 2026

Reranko05 commented on Apr 19, 2026

paleolimbot commented on May 12, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[C++] `RecordBatch::MakeEmpty()` may drop ordered flag and unsignedness of dictionary types when creating dictionary-encoded column #49674