feat(inspect): add files, partitions, and manifests metadata tables by manuzhang · Pull Request #876 · apache/iceberg-cpp

manuzhang · 2026-08-06T09:54:10Z

Depends on #1007 for the shared metadata Arrow row-stream helper. RefsTable, its tests, and the helper are excluded from this PR. Merge #1007 first, then rebase this draft onto main.

Add FilesTable, PartitionsTable, and ManifestsTable to inspect snapshot metadata through Arrow streams. Snapshot selection supports IDs, timestamps, and named references; non-main references cannot be combined with an ID or timestamp. File scans read manifests on demand and emit batches of at most 1,024 rows. Partition aggregation projects across evolved specs and canonicalizes NaNs while preserving signed zeros. Manifest rows include file counts and readable partition bounds.

Tests are organized per metadata table under test/inspect, with shared setup for snapshot-scoped scans. Existing history, snapshots, and base metadata tests and their fixture use the same directory layout as #1007.

Validation: pre-commit and git diff --check passed. The source composition with the updated #1007 passed all 87 metadata tests, including the existing files/partitions/manifests tests and the new stream regressions. #1007 independently passed all 31 metadata tests on main. This draft requires its prerequisite to compile; hosted CI is separate from local combined validation.

Copilot

Pull request overview

Adds new “system” metadata tables under iceberg/inspect/ and shared utilities to scan Iceberg metadata into bounded Arrow batches, aligning C++ inspect APIs with Iceberg’s reference behavior. This builds on the streaming metadata-table API work referenced in #801.

Changes:

Adds new metadata table implementations: branches, tags, files, partitions, manifests (plus supporting shared stream/util code).
Refactors metadata-table base APIs to support typed factories, bounded batch sizing, and time-travel scans via SnapshotSelection.
Adds/updates unit tests and wires them into CMake/Meson builds.

Reviewed changes

Copilot reviewed 33 out of 33 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/iceberg/type_fwd.h	Forward declares the new inspect table classes.
src/iceberg/inspect/metadata_table.h	Expands/modernizes the metadata-table base API (kinds, typed factory, batch size, time travel interface).
src/iceberg/inspect/metadata_table.cc	Implements the refactored base behavior and `TimeTravelMetadataTable` scan dispatch.
src/iceberg/inspect/metadata_table_stream_internal.h	Adds a shared Arrow stream implementation for row-vector-backed metadata tables.
src/iceberg/inspect/metadata_table_util_internal.h	Declares shared internal helpers for snapshot resolution, schema building, partition projection, and live-file loading.
src/iceberg/inspect/metadata_table_util_internal.cc	Implements shared internal utilities used by files/partitions/manifests scans.
src/iceberg/inspect/branches_table.h	Declares the branches metadata table.
src/iceberg/inspect/branches_table.cc	Implements branch reference scanning into Arrow batches.
src/iceberg/inspect/tags_table.h	Declares the tags metadata table.
src/iceberg/inspect/tags_table.cc	Implements tag reference scanning into Arrow batches.
src/iceberg/inspect/files_table.h	Declares the snapshot-scoped files metadata table (time travel capable).
src/iceberg/inspect/files_table.cc	Implements live-file scanning for a selected snapshot into Arrow batches.
src/iceberg/inspect/partitions_table.h	Declares the snapshot-scoped partitions aggregates table (time travel capable).
src/iceberg/inspect/partitions_table.cc	Implements partition aggregation over live files for a selected snapshot.
src/iceberg/inspect/manifests_table.h	Declares the snapshot-scoped manifests table (time travel capable).
src/iceberg/inspect/manifests_table.cc	Implements manifest-list scanning for a selected snapshot, including partition summaries.
src/iceberg/inspect/snapshots_table.h	Extends snapshots table API with schema + streaming Scan().
src/iceberg/inspect/snapshots_table.cc	Implements snapshots streaming scan with bounded batch size and resource cleanup.
src/iceberg/inspect/history_table.h	Updates history table interface to match new base API expectations.
src/iceberg/inspect/history_table.cc	Updates history table implementation; currently returns NotSupported for Scan().
src/iceberg/inspect/meson.build	Installs the new public inspect headers.
src/iceberg/arrow_row_builder_internal.h	Adds `num_rows()` to support batch-size enforcement in streaming producers.
src/iceberg/arrow_row_builder.cc	Implements `ArrowRowBuilder::num_rows()`.
src/iceberg/test/arrow_row_builder_test.cc	Adds assertions covering the new `num_rows()` behavior.
src/iceberg/test/metadata_table_test_base.h	Introduces a shared fixture/helpers for metadata-table tests.
src/iceberg/test/metadata_table_test.cc	Updates tests to the typed metadata-table factory and validates time-travel support flags.
src/iceberg/test/history_table_test.cc	Adds schema-focused unit test coverage for HistoryTable.
src/iceberg/test/snapshots_table_test.cc	Adds snapshots streaming scan tests (schema/values/batching/null handling).
src/iceberg/test/system_metadata_tables_test.cc	Adds integration-style tests for the new system metadata tables (refs/files/partitions/manifests and snapshot selection).
src/iceberg/test/CMakeLists.txt	Wires the new/updated metadata-table tests into the CMake test target.
src/iceberg/test/meson.build	Wires (some) of the new tests into Meson’s `table_test` target.
src/iceberg/CMakeLists.txt	Adds new inspect sources to the main CMake build.
src/iceberg/meson.build	Adds new inspect sources to the main Meson build.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+Result<ArrowArrayStream> HistoryTable::Scan() {
+  return NotSupported("Scan is not supported for the history table");
+}


Copilot

🟡 Changes recommended

Unresolved critical and moderate findings affect decimal handling, schema correctness, manifest counts, and streaming memory bounds.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (4)

src/iceberg/inspect/files_table.cc:54

These schema/type objects are fixed when FilesTable::Make runs, but ScanSnapshot resolves snapshots and loads files from the source table's current metadata. After source_table()->Refresh() with schema or partition evolution, the scan mixes new files/specs with the stale output schema, silently omitting new metrics or partition fields (or projecting them with the wrong type). Rebuild the scan schema from the same metadata snapshot used for the scan, or otherwise make the cached schema and source metadata consistent.

  ICEBERG_ASSIGN_OR_RAISE(auto partition_type, internal::UnifiedPartitionType(*table));
  ICEBERG_ASSIGN_OR_RAISE(auto schema,
                          internal::FilesTableSchema(*table_schema, partition_type));
  return std::unique_ptr<FilesTable>(new FilesTable(std::move(table), std::move(schema),
                                                    std::move(table_schema),
                                                    std::move(partition_type)));

src/iceberg/inspect/manifests_table.cc:123

When a manifest count is absent, .value_or(0) converts Iceberg's unknown count into a definitive zero. ManifestFile documents a null count as “assumed to be non-zero”, so $manifests reports incorrect counts for older manifests; preserve null for the applicable content counts and use zero only for the opposite content type.

      AppendInt(builder.column(5), data ? manifest.added_files_count.value_or(0) : 0));
  ICEBERG_RETURN_UNEXPECTED(
      AppendInt(builder.column(6), data ? manifest.existing_files_count.value_or(0) : 0));

src/iceberg/inspect/metadata_table_util_internal.cc:409

Because specs are processed newest-to-oldest, existing is the newer void definition and partition_field is the older non-void definition. Replacing definition here makes the unified schema use the old field name (for example, old_bucket) instead of the latest name (new_name), so the added legacy-evolution test will fail and consumers see a schema that no longer matches the current spec. Keep the latest definition/name and only adopt the older transform's type for decoding legacy values.

        if (existing_transform == TransformType::kVoid &&
            current_transform != TransformType::kVoid) {
          iter->second.definition = &partition_field;
          iter->second.type = spec_field.type();
        }

src/iceberg/inspect/metadata_table_util_internal.cc:617

This vector accumulates every live file returned by every manifest, while LiveEntries() has already materialized each manifest, and the stream is created only after the full vector is built. For a large snapshot, the Arrow batch limit does not bound scan memory or delay the first batch until the entire snapshot has been read. Feed entries through a cursor/iterator directly into bounded batches (or otherwise avoid retaining the complete result).

  std::vector<LiveFile> files;

Files reviewed: 21/21 changed files
Comments generated: 2
Review effort level: Lite

Copilot

🟡 Changes recommended

Unresolved critical refresh-staleness findings and additional schema, validation, and projection issues remain.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (3)

src/iceberg/inspect/metadata_table_util_internal.cc:383

On an unpartitioned table this erases partition from the public files schema even though DataFile::Type still defines it as an empty struct. Consumers therefore get a different column set for partitioned versus unpartitioned tables; the corresponding conditional in PartitionsTable::Make also drops partition and spec_id. Keep these empty-struct/metadata columns and append empty partition values so the metadata-table schema remains stable.

  if (partition_type->fields().empty()) {
    std::erase_if(fields, [](const SchemaField& field) {
      return field.field_id() == DataFile::kPartitionFieldId;
    });
  }

src/iceberg/inspect/metadata_table_util_internal.cc:75

An unknown named reference is caller-provided selection input, but ICEBERG_CHECK maps this failure to ErrorKind::kValidationFailed. Scan(SnapshotSelection{.ref_name = ...}) therefore reports a state error instead of the invalid-argument error used for invalid selection values. Use ICEBERG_PRECHECK for this lookup, consistent with the null-reference check below.

  ICEBERG_CHECK(ref != metadata->refs.end(), "Cannot find snapshot reference '{}'",
                ref_name);

src/iceberg/inspect/partitions_table.cc:259

PartitionsTable only consumes the partition, content, record count, and file size from each data file, but this calls LiveFiles without a projection. ManifestReader::ReadEntries consequently materializes the complete data-file schema, including all stats maps, bounds, and key metadata, for every live entry before aggregation; wide metrics can make $partitions scans unnecessarily memory-heavy. Add a projection path for LiveFiles and select only the fields needed by partition projection and UpdateCounts, as Java does for this table.

  ICEBERG_ASSIGN_OR_RAISE(auto files, internal::LiveFiles(*source_table(), snapshot));

Files reviewed: 25/25 changed files
Comments generated: 2
Review effort level: Lite

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot

🔵 Needs a closer look

Preserve the latest unified field name when adopting an older transform’s result type.

2 open findings

Although the returned Arrow batches are capped, this call eagerly reads every live manifest entry…
HistoryTable is exposed as a concrete MetadataTable with a defined schema, but Scan()…

2 resolved since last review

The partition schema and unified partition type are cached only during construction, even though…
These derived schemas are computed only once, while source_table() is the same mutable Table…

🧠 Review effort: Lite

Copilot

🟡 Changes recommended

Critical issues remain in partition-evolution schema selection and partition aggregation grouping.

4 open findings

Preserve newest partition field definition during legacy void evolution · New
Include partition spec identity in aggregate grouping key · New
Although the returned Arrow batches are capped, this call eagerly reads every live manifest entry…
HistoryTable is exposed as a concrete MetadataTable with a defined schema, but Scan()…

🧠 Review effort: Lite

+        if (existing_transform == TransformType::kVoid &&
+            current_transform != TransformType::kVoid) {
+          iter->second.definition = &partition_field;
+          iter->second.type = spec_field.type();
+        }


+struct PartitionKey {
+  PartitionValues values;
+  size_t projected_fields;
+
+  bool operator==(const PartitionKey& other) const {
+    if (projected_fields != other.projected_fields ||
+        values.num_fields() != other.values.num_fields()) {


Copilot

🔵 Needs a closer look

Missing manifest partition summaries are emitted as empty lists instead of null, and the duplicate PR concern remains.

3 open findings

1 resolved since last review

Although the returned Arrow batches are capped, this call eagerly reads every live manifest entry…

🧠 Review effort: Lite

Co-authored-by: Codex <codex@openai.com>

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

8 open findings

Include partition spec identity in aggregate grouping key
Preserve newest partition field definition during legacy void evolution
The stream creation currently passes *schema_ (the mutable member) into… · New
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live… · New
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live… · New
AppendOptional unconditionally funnels optionals through AppendInt(... int64_t ...), but… · New
HistoryTable is exposed as a concrete MetadataTable with a defined schema, but Scan()…
AppendReadableMetrics() performs a FindFieldByName() lookup for every (file, metric-column)… · New

🧠 Review effort: Lite

+  auto schema = schema_;
+  auto table_schema = table_schema_;
+  auto partition_type = partition_type_;
+  return internal::MakeMetadataTableStream(
+      *schema_, std::move(files),
+      [schema = std::move(schema), table_schema = std::move(table_schema),
+       partition_type = std::move(partition_type)](ArrowRowBuilder& builder,
+                                                   const internal::LiveFile& file) {
+        return internal::AppendDataFile(builder, *schema, *table_schema, *partition_type,
+                                        file);
+      });


+size_t ProjectedFieldCount(const StructType& partition_type, const PartitionSpec& spec) {
+  size_t count = 0;
+  for (const auto& field : partition_type.fields()) {
+    count += std::ranges::any_of(
+        spec.fields(), [field_id = field.field_id()](const PartitionField& spec_field) {
+          return spec_field.field_id() == field_id;
+        });
+  }
+  return count;
+}


+    ICEBERG_ASSIGN_OR_RAISE(auto partition_values, internal::ProjectPartitionValues(
+                                                       *partition_type_, *live_file.spec,
+                                                       live_file.file->partition));
+    PartitionKey key{
+        .values = std::move(partition_values),
+        .projected_fields = ProjectedFieldCount(*partition_type_, *live_file.spec)};


+Status AppendOptional(ArrowArray* array, const auto& value) {
+  if (!value.has_value()) {
+    return AppendNull(array);
+  }
+  return AppendInt(array, static_cast<int64_t>(*value));
+}


+  for (int64_t index = 0; index < array->n_children; ++index) {
+    auto* column_metrics = array->children[index];
+    ICEBERG_PRECHECK(column_metrics != nullptr && column_metrics->n_children == 6,
+                     "Readable column metrics must contain six fields");
+    const auto readable_name = readable_type.fields()[index].name();
+    ICEBERG_ASSIGN_OR_RAISE(auto source_field,
+                            table_schema.FindFieldByName(readable_name));
+    ICEBERG_PRECHECK(source_field.has_value(),
+                     "Cannot find readable metrics source field '{}'", readable_name);


Co-authored-by: Codex <codex@openai.com>

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

12 open findings

Include partition spec identity in aggregate grouping key
Preserve newest partition field definition during legacy void evolution
This loop copies each ManifestFile into rows (.manifest = manifest). If ManifestFile… · New
ProjectPartitionValues rebuilds a field_id -> index hash map on every call. This runs once per… · New
ProjectedFieldCount is O(|partition_type| * |spec.fields|) and is invoked for every file during… · New
The last_updated_* selection uses only timestamp_ms as a strict > comparison. If multiple… · New
AppendOptional unconditionally funnels optionals through AppendInt(... int64_t ...), but…
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live…
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live…
The stream creation currently passes *schema_ (the mutable member) into…
HistoryTable is exposed as a concrete MetadataTable with a defined schema, but Scan()…
AppendReadableMetrics() performs a FindFieldByName() lookup for every (file, metric-column)…

🧠 Review effort: Lite

+    for (const auto& manifest : manifests) {
+      auto spec = specs_ref.get().find(manifest.partition_spec_id);
+      ICEBERG_CHECK(spec != specs_ref.get().end(),
+                    "Cannot find partition spec {} for manifest '{}'",
+                    manifest.partition_spec_id, manifest.manifest_path);
+      ICEBERG_PRECHECK(spec->second != nullptr, "Partition spec {} is null",
+                       manifest.partition_spec_id);
+      rows.push_back(ManifestRow{.manifest = manifest, .spec = spec->second});


+  std::unordered_map<int32_t, size_t> positions;
+  positions.reserve(spec.fields().size());
+  for (size_t index = 0; index < spec.fields().size(); ++index) {
+    positions.emplace(spec.fields()[index].field_id(), index);
+  }
+
+  std::vector<Literal> projected;
+  projected.reserve(partition_type.fields().size());
+  for (const auto& field : partition_type.fields()) {
+    auto target_type = checked_pointer_cast<PrimitiveType>(field.type());
+    auto position = positions.find(field.field_id());
+    if (position == positions.end()) {
+      projected.push_back(Literal::Null(std::move(target_type)));
+      continue;
+    }
+    ICEBERG_ASSIGN_OR_RAISE(auto value, values.ValueAt(position->second));


+size_t ProjectedFieldCount(const StructType& partition_type, const PartitionSpec& spec) {
+  size_t count = 0;
+  for (const auto& field : partition_type.fields()) {
+    count += std::ranges::any_of(
+        spec.fields(), [field_id = field.field_id()](const PartitionField& spec_field) {
+          return spec_field.field_id() == field_id;
+        });
+  }
+  return count;
+}


+      if (file_snapshot != nullptr &&
+          (!partition.last_updated_at.has_value() ||
+           file_snapshot->timestamp_ms > *partition.last_updated_at)) {
+        partition.spec_id = live_file.spec->spec_id();
+        partition.last_updated_at = file_snapshot->timestamp_ms;
+        partition.last_updated_snapshot_id = file_snapshot->snapshot_id;


Move the common metadata test fixture into inspect and update history and snapshots includes to match PR apache#1007. Co-authored-by: Codex <codex@openai.com>

Copilot

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

18 open findings

The test target now references inspect/history_table_test.cc, inspect/snapshots_table_test.cc,… · New
Include partition spec identity in aggregate grouping key
Preserve newest partition field definition during legacy void evolution
PartitionKey uses projected_fields as a count to distinguish specs, but the count alone is… · New
PartitionKey uses projected_fields as a count to distinguish specs, but the count alone is… · New
ProjectedFieldCount is O(|unified_fields| * |spec_fields|) and is called for every live file… · New
This aggregation keeps a full PartitionValues copy in the unordered_map key and another in… · New
This aggregation keeps a full PartitionValues copy in the unordered_map key and another in… · New
The last_updated_* selection uses only timestamp_ms as a strict > comparison. If multiple…
ProjectedFieldCount is O(|partition_type| * |spec.fields|) and is invoked for every file during…
ProjectPartitionValues rebuilds a field_id -> index hash map on every call. This runs once per…
This loop copies each ManifestFile into rows (.manifest = manifest). If ManifestFile…
AppendOptional unconditionally funnels optionals through AppendInt(... int64_t ...), but…
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live…
ProjectedFieldCount() is an O(|union_fields| * |spec_fields|) scan and is executed per live…
The stream creation currently passes *schema_ (the mutable member) into…
HistoryTable is exposed as a concrete MetadataTable with a defined schema, but Scan()…
AppendReadableMetrics() performs a FindFieldByName() lookup for every (file, metric-column)…

🧠 Review effort: Lite

+                   inspect/history_table_test.cc
+                   inspect/manifests_table_test.cc
+                   inspect/metadata_table_test.cc
+                   inspect/partitions_table_test.cc
+                   inspect/snapshots_table_test.cc)


+struct PartitionKey {
+  PartitionValues values;
+  size_t projected_fields;
+
+  bool operator==(const PartitionKey& other) const {
+    if (projected_fields != other.projected_fields ||
+        values.num_fields() != other.values.num_fields()) {
+      return false;
+    }
+    for (size_t index = 0; index < values.num_fields(); ++index) {
+      const auto& lhs = values.values()[index];
+      const auto& rhs = other.values.values()[index];
+      if (lhs.IsNull() || rhs.IsNull()) {
+        if (lhs.IsNull() != rhs.IsNull()) {
+          return false;
+        }
+      } else if (lhs.IsNaN() && rhs.IsNaN()) {
+        // Java partition equality canonicalizes all NaN signs and payloads.
+        continue;
+      } else if (lhs != rhs) {
+        return false;
+      }
+    }
+    return true;
+  }
+};


+size_t ProjectedFieldCount(const StructType& partition_type, const PartitionSpec& spec) {
+  size_t count = 0;
+  for (const auto& field : partition_type.fields()) {
+    count += std::ranges::any_of(
+        spec.fields(), [field_id = field.field_id()](const PartitionField& spec_field) {
+          return spec_field.field_id() == field_id;
+        });
+  }
+  return count;
+}


+size_t ProjectedFieldCount(const StructType& partition_type, const PartitionSpec& spec) {
+  size_t count = 0;
+  for (const auto& field : partition_type.fields()) {
+    count += std::ranges::any_of(
+        spec.fields(), [field_id = field.field_id()](const PartitionField& spec_field) {
+          return spec_field.field_id() == field_id;
+        });
+  }
+  return count;
+}


+  std::vector<PartitionStats> partitions;
+  std::unordered_map<PartitionKey, size_t, PartitionKeyHash> positions;


+    ICEBERG_ASSIGN_OR_RAISE(auto partition_values, internal::ProjectPartitionValues(
+                                                       *partition_type_, *live_file.spec,
+                                                       live_file.file->partition));
+    PartitionKey key{
+        .values = std::move(partition_values),
+        .projected_fields = ProjectedFieldCount(*partition_type_, *live_file.spec)};
+    auto [position, inserted] = positions.try_emplace(key, partitions.size());
+    if (inserted) {
+      partitions.emplace_back(std::move(key.values));
+    }


manuzhang force-pushed the agent/add-system-metadata-tables branch from f4e4c83 to b0f596c Compare August 17, 2026 10:52

Copilot AI lite review requested due to automatic review settings August 17, 2026 10:52

Copilot started reviewing on behalf of manuzhang August 17, 2026 10:59 View session

Copilot AI reviewed Aug 17, 2026

View reviewed changes

Comment thread src/iceberg/test/meson.build Outdated

Comment thread src/iceberg/inspect/history_table.cc

Comment on lines +53 to +55

Result<ArrowArrayStream> HistoryTable::Scan() {

return NotSupported("Scan is not supported for the history table");

}

manuzhang mentioned this pull request Sep 3, 2026

feat(inspect): implement streaming SnapshotsTable scans #801

Merged

Copilot AI review requested due to automatic review settings September 16, 2026 11:13

manuzhang force-pushed the agent/add-system-metadata-tables branch from b0f596c to cf28ec0 Compare September 16, 2026 11:13

Copilot started reviewing on behalf of manuzhang September 16, 2026 11:14 View session

Copilot AI reviewed Sep 16, 2026

View reviewed changes

Comment thread src/iceberg/inspect/metadata_table_util_internal.cc Outdated

Comment thread src/iceberg/inspect/files_table.cc Outdated

Copilot AI review requested due to automatic review settings September 16, 2026 11:56

Copilot started reviewing on behalf of manuzhang September 16, 2026 11:57 View session

Copilot AI reviewed Sep 16, 2026

View reviewed changes

Comment thread src/iceberg/inspect/files_table.cc Outdated

Comment thread src/iceberg/inspect/partitions_table.cc Outdated

Copilot AI review requested due to automatic review settings September 17, 2026 03:38

Copilot AI reviewed Sep 17, 2026

manuzhang force-pushed the agent/add-system-metadata-tables branch from 8fe47a4 to 5c8c962 Compare October 10, 2026 15:15

Copilot AI lite review requested due to automatic review settings October 10, 2026 15:15

Copilot started reviewing on behalf of manuzhang October 10, 2026 15:15 View session

Copilot AI reviewed Oct 10, 2026

View reviewed changes

manuzhang mentioned this pull request Oct 10, 2026

fix(manifest): round-trip decimal partition values #998

Merged

Copilot AI lite review requested due to automatic review settings October 10, 2026 16:22

manuzhang force-pushed the agent/add-system-metadata-tables branch from 5c8c962 to 93234f2 Compare October 10, 2026 16:22

Copilot started reviewing on behalf of manuzhang October 10, 2026 16:22 View session

Copilot AI reviewed Oct 10, 2026

View reviewed changes

manuzhang mentioned this pull request Oct 10, 2026

fix: preserve partition types after source columns are dropped #999

Merged

manuzhang force-pushed the agent/add-system-metadata-tables branch from 93234f2 to a0902e1 Compare October 10, 2026 16:49

Copilot AI lite review requested due to automatic review settings October 10, 2026 16:49

Copilot started reviewing on behalf of manuzhang October 10, 2026 16:50 View session

Copilot AI reviewed Oct 10, 2026

View reviewed changes

manuzhang and others added 2 commits October 13, 2026 18:09

feat(inspect): add system metadata tables

f090354

Co-authored-by: Codex <codex@openai.com>

fix(inspect): align metadata tables with Java

38da5c6

Co-authored-by: Codex <codex@openai.com>

fix(inspect): address metadata table review findings

8556d19

Co-authored-by: Codex <codex@openai.com>

Copilot AI lite review requested due to automatic review settings October 13, 2026 09:24

manuzhang force-pushed the agent/add-system-metadata-tables branch from a0902e1 to 8556d19 Compare October 13, 2026 09:24

Copilot AI reviewed Oct 13, 2026

View reviewed changes

Copilot stopped reviewing on behalf of manuzhang due to an error October 13, 2026 11:59
The job was not acquired by a runner even after multiple attempts.

manuzhang and others added 2 commits October 13, 2026 21:46

refactor(test): organize metadata table tests by table

49d17dd

Co-authored-by: Codex <codex@openai.com>

refactor(inspect): extract refs table into a separate PR

b92aa64

Co-authored-by: Codex <codex@openai.com>

manuzhang mentioned this pull request Oct 13, 2026

feat(inspect): add refs metadata table #1007

Draft

Copilot AI lite review requested due to automatic review settings October 13, 2026 13:54

manuzhang changed the title ~~feat(inspect): add system metadata tables~~ feat(inspect): add files, partitions, and manifests metadata tables Oct 13, 2026

Copilot AI reviewed Oct 13, 2026

View reviewed changes

refactor(test): align metadata fixture paths with refs prerequisite

10d3c38

Move the common metadata test fixture into inspect and update history and snapshots includes to match PR apache#1007. Co-authored-by: Codex <codex@openai.com>

Copilot AI lite review requested due to automatic review settings October 13, 2026 15:37

manuzhang mentioned this pull request Oct 13, 2026

feat(inspect): support selecting columns in metadata table scans #1008

Open

Copilot AI reviewed Oct 13, 2026

View reviewed changes

Copilot stopped reviewing on behalf of manuzhang due to an error October 13, 2026 19:44
The job was not acquired by a runner even after multiple attempts.

Copilot stopped reviewing on behalf of manuzhang due to an error October 13, 2026 20:17
The job was not acquired by a runner even after multiple attempts.

Navigation Menu

Repository navigation

feat(inspect): add files, partitions, and manifests metadata tables - #876

feat(inspect): add files, partitions, and manifests metadata tables#876
manuzhang wants to merge 6 commits into
apache:mainfrom
manuzhang:agent/add-system-metadata-tables

manuzhang commented Aug 6, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		std::vector<PartitionStats> partitions;
		std::unordered_map<PartitionKey, size_t, PartitionKeyHash> positions;

Conversation

manuzhang commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (4)

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (3)

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

manuzhang commented Aug 6, 2026 •

edited

Loading