perf(table): reuse field ID scratch in files metadata - #1919
perf(table): reuse field ID scratch in files metadata#1919fallintoplace wants to merge 1 commit into
Conversation
zeroshade
left a comment
There was a problem hiding this comment.
I found no code issues in the current change, but this head now conflicts with main after #1924. Please rebase before this is approved.
When resolving table/inspect_files.go, please preserve both #1924's borrowed collection access and this PR's shared scratch slice:
columnSizes, keyMetadata, splitOffsets, equalityFieldIDs := dataFileCollections(file)
if err := appendInspectInt64Map(b.columnSizes, columnSizes, &b.mapFieldIDs); err != nil {I tested that combined resolution with the focused race tests repeated 100 times and the full table package; both passed. I'll re-review the resulting SHA after the rebase.
This review was drafted by an AI-assisted tool and confirmed by an Iceberg Go maintainer. After you've rebased and pushed an update, an Iceberg Go maintainer — a real person — will take the next look at the PR. The review applies the project's review criteria; if you think it is mis-applied, please reply on the PR and a maintainer will weigh in.
More on how Iceberg Go handles maintainer review:
CONTRIBUTING.md.
a5b7b1b to
3293535
Compare
What changed
[]intscratch slice while sorting field IDs for the six stats maps written by the files metadata table.Benchmark
go test ./table -run '^$' -bench '^BenchmarkInspectContentFileAppenderDataFileStats/files=(4096|16384)/stats=(32|128)/(borrowed|public)$' -benchmem -count=5 -benchtime=1sRuntime varied between runs, while the allocation reduction was consistent.