Skip to content

fix(factories): infer typed array elements as Binary - #510

Open
kentkwu wants to merge 2 commits into
apache:mainfrom
kentkwu:fix/infer-typed-array-elements
Open

kentkwu wants to merge 2 commits into
apache:mainfrom
kentkwu:fix/infer-typed-array-elements

Conversation

@kentkwu

@kentkwu kentkwu commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Summary

Infers arrays of typed arrays as proper Arrow types, so vectorFromArray([uint8Array, ...]) produces a Binary vector instead of a struct with one Float64 field per byte.

The problem

Type inference treated typed arrays as plain objects and built a struct from their index keys. An array of 16-byte Uint8Array values became Struct<{0: Float64, ..., 15: Float64}> instead of Binary. The output was also lossy: values of different lengths were padded with nulls to the longest one, and the static type resolved to never.

Behavior change

This changes the inferred type for these inputs, but it's a bug fix: the old output didn't preserve the input values and didn't match its own static type. Code that needs a specific type can still pass one explicitly to vectorFromArray.

Changes

  • vectorFromArray infers Uint8Array (and Node Buffer) elements as Binary, and other typed arrays as List of their element type (e.g. Int32Array → List<Int32>)
  • tableFromArrays and tableFromJSON pick this up for columns and struct fields, so tableFromArrays({ values: [[bytes, bytes]] }) gives List<Binary>
  • JavaScriptArrayDataType resolves to the same types at compile time instead of never
  • Mixed typed array kinds and DataView elements throw the existing "explicit type declaration expected" error

Test Plan

  • npx jest -c jestconfigs/jest.src.config.js test/unit
  • npx eslint src/factories.ts src/interfaces.ts test/unit/vector/vector-tests.ts
  • Checked inferred static types for Uint8Array, Buffer, Int32Array, Uint8ClampedArray, and BigInt64Array with tsc

vectorFromArray and tableFromArrays treated typed array elements as
plain objects, so an array of Uint8Array became a Struct with one
Float64 field per byte. Ragged lengths were padded with nulls, and the
static type resolved to never.

Infer Uint8Array (and Buffer) elements as Binary and other typed arrays
as List of their element type, keeping the static types in step.
Mixed typed array kinds and DataView now throw like other ambiguous
inputs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant