Skip to content

Lance blob columns render as descriptor structs, not as media #88

Description

@gordonmurray

PR #72 added inline preview for media held in binary columns. It covers binary and large_binary, but not the column type Lance provides for storing large media.

A blob column does not return its bytes through the read path in /rows. It returns a descriptor struct, so there are no bytes for serialize_value to detect, and the value reaches the frontend through the generic object renderer.

Lance marks a blob field two ways, and both behave like this. Checked against lancedb 0.36.0 with a 208-byte PNG in each, through take_offsets([0]).to_arrow(), which is what /rows calls:

  • Legacy, field metadata key lance-encoding:blob. The column comes back as struct<position: uint64, size: uint64> and serializes to {"position": 0, "size": 208}.
  • Current, field metadata key ARROW:extension:name set to lance.blob.v2. The column comes back as struct<kind: uint8, position: uint64, size: uint64, blob_id: uint32, blob_uri: string, _lance_row_id: uint64> and serializes to {"kind": 0, "position": 0, "size": 208, "blob_id": 0, "blob_uri": "", "_lance_row_id": 0}.

The same bytes in a plain large_binary column serialize to a media object and render as an image.

This is the storage mode Lance intends for images, video, and audio, and the v2 form is what its guide recommends for new datasets. So the datasets most likely to want a preview are the ones that do not get one.

The inline size limit is not involved. No value of MEDIA_INLINE_MAX_BYTES changes this, because the bytes never reach the serializer.

Blob v2 is serviceable

lancedb 0.36.0 has a first-class API for v2 columns:

  • table.blob_columns() lists them, so /columns can flag them the way it already flags vectors.
  • table.fetch_blobs(column, row_ids) returns the bytes, for small values.
  • table.fetch_blob_files(column, row_ids) returns lazy seekable BlobFile handles with seek and read_range, which suit large values and range reads.

The row id these need comes back in the row itself. take_offsets([0]).to_arrow() on a v2 column includes _lance_row_id, so a fetch route needs no extra read to find it.

Two constraints. None of these methods exist in 0.33.0 or earlier, so the older images need a capability check and a fallback to current behaviour, in the same shape as the list_tables() fallback from #54. And all three are local table methods. On a remote table they raise NotImplementedError, which matters only if #12 lands and the viewer reads from object storage.

Legacy blob columns have no route

The same APIs do not serve the legacy form. blob_columns() returns [] for it, and both fetch methods raise ValueError: column 'img' is a legacy blob column; blob APIs require blob v2 columns.

Reading those bytes needs the dataset-level scanner in pylance with blob_handling="all_binary", which reads the values whole and gives up the lazy access. It may be reasonable to support v2 only and leave the legacy form rendering as it does now.

Why it is worth doing

A fetch route for large ordinary binary is wanted anyway, since anything above the inline limit currently shows its size and nothing else. One endpoint returning the bytes for a dataset, a column, and a row would serve both. Reading a v2 value through fetch_blob_files rather than fetch_blobs also gives read_range on the handle, so that endpoint could answer HTTP range requests and let video stream rather than be read whole.

Part of #3.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions