PR #72 added inline preview for media held in binary columns. It covers binary and large_binary, but not the column type Lance provides for storing large media.
A blob column does not return its bytes through the read path in /rows. It returns a descriptor struct, so there are no bytes for serialize_value to detect, and the value reaches the frontend through the generic object renderer.
Lance marks a blob field two ways, and both behave like this. Checked against lancedb 0.36.0 with a 208-byte PNG in each, through take_offsets([0]).to_arrow(), which is what /rows calls:
- Legacy, field metadata key
lance-encoding:blob. The column comes back as struct<position: uint64, size: uint64> and serializes to {"position": 0, "size": 208}.
- Current, field metadata key
ARROW:extension:name set to lance.blob.v2. The column comes back as struct<kind: uint8, position: uint64, size: uint64, blob_id: uint32, blob_uri: string, _lance_row_id: uint64> and serializes to {"kind": 0, "position": 0, "size": 208, "blob_id": 0, "blob_uri": "", "_lance_row_id": 0}.
The same bytes in a plain large_binary column serialize to a media object and render as an image.
This is the storage mode Lance intends for images, video, and audio, and the v2 form is what its guide recommends for new datasets. So the datasets most likely to want a preview are the ones that do not get one.
The inline size limit is not involved. No value of MEDIA_INLINE_MAX_BYTES changes this, because the bytes never reach the serializer.
Blob v2 is serviceable
lancedb 0.36.0 has a first-class API for v2 columns:
table.blob_columns() lists them, so /columns can flag them the way it already flags vectors.
table.fetch_blobs(column, row_ids) returns the bytes, for small values.
table.fetch_blob_files(column, row_ids) returns lazy seekable BlobFile handles with seek and read_range, which suit large values and range reads.
The row id these need comes back in the row itself. take_offsets([0]).to_arrow() on a v2 column includes _lance_row_id, so a fetch route needs no extra read to find it.
Two constraints. None of these methods exist in 0.33.0 or earlier, so the older images need a capability check and a fallback to current behaviour, in the same shape as the list_tables() fallback from #54. And all three are local table methods. On a remote table they raise NotImplementedError, which matters only if #12 lands and the viewer reads from object storage.
Legacy blob columns have no route
The same APIs do not serve the legacy form. blob_columns() returns [] for it, and both fetch methods raise ValueError: column 'img' is a legacy blob column; blob APIs require blob v2 columns.
Reading those bytes needs the dataset-level scanner in pylance with blob_handling="all_binary", which reads the values whole and gives up the lazy access. It may be reasonable to support v2 only and leave the legacy form rendering as it does now.
Why it is worth doing
A fetch route for large ordinary binary is wanted anyway, since anything above the inline limit currently shows its size and nothing else. One endpoint returning the bytes for a dataset, a column, and a row would serve both. Reading a v2 value through fetch_blob_files rather than fetch_blobs also gives read_range on the handle, so that endpoint could answer HTTP range requests and let video stream rather than be read whole.
Part of #3.
PR #72 added inline preview for media held in binary columns. It covers
binaryandlarge_binary, but not the column type Lance provides for storing large media.A blob column does not return its bytes through the read path in
/rows. It returns a descriptor struct, so there are no bytes forserialize_valueto detect, and the value reaches the frontend through the generic object renderer.Lance marks a blob field two ways, and both behave like this. Checked against lancedb 0.36.0 with a 208-byte PNG in each, through
take_offsets([0]).to_arrow(), which is what/rowscalls:lance-encoding:blob. The column comes back asstruct<position: uint64, size: uint64>and serializes to{"position": 0, "size": 208}.ARROW:extension:nameset tolance.blob.v2. The column comes back asstruct<kind: uint8, position: uint64, size: uint64, blob_id: uint32, blob_uri: string, _lance_row_id: uint64>and serializes to{"kind": 0, "position": 0, "size": 208, "blob_id": 0, "blob_uri": "", "_lance_row_id": 0}.The same bytes in a plain
large_binarycolumn serialize to a media object and render as an image.This is the storage mode Lance intends for images, video, and audio, and the v2 form is what its guide recommends for new datasets. So the datasets most likely to want a preview are the ones that do not get one.
The inline size limit is not involved. No value of
MEDIA_INLINE_MAX_BYTESchanges this, because the bytes never reach the serializer.Blob v2 is serviceable
lancedb 0.36.0 has a first-class API for v2 columns:
table.blob_columns()lists them, so/columnscan flag them the way it already flags vectors.table.fetch_blobs(column, row_ids)returns the bytes, for small values.table.fetch_blob_files(column, row_ids)returns lazy seekableBlobFilehandles withseekandread_range, which suit large values and range reads.The row id these need comes back in the row itself.
take_offsets([0]).to_arrow()on a v2 column includes_lance_row_id, so a fetch route needs no extra read to find it.Two constraints. None of these methods exist in 0.33.0 or earlier, so the older images need a capability check and a fallback to current behaviour, in the same shape as the
list_tables()fallback from #54. And all three are local table methods. On a remote table they raiseNotImplementedError, which matters only if #12 lands and the viewer reads from object storage.Legacy blob columns have no route
The same APIs do not serve the legacy form.
blob_columns()returns[]for it, and both fetch methods raiseValueError: column 'img' is a legacy blob column; blob APIs require blob v2 columns.Reading those bytes needs the dataset-level scanner in pylance with
blob_handling="all_binary", which reads the values whole and gives up the lazy access. It may be reasonable to support v2 only and leave the legacy form rendering as it does now.Why it is worth doing
A fetch route for large ordinary binary is wanted anyway, since anything above the inline limit currently shows its size and nothing else. One endpoint returning the bytes for a dataset, a column, and a row would serve both. Reading a v2 value through
fetch_blob_filesrather thanfetch_blobsalso givesread_rangeon the handle, so that endpoint could answer HTTP range requests and let video stream rather than be read whole.Part of #3.