Proposal: add a Foyer-backed cache for immutable Lance data files

[zhangstar333]

Motivation

Repeated scans of datasets stored in remote object storage may read the same Lance data ranges multiple times.

Lance's existing Session caches already cover metadata and parsed index structures, but they do not cache data-file contents.

Proposal

Add an optional cache in lance-c backed by Foyer.

The initial implementation would:

• Cache raw range-read bytes from immutable data/*.lance files.
• Support memory and disk caching through Foyer.
• Keep the cache disabled by default and make its capacity and local path configurable.
• Bypass the cache for manifests, transactions, deletion files, index files, and other objects.
• Keep the existing Lance Session metadata and index caches unchanged.

Proposed architecture

The cache would be inserted into the object-store range-read path:

flowchart TD
    A[Lance scanner / reader] --> B[ObjectStore cache wrapper]
    B --> C{Path matches data/*.lance?}

    C -->|No| D[Underlying ObjectStore]
    D --> E[Return range bytes]

    C -->|Yes| F[Foyer cache]
    F -->|Cache hit| E
    F -->|Cache miss| G[Underlying ObjectStore]
    G --> H[Populate Foyer]
    H --> E

Loading

Only immutable Lance data files enter Foyer. All other objects bypass it and continue using Lance's existing read path.

Cache responsibilities

flowchart LR
    A[Lance Dataset] --> B[Session metadata cache]
    A --> C[Session index cache]
    A --> D[Foyer data cache]

    B --> B1[Manifest and transaction metadata]
    B --> B2[Decoded deletion vectors]
    B --> B3[File metadata]

    C --> C1[Opened indices]
    C --> C2[Posting lists and positions]
    C --> C3[Index pages and structures]

    D --> D1[Raw range bytes]
    D1 --> D2[data/*.lance]

Loading

This keeps the responsibilities separate:

• Session metadata cache stores parsed dataset metadata.
• Session index cache stores parsed and opened index structures.
• Foyer stores raw range-read bytes from immutable Lance data files.

Feedback is welcome on the proposed scope, configuration API, and whether Foyer is an appropriate dependency for lance-c.

[zhangstar333]

@jja725 @u70b3 cc

[jja725]

I think it's a bit weird to support Foyer at lance-c layer, we have a similar cache at our internal fork of rust lance-core. Maybe that's something you can consider too

[zhangstar333]

I think it's a bit weird to support Foyer at lance-c layer, we have a similar cache at our internal fork of rust lance-core. Maybe that's something you can consider too

@jja725
Thanks, that makes sense. I would definitely prefer to reuse a data cache provided by the Rust Lance layer.

My understanding is that the public cache infrastructure currently covers metadata and index caches, while data-file range caching is still being tracked like in lance-format/lance#6483 and is not yet available upstream.

For our current needs, I am considering integrating Foyer in lance-c as an interim solution, rather than implementing another cache engine from scratch. The integration would be kept isolated behind a small adapter and configuration API.

Once Rust Lance provides a data-file cache, lance-c should be able to switch to the upstream implementation and only expose its configuration through the C API.

If the implementation in your internal fork may be upstreamed, or if its design can be shared, I would be very happy to align with and reuse it.