Summary
Since #181, a rollout append is durable on return but not visible until the next flush tick (default 30s). RolloutStore::flush() exists (crates/lance-context-core/src/rollout_store.rs:621-640) but is not reachable from the HTTP API, so callers that need read-your-write have no way to get it short of setting the global flush interval very low.
Motivating cases
- an RL training loop that appends a trajectory and immediately reads it back to verify
- write-then-assert in integration tests
- any client doing
POST /rollouts/{name}/records followed by GET
All of these silently observe a missing row for up to rollout_flush_interval_secs.
Proposal
Add an opt-in per-request flag on the rollout append route (crates/lance-context-server/src/routes/rollouts.rs), e.g. ?visible=true or an X-Rollout-Visible header, that performs add() followed by flush() before responding.
Alternative considered
Lowering the global flush interval affects every store in the process and re-serializes writes that do not need the guarantee. A per-request opt-in keeps the choice with the caller.
Related
Summary
Since #181, a rollout append is durable on return but not visible until the next flush tick (default 30s).
RolloutStore::flush()exists (crates/lance-context-core/src/rollout_store.rs:621-640) but is not reachable from the HTTP API, so callers that need read-your-write have no way to get it short of setting the global flush interval very low.Motivating cases
POST /rollouts/{name}/recordsfollowed byGETAll of these silently observe a missing row for up to
rollout_flush_interval_secs.Proposal
Add an opt-in per-request flag on the rollout append route (
crates/lance-context-server/src/routes/rollouts.rs), e.g.?visible=trueor anX-Rollout-Visibleheader, that performsadd()followed byflush()before responding.add(..., wait_visible=True)Alternative considered
Lowering the global flush interval affects every store in the process and re-serializes writes that do not need the guarantee. A per-request opt-in keeps the choice with the caller.
Related
RolloutStore::adddoc comment still promises synchronous visibility #183