fix(rollout): `observe()` row_count systematically undercounts unflushed rows; `add()` return value is now meaningless

[beinan]

Two small API-surface problems that both fall out of the #181 flush decoupling.

1. RolloutObservation::row_count undercounts

crates/lance-context-core/src/rollout_store.rs:119:

/// Logical row count across the base table and every flushed MemWAL shard.
pub row_count: i64,

Rows sitting in an unsealed active memtable are counted by neither the base table nor any flushed generation. Before #181 every append was sealed before returning, so this was accurate. Now, in steady state, row_count is permanently short by roughly one flush interval's worth of writes.

Consumers affected: the control-plane stats scanner, capacity/compaction decisions derived from it, and any row count shown in the UI.

Options:

• document the field as "flushed rows only", and/or
• add an unflushed_rows / flush-lag field to RolloutObservation so the gap is observable rather than invisible.

2. add()'s return value no longer carries information

crates/lance-context-core/src/rollout_store.rs:575:

Ok(self.dataset.manifest.version)

MemWAL appends do not advance the base dataset version — the doc at rollout_store.rs:540-542 already admits it is "retained for API compatibility, not as a per-append snapshot handle". Post-#181 it does not even signal that a seal completed. It is now a constant unrelated to the write the caller just performed, which is more misleading than returning nothing.

Options:

• change to LanceResult<()>, or
• return a real write token (generation id or WAL position) that a caller could later wait on — which would also compose nicely with feat(rollout): expose a synchronous-visibility write option on the HTTP API #186.

Both are breaking changes to the core signature, so worth bundling into one version bump.

Related

• feat(rollout): concurrent writes with periodic flush #181, docs(rollout): RolloutStore::add doc comment still promises synchronous visibility #183, feat(rollout): expose a synchronous-visibility write option on the HTTP API #186