Skip to content

fix(stream): bound range reads by metadata entries - #3595

Draft
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-range-metadata-frontier
Draft

fix(stream): bound range reads by metadata entries#3595
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-range-metadata-frontier

Conversation

@wengsht

@wengsht wengsht commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Problem

Stream metadata defines the logical first and last live entry IDs, but the shared range path scans the raw RocksDB entry-key interval supplied by the caller.

If raw keys outside the metadata bounds remain after an interrupted or older slot migration, they become visible again through XRANGE, XREVRANGE, and new-message XREADGROUP reads. This makes XLEN / XINFO STREAM disagree with XRANGE and can pin retention: a sweeper repeatedly starts from a stale pre-frontier entry while XTRIM correctly treats the newer metadata frontier as authoritative.

The completed #3592 prevents new trim DeleteRange records from being lost during incremental slot migration, but existing divergent streams still need a read-side invariant.

Change

Clamp the shared stream range interval to metadata's [first_entry_id, last_entry_id] bounds:

  • forward and reverse reads use the same logical bounds;
  • exclusion flags are cleared only when their boundary was replaced by a metadata bound;
  • metadata-empty streams return no entries even if stale raw keys exist.

This is read-only hardening. It does not add cleanup writes to read commands or change stream metadata.

Test

The regression trims a stream, reinserts raw entry keys below and above its metadata bounds, and verifies:

  • forward range returns only metadata-live entries;
  • reverse range returns only metadata-live entries;
  • an exact lookup of a stale entry returns empty.

Validation:

  • ./x.py format
  • ./x.py check format
  • fresh ./x.py build build-make --unittest -j 8
  • all 99 RedisStreamTest.* cases pass

AI assistance was used for diagnosis and drafting; I reviewed the code and tests.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Bounds stream range reads to metadata-defined live entries, preventing stale RocksDB keys from resurfacing.

Changes:

  • Clamp forward and reverse ranges to metadata bounds.
  • Preserve endpoint exclusion semantics.
  • Add regression coverage for stale raw entries.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/types/redis_stream.cc Enforces metadata bounds in shared range reads.
tests/cppunit/types/stream_test.cc Tests forward, reverse, and exact stale-entry reads.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants