Skip to content

fix(stream): keep pending counters consistent - #3596

Draft
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-pending-accounting
Draft

fix(stream): keep pending counters consistent#3596
wengsht wants to merge 1 commit into
apache:unstablefrom
wengsht:fix-stream-pending-accounting

Conversation

@wengsht

@wengsht wengsht commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Problem

Stream pending counters can diverge from the real PEL and wrap past INT64_MAX, making XINFO GROUPS and XINFO CONSUMERS undecodable.

Three related command behaviors cause or amplify the divergence:

  1. XACK key group id id reads the same PEL record twice from one snapshot, reports two acknowledgements, deletes one record, and decrements the cached group/consumer counters twice.
  2. XAUTOCLAIM returns an expired entry already owned by the target consumer without refreshing its delivery time. Repeating the command returns the same ID again.
  3. XCLAIM to the current consumer decrements and increments the same consumer through separate metadata writes; the final write increments its cached pending count.

Together, a reclaim loop can produce multiple deliveries of one ID, a client can coalesce those duplicate IDs into one XACK, and the unsigned counters underflow.

Minimal reproduction before this patch:

XREADGROUP ... >              # one real PEL entry
XAUTOCLAIM ... consumer ...   # returns 1-0
XAUTOCLAIM ... consumer ...   # returns 1-0 again
XACK key group 1-0 1-0       # returns 2
XPENDING key group            # real PEL = 0
XINFO GROUPS key              # Bad integer value

Change

  • Deduplicate IDs within XACK and XCLAIM.
  • Make group and consumer pending decrements saturating as a final corruption guard.
  • Aggregate XCLAIM ownership decrements per original consumer.
  • Keep same-consumer XCLAIM ownership accounting neutral.
  • Refresh XAUTOCLAIM delivery time for every claimed entry, including entries already owned by the target consumer; preserve JUSTID retry-count semantics.

Tests

Command-level regressions verify:

  • duplicate XACK IDs acknowledge one PEL record and leave all pending views at zero;
  • same-consumer XCLAIM leaves group and consumer counts unchanged;
  • same-consumer XAUTOCLAIM refreshes idle time and cannot immediately return the same ID again.

Validation:

  • ./x.py format
  • ./x.py check format
  • fresh ./x.py build build-make --unittest -j 8
  • targeted Go stream regressions pass against the fresh binary

./x.py check tidy could not run locally because run-clang-tidy is not installed.

AI assistance was used for diagnosis and drafting; I reviewed the code, reproduction, and tests.

@wengsht
wengsht force-pushed the fix-stream-pending-accounting branch 2 times, most recently from 7368c48 to 4a48cae Compare August 18, 2026 21:18
@PragmaTwice
PragmaTwice requested a balanced review from Copilot September 1, 2026 11:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Keeps stream PEL counters consistent across acknowledgment and claim operations.

Changes:

  • Deduplicates XACK and XCLAIM IDs.
  • Corrects ownership accounting and saturates decrements.
  • Refreshes XAUTOCLAIM delivery timestamps and adds regressions.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/types/redis_stream.cc Corrects pending-counter and claim behavior.
tests/gocase/unit/type/stream/stream_test.go Adds stream command regressions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/types/redis_stream.cc
Comment on lines +488 to +490
if (!pel_entry.consumer_name.empty() && pel_entry.consumer_name != consumer_name) {
original_consumer_decrements[pel_entry.consumer_name] += 1;
consumer_metadata.pending_number += 1;
Comment thread src/types/redis_stream.cc
Comment on lines 449 to +451
for (const auto &id : entry_ids) {
if (!seen.insert(id).second) {
continue;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants