Skip to content

Fix subscription WAL retention at replay cursor - #18769

Merged
jt2594838 merged 3 commits into
apache:masterfrom
Caideyipi:fix/subscription-wal-retention-replay-cursor
Sep 30, 2026
Merged

jt2594838 merged 3 commits into
apache:masterfrom
Caideyipi:fix/subscription-wal-retention-replay-cursor

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Problem

Subscription WAL retention only considered committed progress. Committed progress can advance on another consumer or replica while a local subscription queue is still replaying older WAL entries, allowing WAL deletion to pass the queue's unread replay cursor.

Changes

  • Retain WAL from the earlier of the committed-progress boundary and the local replay cursor.
  • Pass the generalized queue retention boundary through the subscription queue registry and retention calculator.
  • Add a regression test covering committed progress ahead of the replay cursor.

Verification

  • mvnw.cmd spotless:apply -pl iotdb-core/consensus,iotdb-core/datanode
  • mvnw.cmd -nsu test -pl iotdb-core/datanode -am -Dtest=ConsensusPrefetchingQueueTest#testReplayCursorBoundsWalRetentionWhenCommittedProgressIsAhead -DfailIfNoTests=false -Dsurefire.failIfNoSpecifiedTests=false

This PR has:

  • been self-reviewed.
  • added comments explaining the intent where it is not obvious.
  • added a unit test covering the new code path.

Key changed classes
  • ConsensusPrefetchingQueue
  • SubscriptionQueueRegistry
  • SubscriptionWalRetentionCalculator
  • IoTConsensusServerImpl

@jt2594838
jt2594838 merged commit 5d389dd into apache:master Sep 30, 2026
38 of 39 checks passed
@jt2594838
jt2594838 deleted the fix/subscription-wal-retention-replay-cursor branch September 30, 2026 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants