Skip to content

fix: demand accounting in MultiBufferWithTimeoutOp - #2218

Merged
jponge merged 5 commits into
smallrye:mainfrom
jponge:fix/2211
Oct 16, 2026
Merged

jponge merged 5 commits into
smallrye:mainfrom
jponge:fix/2211

Conversation

@jponge

@jponge jponge commented Oct 5, 2026

Copy link
Copy Markdown
Member
  • switch to item-granular demand tracking for partial timer flushes
  • fix double terminal signal in checkedComplete()
  • add deterministic and concurrency tests

Closes: #2211
Supersedes: #2212

@jponge
jponge requested a review from ozangunalp October 5, 2026 20:57
@ozangunalp

Copy link
Copy Markdown
Collaborator

@jponge isn't there something odd in the tests? We always get more than we requested from the subscriber.

@jponge

jponge commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Hum the demand should match lists, not necessarily the number of elements in the list, no?

@jponge

jponge commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Oh good catch, I should not have tried to port the other PR tests, and runTask doesn't add to the demand (I thought it did)

- buffer items when demand exhausted instead of BackPressureFailure (finite size)
- preserve BackPressureFailure for unbounded size (every(Duration) API)
- drain pending buffer on new demand via request()
- fix double terminal signal in checkedComplete()
- add deterministic and concurrency tests

Closes: smallrye#2211
Supersedes: smallrye#2212
@jponge

jponge commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

@ozangunalp updated

@ozangunalp

ozangunalp commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

@jponge Sorry this took me soo long.

I've been writing a couple of tests for this operator, regarding completion and requests, trying to understand how we should handle this section of the Javadoc:
* When the upstream {@link Multi} sends the completion event, the produced {@link Multi} emits the current list,
* and sends the completion event. This last list may also not contain {@code size} items. If the upstream {@link Multi}
* sends the completion event before having emitted any event, the completion event is propagated immediately.

This current version fails most of these new tests. To be fair, a couple of these tests fail on the current main as well.

I'll push the tests onto this branch.

@jponge

jponge commented Oct 15, 2026

Copy link
Copy Markdown
Member Author

@ozangunalp I've added a possible fix with drain-loop (yet another one) and some extra tests.

private List<T> current;
private volatile ScheduledFuture<?> task;

private final ArrayDeque<List<T>> readyBatches = new ArrayDeque<>();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory, we should not need more than one list, right?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a matrix of list size / timers where in-flight items require multiple buckets.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ozangunalp I've added a test

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @jponge. The operator is already very complex, to be honest, I can no longer follow it. Tests seem reasonable, and they pass now 🚀

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your great feedback @ozangunalp

@jponge
jponge merged commit 69ccf92 into smallrye:main Oct 16, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MultiBufferWithTimeoutOp: partial timer flush consumes full batch-unit of demand, causing BackPressureFailure

2 participants