Skip to content

Fix zero-offset limit skipping first deduplicated row - #5701

Merged
qianheng-aws merged 1 commit into
opensearch-project:mainfrom
ruanyl:codex/fix-limit-dedupe-first-row
Aug 20, 2026
Merged

Fix zero-offset limit skipping first deduplicated row#5701
qianheng-aws merged 1 commit into
opensearch-project:mainfrom
ruanyl:codex/fix-limit-dedupe-first-row

Conversation

@ruanyl

@ruanyl ruanyl commented Aug 18, 2026

Copy link
Copy Markdown
Member

Description

Fix LimitOperator.open() so a zero offset does not probe its child iterator.

When PPL fetch_size adds a zero-offset limit around a stateful operator such as
DedupeOperator, the existing condition calls input.hasNext() even though no
rows need to be skipped. DedupeOperator.hasNext() consumes and caches the first
unique row, so the subsequent normal iteration replaces it with the second row
and omits the first result.

This change checks the offset before calling the child iterator and adds a
regression test covering a zero-offset limit around deduplication.

Related Issues

Resolves #5699

Related: opensearch-project/OpenSearch-Dashboards#12606

Testing

  • Confirmed the new regression test fails on main with expected: <3> but was: <2>.
  • ./gradlew :core:test
  • ./gradlew :core:spotlessCheck
  • ./gradlew :core:test --tests org.opensearch.sql.planner.physical.LimitOperatorTest

Check List

  • New functionality includes testing.
  • New functionality has been documented. Not applicable; this restores expected iterator behavior.
  • New functionality has javadoc added. Not applicable.
  • New functionality has a user manual doc added. Not applicable.
  • New PPL command checklist all confirmed. Not applicable.
  • API changes companion pull request created. Not applicable.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created. Not applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Yulong Ruan <ruanyl@amazon.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ No major issues detected

@github-actions

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Increment counter after consuming row

The count variable is incremented before consuming the row with input.next(). This
creates a logical inconsistency where count represents rows to skip rather than rows
already skipped. Consider incrementing count after input.next() to maintain semantic
clarity and prevent potential off-by-one errors in future modifications.

core/src/main/java/org/opensearch/sql/planner/physical/LimitOperator.java [42-45]

 while (count < offset && input.hasNext()) {
+  input.next();
   count++;
-  input.next();
 }
Suggestion importance[1-10]: 3

__

Why: While the suggestion improves semantic clarity by incrementing count after consuming the row, the current implementation is functionally correct. The order of count++ and input.next() doesn't affect the behavior since both operations must complete within the loop iteration. This is a minor style improvement rather than a bug fix.

Low

@qianheng-aws qianheng-aws added bug Something isn't working bugFix and removed bug Something isn't working labels Aug 20, 2026

@qianheng-aws qianheng-aws left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qianheng-aws
qianheng-aws merged commit 81a399b into opensearch-project:main Aug 20, 2026
47 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Dashboard variables definition with PPL on prometheus datasource swallows first row

2 participants