Skip to content

[seed f1a543] Evaluation change; do not merge #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sshiv012 wants to merge 1 commit into
base: coderabbit-eval/base
Choose a base branch
from
Open

[seed f1a543] Evaluation change; do not merge #27

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions .../main/scala/org/apache/texera/amber/operator/source/scan/text/TextInputSourceOpExec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,11 @@ class TextInputSourceOpExec private[text] (
(if (desc.attributeType.isSingle) {
Iterator(desc.textInput)
} else {
// `slice(offset, offset + limit)` overflows Int when the limit is absent
// (it defaults to Int.MaxValue) or large, making `until <= from` and
// silently yielding no rows.
desc.textInput.linesIterator
.drop(desc.fileScanOffset.getOrElse(0))
.take(desc.fileScanLimit.getOrElse(Int.MaxValue))
// Emit the [offset, offset + limit) window of lines in one call.
desc.textInput.linesIterator.slice(
desc.fileScanOffset.getOrElse(0),
desc.fileScanOffset.getOrElse(0) + desc.fileScanLimit.getOrElse(Int.MaxValue)
Comment on lines +39 to +40

@coderabbitai coderabbitai Bot Aug 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n \
  --glob 'build.sbt' \
  --glob '*.sbt' \
  --glob 'pom.xml' \
  --glob 'gradle.properties' \
  'scalaVersion|scala-library|scala.binary.version' . || true

rg -n -C 6 \
  'fileScanOffset|fileScanLimit|linesIterator\.slice|\.drop\(|\.take\(' \
  common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text \
  common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/text || true

Repository: sshiv012/texera

Length of output: 17393

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- descriptor ---'
sed -n '1,110p' common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text/TextSourceOpDesc.scala

printf '%s\n' '--- relevant tests ---'
sed -n '230,290p' common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/text/TextInputSourceOpDescSpec.scala

printf '%s\n' '--- validation and usages ---'
rg -n -C 4 \
  'fileScanOffset|fileScanLimit|TextInputSourceOpDesc' \
  common/workflow-operator/src/main common/workflow-operator/src/test \
  | head -n 500

printf '%s\n' '--- deterministic overflow probe ---'
python3 - <<'PY'
INT_MAX = 2**31 - 1

def int32(value):
    value &= 0xffffffff
    return value - 2**32 if value >= 2**31 else value

offset = 1
limit = INT_MAX
endpoint = int32(offset + limit)
print({
    "offset": offset,
    "limit": limit,
    "mathematical_endpoint": offset + limit,
    "int32_endpoint": endpoint,
    "wrapped_negative": endpoint < 0,
    "clamped_long_endpoint": min(offset + limit, INT_MAX),
})
PY

printf '%s\n' '--- Scala 2.13 Iterator.slice implementation ---'
curl -fsSL \
  https://raw.githubusercontent.com/scala/scala/v2.13.18/src/library/scala/collection/Iterator.scala \
  | rg -n -A 18 -B 4 'def slice'

Repository: sshiv012/texera

Length of output: 50371

🌐 Web query:

Scala 2.13.18 Iterator.slice negative until implementation

💡 Result:

In Scala 2.13.18, the Iterator.slice(from: Int, until: Int) method is implemented by delegating to a protected method named sliceIterator [1][2]. The public slice method normalizes the until parameter by calling until max 0 [1][2]. Consequently, if a negative value is passed as the until parameter to the public slice method, it is treated as 0, which results in an empty iterator (unless from is also negative or such that the slice is invalid/empty, as the slice logic handles standard index bounds) [1][3][2]. Specifically, the implementation is as follows: def slice(from: Int, until: Int): Iterator[A] = sliceIterator(from, until max 0) [1][2] The protected sliceIterator method, however, contains logic that explicitly treats negative values as unbounded [1][2]: protected def sliceIterator(from: Int, until: Int): Iterator[A] = { val lo = from max 0 val rest = if (until < 0) -1 // unbounded else if (until <= lo) 0 // empty else until - lo // finite if (rest == 0) Iterator.empty else new Iterator.SliceIterator(this, lo, rest) } [1][2] Because the public slice method always forces until to be at least 0 before passing it to sliceIterator, the "unbounded" path (where until < 0) inside sliceIterator is effectively unreachable through the public API [1][2]. The normalization (until max 0) ensures that any negative input for until is converted to 0, which then triggers the "empty" condition (until <= lo) or results in an empty iterator [1][3][2].

Citations:

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- text source test cases ---'
sed -n '40,235p' common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/source/scan/text/TextInputSourceOpDescSpec.scala

printf '%s\n' '--- file scan implementation ---'
sed -n '1,180p' common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/file/FileScanUtils.scala

printf '%s\n' '--- focused Scala slice source ---'
curl -fsSL \
  https://raw.githubusercontent.com/scala/scala/v2.13.18/src/library/scala/collection/Iterator.scala \
  | sed -n '/def slice(from: Int, until: Int)/,+8p'

printf '%s\n' '--- behavioral model ---'
python3 - <<'PY'
INT_MAX = 2**31 - 1

def int32(value):
    value &= 0xffffffff
    return value - 2**32 if value >= 2**31 else value

offset = 1
limit = INT_MAX
wrapped_until = int32(offset + limit)
print(f"Scala Int endpoint: {wrapped_until}")
print(f"Iterator.slice until after normalization: {max(wrapped_until, 0)}")
print(f"Existing slice window is empty: {max(wrapped_until, 0) <= max(offset, 0)}")

long_until = min(offset + limit, INT_MAX)
print(f"Long-and-clamped endpoint: {long_until}")
print(f"Correct offset-only window is non-empty for input length > {offset}: {long_until > offset}")
PY

Repository: sshiv012/texera

Length of output: 14534

Prevent Int overflow in the slice endpoint.

When fileScanOffset is positive and fileScanLimit is absent or Int.MaxValue, line 40 can wrap the endpoint to a negative value. Iterator.slice then returns no lines. Use Long arithmetic with an Int.MaxValue cap, or handle the unbounded case separately.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/source/scan/text/TextInputSourceOpExec.scala`
around lines 39 - 40, Update the slice endpoint calculation in
TextInputSourceOpExec to avoid Int overflow when fileScanOffset is combined with
an absent or maximal fileScanLimit. Compute the endpoint using Long arithmetic
and cap it at Int.MaxValue, or handle the unbounded limit separately, while
preserving the existing offset and bounded-limit behavior.

)
}).map(line =>
TupleLike(desc.attributeType match {
case FileAttributeType.SINGLE_STRING => line
Expand Down