Skip to content

fix(expressions): accept a bare true / false as an AND / OR / NOT operand - #3917

Open
jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:parser-bare-boolean-operand
Open

fix(expressions): accept a bare true / false as an AND / OR / NOT operand#3917
jackylee-ch wants to merge 3 commits into
apache:mainfrom
jackylee-ch:parser-bare-boolean-operand

Conversation

@jackylee-ch

@jackylee-ch jackylee-ch commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

The BooleanLiteral to AlwaysTrue/AlwaysFalse conversion is attached to the whole expression, so a bare boolean used as an operand reaches And/Or/Not as a BooleanLiteral and fails pydantic validation. The two spellings of the same filter disagree:

parse("(false) or foo = 1")  # EqualTo(foo, 1)
parse("false or foo = 1")    # ValidationError: 1 validation error for Or

Seeding a filter with true and appending clauses is a common way to build one, so row_filter="true and status = 'x'" raises instead of scanning.

The parenthesized form works by accident: infix_notation returns the same Forward that a parenthesized sub-expression recurses through, so the fold runs only when an operand happens to take that path. Attaching it to predicate instead folds every operand. literal and literal_set consume the boolean further in, so foo = true and foo in (true, false) keep the raw BooleanLiteral. The top-level attachment is then unreachable and is removed.

Are these changes tested?

Yes, test_boolean_as_operand covers and/or/not in both positions, and test_boolean_as_literal_is_unchanged pins foo = true and foo in (true, false). The seven operand cases fail without the change; the literal case passes either way.

Are there any user-facing changes?

Filters using a bare true/false as an operand parse instead of raising.

…rand

The BooleanLiteral to AlwaysTrue/AlwaysFalse conversion was attached only to
the whole expression, so a bare boolean reached And/Or/Not as a
BooleanLiteral and failed pydantic validation:

    parse("(false) or foo = 1")  ->  EqualTo(foo, 1)
    parse("false or foo = 1")    ->  ValidationError: 1 validation error for Or

Seeding a filter with `true` and appending clauses is a common way to build
a row_filter, so `row_filter="true and status = 'x'"` raised instead of
scanning. Give `predicate` its own copy of the boolean element that folds to
AlwaysTrue/AlwaysFalse; `literal` and `literal_set` keep the raw
BooleanLiteral, so `foo = true` and `foo in (true, false)` are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 7, 2026 03:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped, aligns with existing AlwaysTrue/AlwaysFalse absorption logic, and is covered by targeted new tests.

Pull request overview

This PR updates the expressions parser so that a bare true/false token used as an operand to and/or/not is folded into AlwaysTrue/AlwaysFalse during parsing, avoiding pydantic validation failures while preserving boolean literals inside comparisons and IN lists.

Changes:

  • Introduce a dedicated boolean grammar element for operand positions that folds to AlwaysTrue/AlwaysFalse.
  • Adjust the parser’s predicate rule to use the folded-boolean element instead of the literal-boolean element.
  • Add tests covering boolean operands in and/or/not, and pin behavior for foo = true / foo in (true, false).
File summaries
File Description
pyiceberg/expressions/parser.py Adds a separate boolean operand parse element and uses it in predicate so And/Or/Not receive valid BooleanExpression operands.
tests/expressions/test_parser.py Adds regression tests to ensure bare boolean operands parse and boolean literals in comparisons/sets remain unchanged.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pyiceberg/expressions/parser.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

@Fokko Fokko left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm okay widening this, thanks @jackylee-ch. I left one comment which I think we might want to clean up

Comment thread pyiceberg/expressions/parser.py Outdated
Comment on lines +124 to +126
# As an operand a bare boolean has to fold to AlwaysTrue/AlwaysFalse. This needs its own
# copy because `literal` and `literal_set` keep the raw BooleanLiteral for `foo = true`.
always_boolean = boolean.copy()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this copy? We never mutate boolean, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do mutate it: set_parse_action replaces in place, and literal/literal_set hold the same element. So the copy is gone — the existing fold now hangs off predicate. That is where the bug was: infix_notation returns the same Forward parenthesised operands recurse through, so only (true) and x folded.

The extra `boolean.copy()` is not needed. `handle_always_expression` already
did this fold; it was only attached to the whole expression, which is why
`(true) and foo = 1` worked and `true and foo = 1` did not -- infix_notation
returns the same Forward that a parenthesized sub-expression recurses through,
so the fold ran only when an operand happened to take that path.

Attach it to `predicate` instead so every operand folds. `literal` and
`literal_set` consume the boolean further in, so `foo = true` and
`foo in (true, false)` keep the raw `BooleanLiteral`. The top-level attachment
is now unreachable and is removed. Tests are unchanged.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants