Skip to content

feat(sql): support streaming IN subqueries - #2540

Open
pythonyx135793 wants to merge 10 commits into
getdozer:mainfrom
pythonyx135793:codex/dozer1659-dynamic-in-r1367
Open

pythonyx135793 wants to merge 10 commits into
getdozer:mainfrom
pythonyx135793:codex/dozer1659-dynamic-in-r1367

Conversation

@pythonyx135793

@pythonyx135793 pythonyx135793 commented Oct 25, 2026 •

Copy link
Copy Markdown

/claim #1659

Adds live IN (SELECT ...) and NOT IN predicates to WHERE. Each inner query feeds a stateful selection processor alongside the outer stream. Changes on either side produce the matching inserts, updates and deletes; duplicate inner values are reference-counted, and duplicate outer rows keep their multiplicity.

This also handles NULL/empty results, compound predicates, multiple or nested subqueries, and the planner's projections, joins, grouping/HAVING, CTEs and UNION. The existing static-IN path stays intact. Unsupported correlation, row-valued membership and unsupported relational clauses return explicit errors.

Validation on Linux, Rust 1.77.2, source 80027f3d:

  • SQL, expression and core default-feature suites: 332 passed, zero failures, one pre-existing ignored test.
  • Changed-source formatting, scoped Clippy and the actual source → SQL → DAG → sink demo passed.
  • The same public-API regression fails against the original source because dynamic membership is unsupported there.
  • Real-DAG tests cover changing membership, all outer update transitions, duplicates, NULLs, batches, lexical CTE scope, nested DISTINCT, UNION update atomicity, and CTE-backed windows on either JOIN side and chained joins. TTL/nested-operator checks cover construction, not expiration behavior.

Validation run · Demo video

The demo shows actual engine output with its original timing. dozer-sql/tests/DYNAMIC_IN.md explains the support boundaries and how to run it. Retained state is in memory; general compound predicates can require a scan of retained rows when membership changes.

For the bounty payout, PayPal, Wise or USDC work for me. I can provide another compatible wallet if needed; please confirm the payment asset/network.

@CLAassistant

CLAassistant commented Oct 25, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants