Skip to content

fix: push dimension filters into nested scopes when the CTE doesn't project the filtered column#2219

Merged
shangyian merged 3 commits into
DataJunction:mainfrom
shangyian:fix-filtering
Jun 4, 2026
Merged

fix: push dimension filters into nested scopes when the CTE doesn't project the filtered column#2219
shangyian merged 3 commits into
DataJunction:mainfrom
shangyian:fix-filtering

Conversation

@shangyian
Copy link
Copy Markdown
Collaborator

@shangyian shangyian commented Jun 4, 2026

Summary

In _resolve_pushdown_filters_for_cte, when the primary rewrite failed (the filter column was absent from the CTE's declared output), the loop would continue, skipping the second pass entirely. The second pass was column-aware retargeting via scope_column_aliases and resolves a filter into a nested scope even when the outer CTE doesn't project the column. However, since it was gated on primary success, it never got a chance to run.

The pattern this missed: a transform that joins a dimension to compute a derived value but aggregates the dimension's key column away and never projects it.

The fix is to restructure the loop so the column-aware second pass always runs.

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

@netlify
Copy link
Copy Markdown

netlify Bot commented Jun 4, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit 92fbf79
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6a2126c26a08090007dcdc2b

@shangyian shangyian changed the title Fix filter pushdown onto column that was not selected as part of the … fix: push dimension filters into nested scopes when the CTE doesn't project the filtered column Jun 4, 2026
@shangyian shangyian marked this pull request as ready for review June 4, 2026 07:48
@shangyian shangyian merged commit ba47f5d into DataJunction:main Jun 4, 2026
21 checks passed
@shangyian shangyian deleted the fix-filtering branch June 4, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant