Skip to content

Fix v3 filter pushdown#2218

Merged
shangyian merged 4 commits into
DataJunction:mainfrom
brycedesi:investigate/v3-filter-pushdown-bug
Jun 4, 2026
Merged

Fix v3 filter pushdown#2218
shangyian merged 4 commits into
DataJunction:mainfrom
brycedesi:investigate/v3-filter-pushdown-bug

Conversation

@brycedesi
Copy link
Copy Markdown
Contributor

Summary

When filtering on a dimension's label (e.g. is_bot = 'false'), the v3 SQL builder could apply that filter to an upstream table's foreign-key column if it happened to have the same name. That meant comparing a string label against an integer key, which matches no rows and quietly returns empty results.

This fix keeps such filters where they belong after the join (and in the dimension's own table) instead of pushing them onto the wrong column.

Test Plan

Added a regression test, here is the output of the SQL of that test before and after the change:

with the fix:

WITH
v3_lbl_events_xform AS (
SELECT  is_bot,
	amount 
 FROM default.v3.lbl_events
),
v3_lbl_is_bot_dim AS (
SELECT  t.is_bot_key,
	t.is_bot 
 FROM (SELECT  0 AS is_bot_key,
	'false' AS is_bot

UNION ALL
SELECT  1,
	'true') t 
 WHERE  t.is_bot = 'false'
)

SELECT  t2.is_bot,
	SUM(t1.amount) amount_sum_0ca4bf1a 
 FROM v3_lbl_events_xform t1 LEFT OUTER JOIN v3_lbl_is_bot_dim t2 ON t1.is_bot = t2.is_bot_key 
 WHERE  t2.is_bot = 'false' 
 GROUP BY  t2.is_bot

without the fix:

WITH
v3_lbl_events_xform AS (
SELECT  is_bot,
	amount 
 FROM default.v3.lbl_events 
 WHERE  is_bot = 'false'
),
v3_lbl_is_bot_dim AS (
SELECT  t.is_bot_key,
	t.is_bot 
 FROM (SELECT  0 AS is_bot_key,
	'false' AS is_bot

UNION ALL
SELECT  1,
	'true') t 
 WHERE  t.is_bot = 'false'
)

SELECT  t2.is_bot,
	SUM(t1.amount) amount_sum_0ca4bf1a 
 FROM v3_lbl_events_xform t1 LEFT OUTER JOIN v3_lbl_is_bot_dim t2 ON t1.is_bot = t2.is_bot_key 
 WHERE  t2.is_bot = 'false' 
 GROUP BY  t2.is_bot

Notice the WHERE is_bot = 'false' in the first SQL statement.

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

Open to suggestions for other ways to test this or ways to simplify the contribution!

@netlify
Copy link
Copy Markdown

netlify Bot commented Jun 3, 2026

Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit 7453d2a
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6a22091bcd120e0008c0f071

@shangyian shangyian force-pushed the investigate/v3-filter-pushdown-bug branch 2 times, most recently from 38ea2a8 to 8b1afd9 Compare June 4, 2026 08:16
@shangyian shangyian self-requested a review June 4, 2026 22:40
Bryce DeSimone and others added 3 commits June 4, 2026 15:49
@shangyian shangyian force-pushed the investigate/v3-filter-pushdown-bug branch from 68cd43f to ca5d8b4 Compare June 4, 2026 22:52
@shangyian shangyian merged commit 5005097 into DataJunction:main Jun 4, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants