Embucket vs Snowflake parity harness for dbt-snowplow-web - #6

Open

rampage644 wants to merge 30 commits into

mainfrom

rampage644/real-snowplow-source

Commits on Apr 17, 2026

docs: design for loading real Snowplow source via Athena CTAS into S3 Tables

rampage644
and
claude
committed
02f54d4
feat: load real Snowplow source via Athena CTAS, wire dbt to events_0416

rampage644
and
claude
committed
58c57fd

Commits on Apr 22, 2026

docs: design for Embucket vs Snowflake dbt-snowplow-web parity harness
rampage644
committed
f4edfec
docs: implementation plan for Embucket vs Snowflake parity harness
rampage644
committed
aa3ea43
chore: bump snowplow__start_date to 2026-04-22 for parity batches
rampage644
committed
7b4f15b
build: add dbt-snowflake and snowflake-connector-python deps
rampage644
committed
7bb2d49
refactor: extract events_0416 projection SELECT into reusable file
rampage644
committed
572ba1a
refactor: split load_from_glue into init/insert subcommands
rampage644
committed
7629f33
chore: add snowflake output template to profiles.yml.example
rampage644
committed
ca9220b
feat: add parity.py with MD5 diff across Embucket and Snowflake
rampage644
committed
2f2c7dd
feat: add snowflake_setup.py + fix db name to sturukin_db
rampage644
committed
b2a0e5b
fix: use sturukin_db (not sturukin) in parity.py
rampage644
committed
36fbaca
feat: add snowflake_refresh.py -- ALTER ICEBERG TABLE REFRESH wrapper
rampage644
committed
6d3c4de
docs: README section for Embucket vs Snowflake parity flow
rampage644
committed
29241c4
fix: add variant view for Snowflake + graceful rowcount-mismatch handling
rampage644
committed
cd2cf26
docs: parity harness run results — source parity holds, derived tables diverge
rampage644
committed
8b3f734
docs: update parity results -- Embucket MERGE ignores unique_key on dup source
rampage644
committed
5c12048
docs: correct parity findings -- dup source is the root cause, not Embucket MERGE
rampage644
committed
c1afbe6
feat(parity): server-side aggregate parity + document correctness findings
rampage644
committed
961a73f
docs: record Embucket OOM on snowplow_web_base_events_this_run
rampage644
committed
eaecb27
docs: 1/10 source scale results -- tight parity, isolate users.referrer tie-break
rampage644
committed
f2f4f20
docs: batch-2 MERGE finding -- Embucket loses cross-batch user state
rampage644
committed
28bd39d
docs: localize batch-2 divergence to Embucket MERGE UPDATE execution
rampage644
committed
941d33e
docs: consolidated investigation report
rampage644
committed
fc477ff
docs: root-cause Finding 4 -- alias-shadow SQL scoping bug in Embucket
rampage644
committed
4f30397
docs: confirm absolute_time_in_s hypothesis with refined DATEDIFF formula
rampage644
committed
7b54daa

Commits on Apr 24, 2026

test: verify OOM-mitigation dbt patch on Snowflake; NOT semantically equivalent

rampage644
and
claude
committed
702002f
test: patch now byte-equivalent to upstream on Snowflake (fr + inc)

rampage644
and
claude
committed
0fd1d13
test: iteration-2 patch OOMs on Embucket at base_events; log finding + harness

rampage644
and
claude
committed
36cf1dc
feat: three-model base-events decomposition - patch now runs on Embucket

rampage644
and
claude
committed
80b9be5