Skip to content

fix(duckdb): partition initial DuckLake writes - #5942

Open
anxkhn wants to merge 1 commit into
SQLMesh:mainfrom
anxkhn:fix/ducklake-initial-partitioning
Open

anxkhn wants to merge 1 commit into
SQLMesh:mainfrom
anxkhn:fix/ducklake-initial-partitioning

Conversation

@anxkhn

@anxkhn anxkhn commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Description

Fixes #5742.

For partitioned DuckLake tables created from a query, create the table schema without writing data, apply the partition specification, and then insert the initial rows. This ensures the first DuckLake data files are physically partitioned, including for unannotated SQL models and fully qualified DuckLake targets.

Test Plan

  • uv run make style
  • uv run pytest -o addopts='' tests/core/engine_adapter/test_duckdb.py -q (13 passed)
  • uv run make fast-test (2582 passed, 4 skipped; 6 unrelated MSSQL/Fabric tests failed because local pyodbc cannot load the missing unixODBC library)

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

@StuffbyYuki
StuffbyYuki force-pushed the fix/ducklake-initial-partitioning branch from e98bc49 to ab7d72b Compare October 2, 2026 07:00
@StuffbyYuki
StuffbyYuki self-requested a review October 2, 2026 07:00
@StuffbyYuki

Copy link
Copy Markdown
Collaborator

@anxkhn Hey sorry for the late reply here. Could you look into resolving the conflict?

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn
anxkhn force-pushed the fix/ducklake-initial-partitioning branch from ab7d72b to 5af58f8 Compare October 7, 2026 08:39
@anxkhn

anxkhn commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

hey, i've resolved the merge conflicts. could you please review again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DuckLake: partitioned_by on INCREMENTAL_BY_UNIQUE_KEY results in unpartitioned files on initial build

2 participants