Fix !specify first-call failure for lazily-discovered sink schemas - #246
Merged
Conversation
CREATE OR REPLACE MATERIALIZED VIEW via !specify (dry-run) failed with "Table <name> not found in schema <schema>" on the first invocation and only succeeded on an identical second call, for sinks whose schema is discovered lazily (e.g. OpenHouse namespaces). Root cause: DdlMode.SPECIFY navigated the schema via context.getRootSchema(), a per-statement snapshot, and registered the temporary sink view there. But sink resolution (HoptimatorDriver.rowType) reads the live connection root. On a cold cache the snapshot and live roots resolve to different CalciteSchema wrappers, so the temp view was invisible to the read. The failed first call warmed Calcite's caches enough to align the two roots on the second call. Fix: all DDL modes now register against the live (mutable) root schema, so the temporary view is resolvable on the first call. Since that made DdlMode.mutable() always true, it is removed and replaced with a single-purpose dryRun() flag that drives dry-run rollback: SPECIFY restores the temporary view and any deployer side effects; CREATE/UPDATE keep their mutations on success. This keeps !specify side-effect-free while fixing first-call resolution. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Code Coverage
|
ryannedolan
approved these changes
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
CREATE OR REPLACE MATERIALIZED VIEW ... AS SELECT ...via!specify(dry-run)failed with
Table <name> not found in schema <schema>on the firstinvocation, then succeeded on an identical second call. This made specifying
a pipeline into a lazily-discovered sink non-deterministic.
Root cause
DdlMode.SPECIFY.mutable()returnedfalse, soprocessCreateMaterializedView/processCreateTablenavigated the schema viacontext.getRootSchema()— aper-statement snapshot of the root — and registered the temporary sink view
there. But sink resolution (
HoptimatorDriver.rowType) reads the liveconnection root (
connection.calciteConnection().getRootSchema()).For sinks whose sub-schema is implicit/lazy (not pre-enumerated), the snapshot
and live roots resolve to different
CalciteSchemawrappers with different tablemaps on a cold cache, so the view added to the snapshot was invisible to the
live-root read. The failed first call warmed Calcite's schema caches enough that
the two roots aligned on the second.
Fix
so the temporary view is resolvable by sink resolution on the first call.
DdlMode.mutable()alwaystrue(dead), so it's removed andreplaced with a single-purpose
dryRun()flag (trueforSPECIFY,falseforCREATE/UPDATE) that drives rollback:SPECIFYrestores thetemporary view and any deployer side effects;
CREATE/UPDATEkeep theirmutations on success.
This keeps
!specifyside-effect-free while fixing first-call resolution.Testing
HoptimatorDdlUtilsTestpasses; the...MutableReturns...tests are replacedwith
...DryRun...equivalents (SPECIFY.dryRun() == true,CREATE/UPDATE.dryRun() == false).