Repository navigation
fix(databricks): restore the dynamic connection form - #45911
Open
Vitor-Avila wants to merge 3 commits into
Open
Vitor-Avila wants to merge 3 commits into
Vitor-Avila wants to merge 3 commits into
Conversation
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![codeant-ai-for-open-source[bot]](https://avatars.githubusercontent.com/u/151810221?s=60&v=4){kind=small}
| engine_name = "Databricks" | ||
| default_driver = "databricks-sql-python" | ||
| drivers = {"databricks-sql-python": "Databricks SQL Python"} | ||
| default_driver = "databricks" |
Contributor
There was a problem hiding this comment.
Suggestion: If only a legacy Databricks driver is installed, the API omits this spec’s parameters but still selects this default; URI mode then uses the unavailable Python dialect and connections fail.
Assessment: 🟠 Major · 🔁 Occurrence: Sometimes · 🏷️ Api mismatch
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** superset/db_engine_specs/databricks.py
**Line:** 745:745
**Comment:**
*Api Mismatch: If only a legacy Databricks driver is installed, the API omits this spec’s parameters but still selects this default; URI mode then uses the unavailable Python dialect and connections fail.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
Contributor
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #45911 +/- ##
===========================================
+ Coverage 65.62% 84.76% +19.13%
===========================================
Files 3071 3071
Lines 206037 206037
Branches 48506 48506
===========================================
+ Hits 135211 174637 +39426
+ Misses 67735 28101 -39634
- Partials 3091 3299 +208
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
mikebridge
reviewed
Oct 31, 2026
mikebridge
left a comment
mikebridge
left a comment
Contributor
There was a problem hiding this comment.
Thanks Vitor, this looks right to me. build_sqlalchemy_uri still writes a bare databricks:// URI, so stored connection strings don't change, and existing connections now resolve to this spec by exact driver match instead of through the fallback.
A few small things you might fold in before merge:
- A test for the fix itself. Nothing pins the original bug: a unit test where the available drivers include
databricksfor thedatabricksbackend, checking that the Databricks entry comes back withdefault_driver == "databricks"and non-emptyparameters, would stop the mismatch from returning. tests/unit_tests/db_engine_specs/test_databricks.py:1410assertsget_engine_spec("databricks", "databricks-sql-python")under the "exact driver matches" comment. After this change that string only resolves through the fallback, so it still passes but no longer tests an exact match. Switching it to"databricks"keeps the intent; the old string could move up to the fallback assertion, since legacy URIs may still carry it.docs/src/data/databases.jsonstill listsdefault_driver: "databricks-sql-python". Regenerating it (leaving the 6.1.0 versioned copy alone) keeps the docs in step; fine as a follow-up.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
The official
databricks-sqlalchemydialect reports its driver asdatabricks, but Superset advertisesdatabricks-sql-python. The available-databases API only includes dynamic-form parameters when the spec's default driver is installed, so this mismatch omits the form schema.Update the Python connector's default driver and driver mapping to
databricks. This PR is limited to restoring the connection form; legacy-driver removal and connection migration are follow-up work.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before: with the official dialect installed, the Databricks connection form can appear blank because its parameters are missing from the API response.
After: the registered driver matches the spec, allowing the API to include the dynamic-form parameters.
TESTING INSTRUCTIONS
databricks-sqlalchemy2.x./api/v1/database/available/should includedefault_driver: "databricks"andparameters.Local validation: pre-commit passed; 61 focused Databricks, multi-cloud, and engine-discovery unit tests passed. OAuth and installed-dialect SQL-rendering tests were excluded because unrelated failures were also reproduced on the unchanged base. Live UI validation was not performed because the local server was not running.
ADDITIONAL INFORMATION