Skip to content

Commit 990cba4

Browse files
committed
fix(kernel): gate max connections kwarg
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
1 parent 3792886 commit 990cba4

3 files changed

Lines changed: 22 additions & 9 deletions

File tree

CONNECTION_PARAMETERS.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,7 @@ to change without notice.
102102
| ------------------------------------ | ----------- | :----: | :----: | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
103103
| `_socket_timeout` | `float` (s) ||| `900` (Thrift); `120` (kernel) | Thrift: socket send/recv/connect timeout. Kernel: total HTTP request deadline from connect through response-body completion. A positive value is forwarded; unset or `0` selects the kernel's 120s default. On the kernel path, `0` is neither unlimited nor an immediate timeout. |
104104
| `_pool_connections` | `int` || ⚠️ | `10` | Number of urllib3 connection pools. Configures the connector's shared Python HTTP client; the kernel's query transport is its own Rust stack. |
105-
| `_pool_maxsize` | `int` ||| `20` (Thrift); `100` (kernel when unset) | Max idle connections retained per host. An explicit value configures both the shared Python HTTP client and the kernel's Rust HTTP pool. |
105+
| `_pool_maxsize` | `int` ||| `20` (Thrift); `100` (kernel when unset) | Max idle connections retained per host. An explicit value configures the shared Python HTTP client and, when supported by the installed kernel wheel, the kernel's Rust HTTP pool. |
106106
| `_proxy_auth_method` | `str` || ⚠️ | `None` | `basic` or `negotiate` (Kerberos). Applies to the shared Python HTTP client; not threaded to the kernel query transport. See [`docs/proxy.md`](docs/proxy.md). |
107107
| `_retry_stop_after_attempts_count` | `int` ||| `30` | Max attempts in a retry sequence. Bounded to `[1, 60]` on Thrift; forwarded to the kernel's retry policy. |
108108
| `_retry_stop_after_attempts_duration`| `float` (s) ||| `900` | Max total wall-clock seconds spent retrying. Forwarded to the kernel. |
@@ -204,7 +204,8 @@ None — the kernel's parameter surface is currently a subset of Thrift's.
204204

205205
- **Connection pooling / proxy**: `_pool_connections` and `_proxy_auth_method`
206206
configure only the shared Python HTTP client. `_pool_maxsize` also configures
207-
the kernel's Rust HTTP pool when explicitly set.
207+
the kernel's Rust HTTP pool when explicitly set and supported by the installed
208+
kernel wheel.
208209
- **`use_inline_params`** renders parameters inline on Thrift; the kernel uses
209210
native parameter binding.
210211

src/databricks/sql/backend/kernel/client.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -156,7 +156,7 @@ def _kernel_session_accepts_kwarg(name: str) -> bool:
156156
``**kwargs`` catch-all), so forwarding a kwarg it doesn't declare raises
157157
``TypeError`` at construction, so we gate kwargs on what the installed
158158
wheel supports. Falls **closed** (returns ``False``) when the signature
159-
can't be introspected because omitting an accepted telemetry kwarg is safer
159+
can't be introspected because omitting an accepted optional kwarg is safer
160160
than forwarding an unsupported one.
161161
"""
162162
try:
@@ -401,7 +401,9 @@ def open_session(
401401
if forwarded:
402402
http_headers_kwargs["http_headers"] = forwarded
403403
pool_kwargs: Dict[str, Any] = {}
404-
if self._max_connections is not None:
404+
if self._max_connections is not None and _kernel_session_accepts_kwarg(
405+
"max_connections"
406+
):
405407
pool_kwargs["max_connections"] = self._max_connections
406408
self._kernel_session = _kernel.Session(
407409
host=self._server_hostname,

tests/unit/test_kernel_client.py

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -368,8 +368,13 @@ def fake_session(**kw):
368368
assert captured["request_timeout_secs"] == timeout
369369

370370

371-
@pytest.mark.parametrize("max_connections", [None, 41])
372-
def test_open_session_passes_max_connections_to_kernel(monkeypatch, max_connections):
371+
@pytest.mark.parametrize(
372+
("max_connections", "kernel_accepts_kwarg", "expected_forwarded"),
373+
[(None, True, False), (41, True, True), (41, False, False)],
374+
)
375+
def test_open_session_gates_max_connections_for_kernel_compatibility(
376+
monkeypatch, max_connections, kernel_accepts_kwarg, expected_forwarded
377+
):
373378
captured = {}
374379

375380
def fake_session(**kw):
@@ -379,6 +384,11 @@ def fake_session(**kw):
379384
return sess
380385

381386
monkeypatch.setattr(kernel_client._kernel, "Session", fake_session)
387+
monkeypatch.setattr(
388+
kernel_client,
389+
"_kernel_session_accepts_kwarg",
390+
lambda name: name == "max_connections" and kernel_accepts_kwarg,
391+
)
382392
c = kernel_client.KernelDatabricksClient(
383393
server_hostname="example.cloud.databricks.com",
384394
http_path="/sql/1.0/warehouses/abc",
@@ -389,10 +399,10 @@ def fake_session(**kw):
389399

390400
c.open_session(session_configuration=None, catalog=None, schema=None)
391401

392-
if max_connections is None:
393-
assert "max_connections" not in captured
394-
else:
402+
if expected_forwarded:
395403
assert captured["max_connections"] == max_connections
404+
else:
405+
assert "max_connections" not in captured
396406

397407

398408
def test_open_session_passes_phase_7_telemetry_kwargs_to_kernel(monkeypatch):

0 commit comments

Comments
 (0)