[BUG] Performance regression: JDBC request latency grows with repeated connections #1719

Summary

JDBC requests become progressively slower in a long-running JVM that repeatedly opens and closes physical connections, even for a trivial, unchanged query such as SELECT 1. Investigation reproduces growing client-side overhead and lock contention tied to connection history.

The cause is duplicate registrations in the Java SDK's JVM-global User-Agent state. This report tracks the performance problem addressed by JDBC PR #1714 and Java SDK PR #939.

Root Cause And Impact

JDBC's UserAgentManager.setUserAgent() registers the client marker and optional UserAgentEntry when connections are opened. The affected SDK appends every registration to a static list, including identical key/value pairs. Closing a connection does not remove them.

UserAgent.asString() formats the entire accumulated list under a shared lock whenever it builds a header. It removes duplicates only from the rendered output, after doing the work. Consequently, the outgoing header can remain short while CPU cost, allocations, and lock contention grow with connection history. This affects header construction for both Thrift and SQL Execution API requests, and can increase connect, execute, and close latency.

Reproduced locally with JDBC 3.4.3 main source (a64ae0e0) plus SDK 0.118.0. The same growth reproduces in the SDK source immediately before PR #939 (c6973a2a, version 0.161.0). These are verified examples, not an exhaustive affected-version range; the first affected release has not been established.

Reproduction

1. Keep one JVM running with an affected driver.
2. Repeatedly open a physical connection, execute SELECT 1, and close the connection. Keep the SQL, warehouse, credentials, and concurrency unchanged; 16 concurrent workers amplify contention.
3. Compare early and late connect/execute/close latency, grouped by cumulative connection count. Capture JFR with -XX:StartFlightRecording=settings=profile.
4. Look for increasing CPU samples in com.databricks.sdk.core.UserAgent.asString and monitor waits on its shared otherInfo list.

A custom UserAgentEntry is not required: the client marker also accumulates. A stable custom entry adds another repeated segment.

Current Unfixed Measurements: October 6, 2026

A source-level JMH 1.37 benchmark repeatedly called the actual UserAgentManager.setUserAgent() with a local connection-context stub returning THttpClient and PerfRepro/1.0, then measured getUserAgentString(). The benchmark ran locally without SQL warehouse access or authentication.

Each trial started in a fresh JVM. These counts are registration calls, not real connections; the setup and registration work were outside the timed section.

Despite the growing internal list, the rendered header remained 114 characters and contained both expected segments.

With 16 concurrent header-building threads after 40,000 registrations, average time per call was 557 +/- 60 ms.

Environment: Linux x86_64, Intel Xeon 6975P-C, OpenJDK 11.0.32. JMH used 2 JVM forks, 2 x 1-second warmup iterations, 3 x 1-second measurement iterations, and a 512 MiB heap. Values are mean time per header-building call with JMH's 99.9% confidence interval (us = microseconds), not SQL latency or p50; concurrent values include contention. Absolute timings depend on the host. This isolates the growing client-side cost and does not predict end-to-end query speedup.

Measured source: JDBC a64ae0e0 with SDK 0.118.0.

Expected Behavior

Repeating the same User-Agent registration should not increase retained state or header-building cost with JVM lifetime.

Proposed Fix Verification

Run the following benchmarks on a candidate build containing the fix, alongside the unfixed baseline under the same conditions:

1. Header-construction benchmark: In fresh JVMs, repeat the same registrations 1, 1,000, 10,000, and 40,000 times, then measure getUserAgentString() with 1 and 16 threads using the JMH settings above. Cover both THttpClient and SQLExecHttpClient, with and without a stable custom UserAgentEntry. Record stored-entry count and time per call. Verify that identical registrations do not grow the retained state, header cost stays approximately flat across history sizes, and expected header segments remain present.
2. Connection-churn benchmark: Run 16 workers looping connect -> SELECT 1 -> close for at least 2 minutes in one JVM. Every 10 seconds, record cumulative connections, connect/execute/close latency, stored-entry count, and header-build time. Use JFR to inspect CPU samples and monitor waits in UserAgent.asString. Verify that connection history no longer produces rising header overhead or accumulating lock contention; report warehouse and network variability separately.

Related: issue #1710 tracks making JDBC User-Agent attribution connection-scoped.