fix(adapters): fix exception mapping, transactions, and settings consistency across adapters

docs/changelog.md

@@ -11,6 +11,8 @@ important operational fixes.
 
 **Added:**
 
+* Added a pool for the native PostgreSQL ADBC driver. Set `AdbcConfig(pool_config={})` to reuse connections. Each lease has a wait timeout; each return clears session state. Pooling is off by default. See the [ADBC guide](databases/adbc.md#postgresql-connection-reuse) for supported settings.
+
 * Added Sybase Adaptive Server Enterprise (`sqlspec.adapters.sybase` with `SybaseSyncConfig` and `SybaseSyncDriver`) and SAP SQL Anywhere (`sqlspec.adapters.sqlanywhere` with `SqlAnywhereSyncConfig` and `SqlAnywhereSyncDriver`) database adapters, custom SQLGlot dialects (`Sybase` and `SQLAnywhere` in `sqlspec.dialects.sybase`), data dictionaries (`SybaseSyncDataDictionary` and `SqlAnywhereSyncDataDictionary` with `SYBASE_CONFIG` and `SQLANYWHERE_CONFIG`), type converters (`SybaseOutputConverter` and `SqlAnywhereOutputConverter`), and extension stores for Litestar sessions (`SybaseStore`, `SqlAnywhereStore`), Event queues (`SybaseEventQueueStore`, `SqlAnywhereEventQueueStore`), and Google ADK sessions and memory (`SybaseADKStore`, `SybaseADKMemoryStore`, `SqlAnywhereADKStore`, `SqlAnywhereADKMemoryStore`).
   ([#870](https://github.com/litestar-org/sqlspec/pull/870))
 * Added `arrow_odbc` dialect auto-detection and `information_schema` / system catalog introspection for Sybase ASE (`Adaptive Server Enterprise` / `ASE`) and SAP SQL Anywhere (`SQL Anywhere` / `Sybase IQ`), routing detected connections to the `sybase` and `sqlanywhere` data dictionaries.
@@ -25,24 +27,85 @@ important operational fixes.
 
 **Changed:**
 
-* Replaced correlated per-column scalar subqueries in data dictionary `columns.sql` and `indexes.sql` catalog queries across `postgres`, `cockroach`, `mssql`, `mysql`, `oracle`, `db2`, and `duckdb` with pre-aggregated CTEs and joins, and reduced `ColumnMetadata` to its 18 core catalog fields.
+
+* Renamed public Litestar stores: `SQLiteStore` to `SqliteStore`, `ADBCStore` to `AdbcStore`, and `DuckdbStore` to `DuckDBStore`. Update imports and store construction accordingly. DuckDB ADK exports likewise use `DuckDBADKStore`, `DuckDBADKMemoryStore`, `DuckDBADKConfig`, and `DuckDBADKFTSOptions` instead of their `Duckdb` spellings.
+
+* Removed `SybaseSyncEventQueueStore` and `SqlanywhereSyncEventQueueStore`; use `SybaseEventQueueStore` and `SqlanywhereEventQueueStore` respectively.
+
+* Standardized database failures as SQLSpec exceptions across statement execution, cached queries, bulk loads, and transaction boundaries. Code catching native driver errors should catch the corresponding `sqlspec.exceptions` classes instead. Constraint violations retain specific classes such as `UniqueViolationError` and `ForeignKeyViolationError`; syntax errors raise `SQLParsingError`. MySQL errors 1061 and 1091 now raise `OperationalError` instead of being suppressed. Original driver exceptions remain available as causes when the driver supplies an exception object.
+
+* Changed default SQLite `:memory:` pools to share one database across connections within that pool. Shared cache does not enable dirty reads by default; applications that deliberately need dirty reads must explicitly configure `read_uncommitted`.
+
+* Successful Psycopg `on_connection_create` hooks now commit their setup transaction before a connection enters the pool. Session settings such as `search_path` and timezone persist, and failed setup is rolled back. Hooks must not rely on their writes remaining uncommitted.
+
+* ADK table-name validation now rejects schema or dataset qualifiers where the selected adapter cannot generate valid qualified DDL. Supported schema-qualified stores retain that capability.
+
+* ADK schema changes target newly initialized stores. Existing ADK tables must be reset and recreated for this release; resetting removes their stored sessions and memories. Upgrade migrations for these schema changes are deferred.
+
+* Removed legacy `pgbouncer` and `pgbouncer_mode` flags from PostgreSQL driver features across `asyncpg`, `psycopg`, and `psqlpy` in favor of canonical `pooler_mode="transaction"`.
+
+* Removed the `pgbouncer` key from `AsyncpgConnectionConfig`; set `driver_features={"pooler_mode": "transaction"}` instead.
+
+* Removed the empty `SybaseADKConfig`, `SqlanywhereADKConfig`, `Db2LitestarConfig`, and `Db2EventsConfig` TypedDicts; use the base `ADKConfig`, `LitestarConfig`, and `EventsConfig`.
+
+* Renamed `use_pool` to `enable_pooling` (default `True`) across `mssql_python` and `pymssql` driver features.
+
+* Renamed `arrow_odbc` `enable_driver_pooling` to `enable_pooling` and BigQuery `use_query_and_wait` to `enable_query_and_wait`.
+
+* Removed the ADBC `strict_type_coercion` and `arrow_extension_types` driver feature aliases; use `enable_strict_type_coercion` and `enable_arrow_extension_types`.
+
+* Declared driver feature keys that adapters already read: `events_backend` on `ArrowOdbcDriverFeatures` and `default_query_job_config` on `BigQueryDriverFeatures`.
+
+* Replaced `null_pool` in `psycopg` connection configuration and driver features with canonical `enable_pooling=False`.
+
+* Defaulted `enable_correlation_middleware` to `True` across all web framework extensions (`Litestar`, `FastAPI`, `Starlette`, `Sanic`, `Flask`).
+
+* Standardized adapter driver imports: importing `sqlspec.adapters.<name>` eagerly imports that adapter's driver, raising the native `ModuleNotFoundError` when the driver is not installed, while non-driver optional dependencies load on first use.
+
+* Documented Oracle AQ dequeue commit behavior as at-most-once delivery.
+
+* Added SAP SQL Anywhere integration driver test suite with hermetic in-container client runtime extraction.
+
+* Replaced correlated per-column scalar subqueries in data dictionary `columns.sql` and `indexes.sql` catalog queries across `postgres`, `cockroach`, `mysql`, `oracle`, `db2`, and `duckdb` with pre-aggregated CTEs and joins, and reduced `ColumnMetadata` to its 18 core catalog fields.
   ([#866](https://github.com/litestar-org/sqlspec/pull/866))
+
 * Redesigned the documentation site architecture around Zensical and `mkdocstrings-python` with Diataxis-aligned navigation (`Getting Started`, `Learn`, `Adapters`, `Extensions`, `Examples`, `Reference`), interactive WebAssembly SQL playground, runnable `docs/examples/` snippets, conceptual Mermaid diagrams, and tabbed package manager and `SQLFileLoader` examples.
   ([#864](https://github.com/litestar-org/sqlspec/pull/864),
   [#867](https://github.com/litestar-org/sqlspec/pull/867),
   [#869](https://github.com/litestar-org/sqlspec/pull/869))
+
 * Migrated integration test infrastructure to clientless `pytest-databases` fixtures (`#859`).
   ([#859](https://github.com/litestar-org/sqlspec/pull/859))
+
 * Upgraded CI GitHub Actions `astral-sh/setup-uv` from `10.3.1` to `10.3.3` and pinned `actions/checkout` (`6.0.2`), `actions/setup-python` (`7.1.0`), `actions/cache` (`6.0.0`), and `actions/upload-pages-artifact` / `actions/deploy-pages` in `.github/workflows/ci.yml` and `.github/workflows/docs.yml`.
   ([#872](https://github.com/litestar-org/sqlspec/pull/872))
 
 **Fixed:**
 
+
+* ADBC now closes its own connections if a hook, profile setup, driver setup, or session setup fails. It keeps the first error and leaves supplied connections open.
+
+* Explicit ADBC transactions now honor native `autocommit=True` on supported drivers. If rollback cleanup fails, autocommit stays off so mode restoration cannot commit pending writes.
+
+* Preserved supported Arrow conversion and mutation paths for Asyncmy and Spanner while reporting their native capabilities accurately; corrected ADBC lowercase readers, DuckDB stream ownership and table filtering, and SQL Server Arrow cursor lifetimes.
+
+* Corrected Oracle session-renewal bindings, pipeline error conversion, ADK batch-error handling, standalone connection setup, and deferred Arrow failures. Text-only Oracle memory stores remain available on servers without native vector support.
+
+* Preserved PostgreSQL cache-hit query semantics, affected-row counts, mapped exceptions, JSON scalar values, and ADK embeddings. Psycopg connection hooks leave pooled connections idle, COPY finalization errors are translated, and Asyncpg continue-on-error stacks recover through savepoints inside caller transactions.
+
+* Preserved SQL Server 2016 constraint and index metadata queries with compatible catalog aggregation.
+
+* Corrected SQL Anywhere implicit-transaction cleanup, Db2 ADK index-name matching and event deduplication, MySQL batch percent escaping, optional Connector/Python async-pool imports, privilege filters, and mutable DDL capture settings.
+
 * Removed runtime `duckdb` imports from `sqlspec.adapters.adbc` (`AdbcDataDictionary` and `sqlspec/adapters/adbc/core.py`) and enforced strict adapter import isolation across all 20 database adapters in `tests/unit/test_import_time.py`.
   ([#865](https://github.com/litestar-org/sqlspec/pull/865))
+
 * Eliminated extra row copies and `UserWarning` in `DuckDBDriver.select_to_arrow` on DuckDB 2.0+ by routing parameterless `SELECT` queries through `connection.sql(query).to_arrow_table()` and `to_arrow_reader(batch_size=...)` when no open DB-API transaction is active on the connection.
+
 * Fixed `compile_table_ddl_results` and `CreateIndex` cross-dialect DDL synthesis to emit `COMMENT ON` only on dialects that support it (`postgres`, `oracle`, `db2`, `duckdb`, `snowflake`), inline `COMMENT` clauses on `mysql` and `mariadb`, render `STORING (...)` on `spanner` and `spangres`, fold unquoted source identifiers to target case before quoting target-only reserved keywords, and preserve SQLite `NUMERIC` type affinity.
+
 * Fixed `MigrationSquasher.fold_canonical_ddl` to fall back cleanly to raw statement concatenation when a migration contains interleaved DML or procedural blocks (`INSERT`, `UPDATE`, `DELETE`, `MERGE`, `COPY`, `DO`, `CALL`), preserve table-level constraints across `ALTER TABLE ... RENAME COLUMN` and `DROP COLUMN`, and update `sqlspec_migrations` tracker history atomically inside a single transaction when applying squashed migrations.
+
 * Fixed `write_fixture_shards` to unlink stale shard and legacy single-file artifacts (`<table>.json`, `<table>.json.gz`, and higher-numbered shards) before writing new shards so subsequent `load_fixture` calls do not read orphaned records.
 
 ### v0.69.0 - Oracle PKCS#12 wallet conversion, TNS alias resolution, proximity RRF, and server-side DBMS_CLOUD ingestion
@@ -260,7 +323,7 @@ important operational fixes.
   savepoints, SQLSTATE/vendor exception mapping, and default
   `enable_lowercase_column_names=True` normalization for Oracle and Db2.
   ([#839](https://github.com/litestar-org/sqlspec/pull/839))
-* `ADBCStore` (Litestar sessions), `AdbcEventQueueStore` (event channels),
+* `AdbcStore` (Litestar sessions), `AdbcEventQueueStore` (event channels),
   `AdbcADKStore`, and `AdbcADKMemoryStore` (Google ADK sessions and memory)
   add Oracle (`oracle`) and IBM Db2 (`db2`) backend support with idempotent
   DDL guards (PL/SQL `SQLCODE` blocks for Oracle and `SYSCAT` catalog probes

docs/contributing/creating-adapters.md

@@ -41,6 +41,18 @@ sqlspec/adapters/<name>/
 - **`driver.py`**: Subclasses `SyncDriverAdapterBase` or `AsyncDriverAdapterBase`, implementing `dispatch_execute`, `dispatch_execute_many`, `dispatch_execute_script`, `begin`, `commit`, `rollback`, `with_cursor`, `handle_database_exceptions`, `_connection_in_transaction`, and `data_dictionary`.
 - **`data_dictionary.py`**: Subclasses `SyncDataDictionaryBase` or `AsyncDataDictionaryBase`, binding the dialect's SQL query pack from `sqlspec/data_dictionary/dialects/<dialect>/` to introspect tables, columns, indexes, foreign keys, and server versions.
 
+## Import Contract
+
+Every driver is optional: users install only the extras for the adapters they use, and any one adapter (or all of them) works without the other drivers installed. Each adapter follows the same import rules:
+
+- **Own driver, imported eagerly.** `import sqlspec.adapters.<name>` imports that adapter's driver once, at the top of `_typing.py`, as aliased re-exports (`from asyncpg import Connection as AsyncpgRawConnection`). A missing driver raises the driver's own `ModuleNotFoundError`. Do not wrap the adapter's own driver in `import_optional`, `import_optional_attr`, `importlib.import_module`, `try`/`except ImportError` fallbacks, `None` placeholders, stub classes, or a module `__getattr__`.
+- **No other drivers.** An adapter never imports another adapter's driver or package. The CockroachDB adapters wrapping `asyncpg` and `psycopg` are the only allowed cross-adapter imports.
+- **One source of driver names.** `config.py`, `core.py`, `driver.py`, `pool.py`, and the extension stores import driver names from `_typing`, not from the driver package.
+- **One `TYPE_CHECKING` block.** `_typing.py` uses a single `if TYPE_CHECKING: ... else: ...` block, only for subscripted generics (`Pool[Record]`), `Protocol` types for compiled driver classes, sqlspec imports that would be circular, and secondary optional types.
+- **Secondary dependencies through the shared shim.** When code cannot avoid referencing an optional dependency at import time (a runtime-evaluated annotation, an `isinstance` target, or an availability check), import the type or flag from `sqlspec.typing` (for example `ArrowTable`, `NumpyArray`, `PYARROW_INSTALLED`, `PGVECTOR_INSTALLED`). `sqlspec/_typing.py` resolves these lazily and supplies protocol fallbacks when the package is absent. If the shim lacks a type or flag you need, add it there instead of writing a local fallback. When the module itself is needed (`pyarrow`, `pandas`, `polars`, `numpy`, `pgvector`, `rich`, cloud connectors, storage backends, observability packages), load it inside the function that uses it (`ensure_*` when required, `import_optional` when there is a fallback), never at module import.
+
+Register a new adapter's driver module(s) in `ADAPTER_DRIVERS` in `tests/unit/test_import_time.py`. The import-contract tests fail for unregistered adapters and check that the adapter raises the driver's own error when the driver is missing, imports with every other driver blocked, and loads no secondary optional module at import. If the driver package itself imports an optional module, list it in `DRIVER_OWNED_OPTIONAL_IMPORTS` with the reason. For development, add the driver to the `extras` dependency group in `pyproject.toml`; dependency groups are not published to PyPI.
+
 ## The `apply_driver_features` Contract
 
 Each adapter's `core.py` normalizes user-supplied `driver_features` and applies custom JSON serializers or type converters to the adapter's `StatementConfig`:

docs/databases/adbc.md

@@ -118,6 +118,66 @@ with spec.provide_session(config) as session:
     )
 ```
 
+## PostgreSQL Connection Reuse
+
+ADBC opens and closes a connection for each session by default. For short
+PostgreSQL transactions, enable connection reuse with `pool_config={}` or supply
+explicit bounds:
+
+```python
+import os
+
+from sqlspec.adapters.adbc import AdbcConfig
+
+config = AdbcConfig(
+    connection_config={"uri": os.environ["DATABASE_URI"]},
+    pool_config={"max_size": 4, "timeout": 30.0},
+)
+try:
+    with config.provide_session() as session:
+        value = session.select_value("SELECT :value", value=42)
+        session.commit()
+finally:
+    config.close_pool()
+```
+
+`pool_config=None` keeps a fresh connection per session. An empty mapping gives
+four leases and a 30-second wait limit. Each lease gives one caller sole use of a
+connection. The pool holds one extra seed, for a total of `max_size + 1` native
+connections. Nested sessions need separate leases and can time out when the pool
+is full. Standalone connections do not count toward this limit.
+
+Both native packages, `adbc-driver-manager` and `adbc-driver-postgresql`, must be
+version 1.12.0 or newer. This pool supports only the native PostgreSQL driver
+with `autocommit=False`. It rejects supplied connections and
+`on_connection_create` hooks. Other backends keep a fresh connection per
+session. The `supports_connection_pooling` flag means the adapter offers a
+pool; the flag does not say whether a given config has enabled it.
+
+On clean exit, the pool rolls back unfinished work, runs `DISCARD ALL` outside
+a transaction, and turns native autocommit off. This clears committed session
+settings, temporary tables, prepared statements, listeners, and advisory locks.
+A failed session or reset drops that connection from reuse. Direct changes to
+native connection options fall outside this reset contract. Measure complete
+transactions, including reset, on your own workload before choosing a pool.
+
+Lazy row streams and Arrow readers belong to their session. On exit, the
+session closes their native readers and cursors, then drops the connection
+from reuse. This also applies to streams that were never read or fully read.
+Consume streams inside the session. Materialized tables and eagerly drained
+batch lists stay valid afterward and allow reuse.
+
+`close_pool()` blocks new checkouts and closes idle connections. Active leases
+finish first, then close; their work is not interrupted. If native cleanup
+fails, call `close_pool()` again to retry. The pool keeps ownership until every
+handle is closed. A later explicit session can then start a new generation.
+Create configs after forking: a pool from the parent process refuses use in a
+child. Pools do not refresh credentials on their own.
+
+`create_connection()` and `provide_connection()` remain separate from the pool.
+An unpooled `connection_instance` stays open when a session ends; the caller
+owns it.
+
 ## Key Driver Features and Pool Tuning
 
 | Setting | Location | Description |

docs/databases/arrow-odbc.md

@@ -50,7 +50,7 @@ config = spec.add_config(
         },
         statement_config=StatementConfig(dialect="tsql"),
         driver_features={
-            "enable_driver_pooling": True,
+            "enable_pooling": True,
             "chunk_size": 65536,
             "fetch_concurrently": True,
         },
@@ -60,13 +60,13 @@ config = spec.add_config(
 
 ## Key Driver Features and Pool Tuning
 
-`ArrowOdbcConfig` subclasses `NoPoolSyncConfig` and delegates connection reuse to ODBC Driver Manager pooling when `enable_driver_pooling=True`:
+`ArrowOdbcConfig` subclasses `NoPoolSyncConfig` and delegates connection reuse to ODBC Driver Manager pooling when `enable_pooling=True`:
 
 | Setting | Location | Description |
 | --- | --- | --- |
 | `connection_string` | `connection_config` | Standard semicolon-delimited ODBC connection string (`Driver={...};Server=...;`). |
 | `user` / `password` | `connection_config` | Optional credentials passed alongside the ODBC connection string. |
-| `enable_driver_pooling` | `driver_features` | Enables process-wide ODBC Driver Manager connection pooling for low-latency session checkouts. |
+| `enable_pooling` | `driver_features` | Enables process-wide ODBC Driver Manager connection pooling for low-latency session checkouts. |
 | `chunk_size` / `max_bytes_per_batch` | `driver_features` | Maximum rows and memory bytes allocated per Arrow `RecordBatch` buffer during streaming reads. |
 | `fetch_concurrently` | `driver_features` | Allocates double-buffered background ODBC fetch threads in Rust while Python processes the current Arrow batch. |
 | `max_text_size` / `max_binary_size` | `driver_features` | Upper byte bounds for unbounded `VARCHAR(MAX)` / `NVARCHAR(MAX)` / `VARBINARY(MAX)` columns. |